Browsing by Subject "Multimodality"

Now showing 1 - 4 of 4
  • Thumbnail Image
    Item
    Approaching Composition as Showing–Telling through Translanguaging: Weaving Multilingualism, Multimodality, and Multiliteracies in a Digital Collage Proyecto Final
    (MDPI, 2022) Prada, Josh; World Languages and Cultures, School of Liberal Arts
    Couched in theories of translanguaging, multimodality, and multiliteracies, this article explores digital compositions (i.e., digital collages) as spaces for identity representation through the proyectos finales produced by 22 students in a Spanish composition class for heritage/native speakers in a U.S. university. Each digital collage was accompanied by two written documents: one describing the processes leading to its creation, and another one explaining the meaning of the collage and its components. Qualitative content analysis was used to investigate the submissions, with particular attention paid to instances of identity, experience, and self-representation through complex orchestrations of flexible multilingual and multimodal meaning- and sense-making. The proyecto final is discussed in terms of the curricular innovation for courses designed for racialized language-minoritized multilingual students, describing the nature and affordances of translanguaging in this context, and advancing an approach to digital composing as showing–telling.
  • Thumbnail Image
    Item
    Designing a Multimodal and Culturally Relevant Alzheimer Disease and Related Dementia Generative Artificial Intelligence Tool for Black American Informal Caregivers: Cognitive Walk-Through Usability Study
    (JMIR, 2025-01-08) Bosco, Cristina; Otenen, Ege; Torres, John Osorio; Nguyen, Vivian; Chheda, Darshil; Peng, Xinran; Jessup, Nenette M.; Himes, Anna K.; Cureton, Bianca; Lu, Yvonne; Hill, Carl V.; Hendrie, Hugh C.; Barnes, Priscilla A.; Shih, Patrick C.; School of Nursing
    Background: Many members of Black American communities, faced with the high prevalence of Alzheimer disease and related dementias (ADRD) within their demographic, find themselves taking on the role of informal caregivers. Despite being the primary individuals responsible for the care of individuals with ADRD, these caregivers often lack sufficient knowledge about ADRD-related health literacy and feel ill-prepared for their caregiving responsibilities. Generative AI has become a new promising technological innovation in the health care domain, particularly for improving health literacy; however, some generative AI developments might lead to increased bias and potential harm toward Black American communities. Therefore, rigorous development of generative AI tools to support the Black American community is needed. Objective: The goal of this study is to test Lola, a multimodal mobile app, which, by relying on generative AI, facilitates access to ADRD-related health information by enabling speech and text as inputs and providing auditory, textual, and visual outputs. Methods: To test our mobile app, we used the cognitive walk-through methodology, and we recruited 15 informal ADRD caregivers who were older than 50 years and part of the Black American community living within the region. We asked them to perform 3 tasks on the mobile app (ie, searching for an article on brain health, searching for local events, and finally, searching for opportunities to participate in scientific research in their area), then we recorded their opinions and impressions. The main aspects to be evaluated were the mobile app's usability, accessibility, cultural relevance, and adoption. Results: Our findings highlight the users' need for a system that enables interaction with different modalities, the need for a system that can provide personalized and culturally and contextually relevant information, and the role of community and physical spaces in increasing the use of Lola. Conclusions: Our study shows that, when designing for Black American older adults, a multimodal interaction with the generative AI system can allow individuals to choose their own interaction way and style based upon their interaction preferences and external constraints. This flexibility of interaction modes can guarantee an inclusive and engaging generative AI experience.
  • Thumbnail Image
    Item
    M3S: a comprehensive model selection for multi-modal single-cell RNA sequencing data
    (BMC, 2019-12-20) Zhang, Yu; Wan, Changlin; Wang, Pengcheng; Chang, Wennan; Huo, Yan; Chen, Jian; Ma, Qin; Cao, Sha; Zhang, Chi; Medical and Molecular Genetics, School of Medicine
    Background Various statistical models have been developed to model the single cell RNA-seq expression profiles, capture its multimodality, and conduct differential gene expression test. However, for expression data generated by different experimental design and platforms, there is currently lack of capability to determine the most proper statistical model. Results We developed an R package, namely Multi-Modal Model Selection (M3S), for gene-wise selection of the most proper multi-modality statistical model and downstream analysis, useful in a single-cell or large scale bulk tissue transcriptomic data. M3S is featured with (1) gene-wise selection of the most parsimonious model among 11 most commonly utilized ones, that can best fit the expression distribution of the gene, (2) parameter estimation of a selected model, and (3) differential gene expression test based on the selected model. Conclusion A comprehensive evaluation suggested that M3S can accurately capture the multimodality on simulated and real single cell data. An open source package and is available through GitHub at https://github.com/zy26/M3S.
  • Thumbnail Image
    Item
    Physiological Metrics of Surgical Difficulty and Multi-Task Requirement during Robotic Surgery Skills
    (MDPI, 2023-04-28) Lim, Chiho; Barragan, Juan Antonio; Farrow, Jason Michael; Wachs, Juan P.; Sundaram, Chandru P.; Yu, Denny; Urology, School of Medicine
    Previous studies in robotic-assisted surgery (RAS) have studied cognitive workload by modulating surgical task difficulty, and many of these studies have relied on self-reported workload measurements. However, contributors to and their effects on cognitive workload are complex and may not be sufficiently summarized by changes in task difficulty alone. This study aims to understand how multi-task requirement contributes to the prediction of cognitive load in RAS under different task difficulties. Multimodal physiological signals (EEG, eye-tracking, HRV) were collected as university students performed simulated RAS tasks consisting of two types of surgical task difficulty under three different multi-task requirement levels. EEG spectral analysis was sensitive enough to distinguish the degree of cognitive workload under both surgical conditions (surgical task difficulty/multi-task requirement). In addition, eye-tracking measurements showed differences under both conditions, but significant differences of HRV were observed in only multi-task requirement conditions. Multimodal-based neural network models have achieved up to 79% accuracy for both surgical conditions.