Browsing by Author "Dexter, Paul"
Now showing 1 - 10 of 19
Results Per Page
Sort Options
Item An Assessment of ChatGPT’s Performance as a Patient Counseling Tool: Exploring the Potential Integration of Large Language Model-based ChatBots into Online Patient Portals(2024-04-26) Price, Charles; Brougham, Albert; Burton, Kyle; Dexter, PaulBACKGROUND: With the advancement of online patient portals, patients now have unprecedented access to their healthcare providers. This has led to increased physician burden associated with electronic inbox overload [1]. Recent developments in artificial intelligence, specifically in Large Language Model-based chatbots (i.e. ChatGPT), may prove to be useful tools in reducing such burden. Can ChatGPT reliably be utilized as a patient counseling tool? ChatGPT can be described as “an advanced language model that uses deep learning techniques to produce human-like responses to natural language inputs” [5]. Despite concerns surrounding this technology (i.e. spreading of misinformation, inconsistent reproducibility, “hallucination” phenomena), several studies have demonstrated ChatGPT’s clinical savviness. One study examined ChatGPT’s ability to answer frequently asked fertility-related questions, finding the model’s responses to be comparable to the CDC’s published answers in respect to length, factual content, and sentiment [6]. Additionally, ChatGPT was found capable of achieving a passing score on the STEP 1 licensing exam, a benchmark set for third year medical students [7]. OBJECTIVE: This study aims to further evaluate the clinical decision making of ChatGPT, specifically the ability for ChatGPT to provide accurate medical counseling in response to frequently asked patient questions within the field of cardiology. METHODS: 35 frequently asked cardiovascular questions (FAQs) published by the OHSU Knight Cardiovascular Institute were processed through ChatGPT 4 (Classic Version) by OpenAI. ChatGPT’s answers and the provided answers by the OHSU Knight Cardiovascular Institute were assessed in respect to length, factual content, sentiment analysis, and the presence of incorrect/false statements. RESULTS: When comparing ChatGPT’s responses to the 35 FAQs against the published responses by OHSU, Chat GPT’s responses were significantly longer in length (295.4 vs 112.5 (words/response)) and included more factual statements per response (7.2 vs 3.5). Chat GPT was able to produce responses of similar sentiment polarity (0.10 vs 0.11 on a scale of -1 (negative) to 1 (positive)) and subjectivity (0.46 vs 0.43 on a scale from 0 (objective) to 1 (subjective)). 0% of ChatGPT’s factual statements were found to be false or harmful. CONCLUSIONS: The results of this study provide valuable insight into the clinical “knowledge” and fluency of ChatGPT, demonstrating its ability to produce accurate and effective responses to frequently asked cardiovascular questions. Larger scale studies with an additional focus on ChatGPT’s reproducibility/consistency may provide important implications for the future of patient education. Implementation of AI-based chatbots into online patient portals may prove to be assistive to physicians, alleviating the growing burden of electronic inbox volume.Item Application of unsupervised deep learning algorithms for identification of specific clusters of chronic cough patients from EMR data(BMC, 2022-04-19) Shao, Wei; Luo, Xiao; Zhang, Zuoyi; Han, Zhi; Chandrasekaran, Vasu; Turzhitsky, Vladimir; Bali, Vishal; Roberts, Anna R.; Metzger, Megan; Baker, Jarod; La Rosa, Carmen; Weaver, Jessica; Dexter, Paul; Huang, Kun; Biostatistics and Health Data Science, School of MedicineBackground: Chronic cough affects approximately 10% of adults. The lack of ICD codes for chronic cough makes it challenging to apply supervised learning methods to predict the characteristics of chronic cough patients, thereby requiring the identification of chronic cough patients by other mechanisms. We developed a deep clustering algorithm with auto-encoder embedding (DCAE) to identify clusters of chronic cough patients based on data from a large cohort of 264,146 patients from the Electronic Medical Records (EMR) system. We constructed features using the diagnosis within the EMR, then built a clustering-oriented loss function directly on embedded features of the deep autoencoder to jointly perform feature refinement and cluster assignment. Lastly, we performed statistical analysis on the identified clusters to characterize the chronic cough patients compared to the non-chronic cough patients. Results: The experimental results show that the DCAE model generated three chronic cough clusters and one non-chronic cough patient cluster. We found various diagnoses, medications, and lab tests highly associated with chronic cough patients by comparing the chronic cough cluster with the non-chronic cough cluster. Comparison of chronic cough clusters demonstrated that certain combinations of medications and diagnoses characterize some chronic cough clusters. Conclusions: To the best of our knowledge, this study is the first to test the potential of unsupervised deep learning methods for chronic cough investigation, which also shows a great advantage over existing algorithms for patient data clustering.Item Connecting research discovery with care delivery in dementia: the development of the Indianapolis Discovery Network for Dementia(Dove Press, 2012) Boustani, Malaz A.; Frame, Amie; Munger, Stephanie; Healey, Patrick; Westlund, Jessie; Farlow, Martin; Hake, Ann; Guerriero Austrom, Mary; Shepard, Polly; Bubp, Corby; Azar, Jose; Nazir, Arif; Adams, Nadia; Campbell, Noll L.; Chehresa, Azita; Dexter, Paul; Neurology, School of MedicineBackground: The US Institute of Medicine has recommended an integrated, locally sensitive collaboration among the various members of the community, health care systems, and research organizations to improve dementia care and dementia research. Methods: Using complex adaptive system theory and reflective adaptive process, we developed a professional network called the "Indianapolis Discovery Network for Dementia" (IDND). The IDND facilitates effective and sustainable interactions among a local and diverse group of dementia researchers, clinical providers, and community advocates interested in improving care for dementia patients in Indianapolis, Indiana. Results: The IDND was established in February 2006 and now includes more than 250 members from more than 30 local (central Indiana) organizations representing 20 disciplines. The network uses two types of communication to connect its members. The first is a 2-hour face-to-face bimonthly meeting open to all members. The second is a web-based resource center (http://www.indydiscoverynetwork.org ). To date, the network has: (1) accomplished the development of a network website with an annual average of 12,711 hits per day; (2) produced clinical tools such as the Healthy Aging Brain Care Monitor and the Anticholinergic Cognitive Burden Scale; (3) translated and implemented the collaborative dementia care model into two local health care systems; (4) created web-based tracking software, the Enhanced Medical Record for Aging Brain Care (eMR-ABC), to support care coordination for patients with dementia; (5) received more than USD$24 million in funding for members for dementia-related research studies; and (6) adopted a new group-based problem-solving process called the "IDND consultancy round." Conclusion: A local interdisciplinary "think-tank" network focused on dementia that promotes collaboration in research projects, educational initiatives, and quality improvement efforts that meet the local research, clinical, and community needs relevant to dementia care has been built.Item DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx(Elsevier, 2015-04) Mehrabi, Saeed; Krishnan, Krishnan; Sohn, Sunghwan; Roch, Alexandra M; Schmidt, Heidi; Kesterson, Joe; Beesley, Chris; Dexter, Paul; Schmidt, C. Max; Liu, Hongfang; Palakal, Mathew; Surgery, School of MedicineIn Electronic Health Records (EHRs), much of valuable information regarding patients’ conditions is embedded in free text format. Natural language processing (NLP) techniques have been developed to extract clinical information from free text. One challenge faced in clinical NLP is that the meaning of clinical entities is heavily affected by modifiers such as negation. A negation detection algorithm, NegEx, applies a simplistic approach that has been shown to be powerful in clinical NLP. However, due to the failure to consider the contextual relationship between words within a sentence, NegEx fails to correctly capture the negation status of concepts in complex sentences. Incorrect negation assignment could cause inaccurate diagnosis of patients’ condition or contaminated study cohorts. We developed a negation algorithm called DEEPEN to decrease NegEx’s false positives by taking into account the dependency relationship between negation words and concepts within a sentence using Stanford dependency parser. The system was developed and tested using EHR data from Indiana University (IU) and it was further evaluated on Mayo Clinic dataset to assess its generalizability. The evaluation results demonstrate DEEPEN, which incorporates dependency parsing into NegEx, can reduce the number of incorrect negation assignment for patients with positive findings, and therefore improve the identification of patients with the target clinical findings in EHRs.Item Demonstration of the Indianapolis SPIN Query Tool for De-identified Access to Content of the Indiana Network for Patient Care’s (a Real RHIO) Database(American Medical Informatics Association, 2006) McDonald, Clement J.; Blevins, Lonnie; Dexter, Paul; Schadow, Gunther; Hook, John; Abernathy, Greg; Dugan, Tammy; Martin, Andrew; Phillips, Ryan; Davis, Mary; Medicine, School of MedicineItem Development and Temporal Validation of an Electronic Medical Record-Based Insomnia Prediction Model Using Data from a Statewide Health Information Exchange(MDPI, 2023-05-05) Holler, Emma; Chekani, Farid; Ai, Jizhou; Meng, Weilin; Khandker, Rezaul Karim; Ben Miled, Zina; Owora, Arthur; Dexter, Paul; Campbell, Noll; Solid, Craig; Boustani, Malaz; Electrical and Computer Engineering, School of Engineering and TechnologyThis study aimed to develop and temporally validate an electronic medical record (EMR)-based insomnia prediction model. In this nested case-control study, we analyzed EMR data from 2011–2018 obtained from a statewide health information exchange. The study sample included 19,843 insomnia cases and 19,843 controls matched by age, sex, and race. Models using different ML techniques were trained to predict insomnia using demographics, diagnosis, and medication order data from two surveillance periods: −1 to −365 days and −180 to −365 days before the first documentation of insomnia. Separate models were also trained with patient data from three time periods (2011–2013, 2011–2015, and 2011–2017). After selecting the best model, predictive performance was evaluated on holdout patients as well as patients from subsequent years to assess the temporal validity of the models. An extreme gradient boosting (XGBoost) model outperformed all other classifiers. XGboost models trained on 2011–2017 data from −1 to −365 and −180 to −365 days before index had AUCs of 0.80 (SD 0.005) and 0.70 (SD 0.006), respectively, on the holdout set. On patients with data from subsequent years, a drop of at most 4% in AUC is observed for all models, even when there is a five-year difference between the collection period of the training and the temporal validation data. The proposed EMR-based prediction models can be used to identify insomnia up to six months before clinical detection. These models may provide an inexpensive, scalable, and longitudinally viable method to screen for individuals at high risk of insomnia.Item Enrollment of Diverse Populations in the INGENIOUS Pharmacogenetics Clinical Trial(Frontiers, 2020-06) Shah-Williams, Ebony; Levy, Kenneth D.; Zang, Yong; Holmes, Ann M.; Stoughton, Christa; Dexter, Paul; Skaar, Todd C.; Medicine, School of MedicineRecruitment of diverse populations and subjects living in Medically Underserved Areas and Populations (MUA/P’s) into clinical trials is a considerable challenge. Likewise, representation of African-Americans in pharmacogenetic trials is often inadequate, but critical for identifying genetic variation within and between populations. To identify enrollment patterns and variables that predict enrollment in a diverse underserved population, we analyzed data from the INGENIOUS (Indiana GENomics Implementation and Opportunity for the UnderServed), pharmacogenomics implementation clinical trial conducted at a community hospital for underserved subjects (Safety net hospital), and a statewide healthcare system (Academic hospital). We used a logistic regression model to identify patient variables that predicted successful enrollment after subjects were contacted and evaluated the reasons that clinical trial eligible subjects refused enrollment. In both healthcare systems, African-Americans were less likely to refuse the study than non-Hispanic Whites (Safety net, OR = 0.68, and p < 0.002; Academic hospital, OR = 0.64, and p < 0.001). At the Safety net hospital, other minorities were more likely to refuse the study than non-Hispanic Whites (OR = 1.58, p < 0.04). The odds of refusing the study once contacted increased with patient age (Safety net hospital, OR = 1.02, p < 0.001, Academic hospital, OR = 1.02, and p < 0.001). At the Academic hospital, females were less likely to refuse the study than males (OR = 0.81, p = 0.01) and those not living in MUA/P’s were less likely to refuse the study than those living in MUA/P’s (OR = 0.81, p = 0.007). The most frequent barriers to enrollment included not being interested, being too busy, transportation, and illness. A lack of trust was reported less frequently. In conclusion, African-Americans can be readily recruited to pharmacogenetic clinical trials once contact has been successfully initiated. However, health care initiatives and increased recruitment efforts of subjects living in MUA/Ps are needed. Enrollment could be further enhanced by improving research awareness and knowledge of clinical trials, reducing time needed for participation, and compensating for travel.Item Identification of Patients with Family History of Pancreatic Cancer - Investigation of an NLP System Portability(IOS, 2015) Mehrabi, Saeed; Krishnan, Anand; Roch, Alexandra M.; Schmidt, Heidi; Li, DingCheng; Kesterson, Joe; Beesley, Chris; Dexter, Paul; Schmidt, Max; Palakal, Mathew; Liu, Hongfang; Department of BioHealth Informatics, School of Informatics and ComputingIn this study we have developed a rule-based natural language processing (NLP) system to identify patients with family history of pancreatic cancer. The algorithm was developed in a Unstructured Information Management Architecture (UIMA) framework and consisted of section segmentation, relation discovery, and negation detection. The system was evaluated on data from two institutions. The family history identification precision was consistent across the institutions shifting from 88.9% on Indiana University (IU) dataset to 87.8% on Mayo Clinic dataset. Customizing the algorithm on the the Mayo Clinic data, increased its precision to 88.1%. The family member relation discovery achieved precision, recall, and F-measure of 75.3%, 91.6% and 82.6% respectively. Negation detection resulted in precision of 99.1%. The results show that rule-based NLP approaches for specific information extraction tasks are portable across institutions; however customization of the algorithm on the new dataset improves its performance.Item Improving Early Dementia Detection Among Diverse Older Adults With Cognitive Concerns With the 5-Cog Paradigm: Protocol for a Hybrid Effectiveness-Implementation Clinical Trial(JMIR, 2025-04-03) Rosansky Chalmer, Rachel Beth; Ayers, Emmeline; Weiss, Erica F.; Fowler, Nicole R.; Telzak, Andrew; Summanwar, Diana; Zwerling, Jessica; Wang, Cuiling; Xu, Huiping; Holden, Richard J.; Fiori, Kevin; French, Dustin D.; Nsubayi, Celeste; Ansari, Asif; Dexter, Paul; Higbie, Anna; Yadav, Pratibha; Walker, James M.; Congivaram, Harrshavasan; Adhikari, Dristi; Melecio-Vazquez, Mairim; Boustani, Malaz; Verghese, Joe; Medicine, School of MedicineBackground: The 5-Cog paradigm is a 5-minute brief cognitive assessment coupled with a clinical decision support tool designed to improve clinicians' early detection of cognitive impairment, including dementia, in their diverse older primary care patients. The 5-Cog battery uses picture- and symbol-based assessments and a questionnaire. It is low cost, simple, minimizes literacy bias, and is culturally fair. The decision support component of the paradigm helps nudge appropriate care provider response to an abnormal 5-Cog battery. Objective: The objective of our study is to evaluate the effectiveness, implementation, and cost of the 5-Cog paradigm. Methods: We will enroll 6600 older patients with cognitive concerns from 22 primary care clinics in the Bronx, New York, and in multiple locations in Indiana for this hybrid type 1 effectiveness-implementation trial. We will analyze the effectiveness of the 5-Cog paradigm to increase the rate of new diagnoses of mild cognitive impairment syndrome or dementia using a pragmatic, cluster randomized clinical trial design. The secondary outcome is the ordering of new tests, treatments, and referrals for cognitive indications within 90 days after the study visit. The 5-Cog's decision support component will be deployed as an electronic medical record feature. We will analyze the 5-Cog's implementation process, context, and outcomes through the Consolidated Framework for Implementation Research using a mixed methods design (surveys and interviews). The study will also examine cost-effectiveness from societal and payer (Medicare) perspectives by estimating the cost per additional dementia diagnosis. Results: The study is funded by the National Institute of Neurological Disorders and Stroke of the National Institutes of Health (2U01NS105565). The protocol was approved by the Albert Einstein College of Medicine Institutional Review Board in September 2022. A validation study was completed to select cut scores for the 5-Cog battery. Among the 76 patients enrolled, the resulting clinical diagnoses were as follows: dementia in 32 (42%); mild cognitive impairment in 28 (37%); subjective cognitive concerns without objective cognitive impairment in 12 (16%); no cognitive diagnosis assigned in 2 (3%). The mean scores were Picture-Based Memory Impairment Screen 5.8 (SD 2.7), Symbol Match 27.2 (SD 18.2), and Subjective Motoric Cognitive Risk 2.4 (SD 1.7). The cut scores for an abnormal or positive result on the 5-Cog components were as follows: Picture-Based Memory Impairment Screen ≤6 (range 0-8), Symbol Match ≤25 (range 0-65), and Subjective Motoric Cognitive Risk >5 (range 0-7). As of December 2024, a total of 12 clinics had completed the onboarding processes, and 2369 patients had been enrolled. Conclusions: The findings of this study will facilitate the rapid adaptation and dissemination of this effective and practical clinical tool across diverse primary care clinical settings.Item Integrating Clinical Decision Support into Workflow(2011) Doebbeling, Bradley N.; Saleem, Jason; Haggstrom, David; Militello, Laura; Flanagan, Mindy; Arbuckle, Nicole; Kiess, Chris; Hoke, Shawn; Dexter, Paul; Linder, Jeff; Sarbah, Steedman; Burgo, LucillePurpose: The aims were to (1) identify barriers and facilitators related to integration of clinical decision support (CDS) into workflow and (2) develop and test CDS design alternatives. Scope: To better understand CDS integration, we studied its use in practice, focusing on CDS for colorectal cancer (CRC) screening and followup. Phase 1 involved outpatient clinics of four different systems—120 clinic staff and providers and 118 patients were observed. In Phase 2, prototyped design enhancements to the Veterans Administration’s CRC screening reminder were compared against its current reminder in a simulation experiment. Twelve providers participated. Methods: Phase 1 was a qualitative project, using key informant interviews, direct observation, opportunistic interviews, and focus groups. All data were analyzed using a coding template, based on the sociotechnical systems theory, which was modified as coding proceeded and themes emerged. Phase 2 consisted of rapid prototyping of CDS design alternatives based on Phase 1 findings and a simulation experiment to test these design changes in a within-subject comparison. Results: Very different CDS types existed across sites, yet there are common barriers: (1) lack of coordination of “outside” results and between primary and specialty care; (2) suboptimal data organization and presentation; (3) needed provider and patient education; (4) needed interface flexibility; (5) needed technological enhancements; (6) unclear role assignments; (7) organizational issues; and (8) disconnect with quality reporting. Design enhancements positively impacted usability and workflow integration but not workload. Conclusions: Effective CDS design and integration requires: (1) organizational and workflow integration; (2) integrating outside results; (3) improving data organization and presentation in a flexible interface; and (4) providing just-in time education, cognitive support, and quality reporting.