Browsing by Author "Dexter, Paul"
Now showing 1 - 10 of 16
Results Per Page
Sort Options
Item An Assessment of ChatGPT’s Performance as a Patient Counseling Tool: Exploring the Potential Integration of Large Language Model-based ChatBots into Online Patient Portals(2024-04-26) Price, Charles; Brougham, Albert; Burton, Kyle; Dexter, PaulBACKGROUND: With the advancement of online patient portals, patients now have unprecedented access to their healthcare providers. This has led to increased physician burden associated with electronic inbox overload [1]. Recent developments in artificial intelligence, specifically in Large Language Model-based chatbots (i.e. ChatGPT), may prove to be useful tools in reducing such burden. Can ChatGPT reliably be utilized as a patient counseling tool? ChatGPT can be described as “an advanced language model that uses deep learning techniques to produce human-like responses to natural language inputs” [5]. Despite concerns surrounding this technology (i.e. spreading of misinformation, inconsistent reproducibility, “hallucination” phenomena), several studies have demonstrated ChatGPT’s clinical savviness. One study examined ChatGPT’s ability to answer frequently asked fertility-related questions, finding the model’s responses to be comparable to the CDC’s published answers in respect to length, factual content, and sentiment [6]. Additionally, ChatGPT was found capable of achieving a passing score on the STEP 1 licensing exam, a benchmark set for third year medical students [7]. OBJECTIVE: This study aims to further evaluate the clinical decision making of ChatGPT, specifically the ability for ChatGPT to provide accurate medical counseling in response to frequently asked patient questions within the field of cardiology. METHODS: 35 frequently asked cardiovascular questions (FAQs) published by the OHSU Knight Cardiovascular Institute were processed through ChatGPT 4 (Classic Version) by OpenAI. ChatGPT’s answers and the provided answers by the OHSU Knight Cardiovascular Institute were assessed in respect to length, factual content, sentiment analysis, and the presence of incorrect/false statements. RESULTS: When comparing ChatGPT’s responses to the 35 FAQs against the published responses by OHSU, Chat GPT’s responses were significantly longer in length (295.4 vs 112.5 (words/response)) and included more factual statements per response (7.2 vs 3.5). Chat GPT was able to produce responses of similar sentiment polarity (0.10 vs 0.11 on a scale of -1 (negative) to 1 (positive)) and subjectivity (0.46 vs 0.43 on a scale from 0 (objective) to 1 (subjective)). 0% of ChatGPT’s factual statements were found to be false or harmful. CONCLUSIONS: The results of this study provide valuable insight into the clinical “knowledge” and fluency of ChatGPT, demonstrating its ability to produce accurate and effective responses to frequently asked cardiovascular questions. Larger scale studies with an additional focus on ChatGPT’s reproducibility/consistency may provide important implications for the future of patient education. Implementation of AI-based chatbots into online patient portals may prove to be assistive to physicians, alleviating the growing burden of electronic inbox volume.Item Application of unsupervised deep learning algorithms for identification of specific clusters of chronic cough patients from EMR data(BMC, 2022-04-19) Shao, Wei; Luo, Xiao; Zhang, Zuoyi; Han, Zhi; Chandrasekaran, Vasu; Turzhitsky, Vladimir; Bali, Vishal; Roberts, Anna R.; Metzger, Megan; Baker, Jarod; La Rosa, Carmen; Weaver, Jessica; Dexter, Paul; Huang, Kun; Biostatistics and Health Data Science, School of MedicineBackground: Chronic cough affects approximately 10% of adults. The lack of ICD codes for chronic cough makes it challenging to apply supervised learning methods to predict the characteristics of chronic cough patients, thereby requiring the identification of chronic cough patients by other mechanisms. We developed a deep clustering algorithm with auto-encoder embedding (DCAE) to identify clusters of chronic cough patients based on data from a large cohort of 264,146 patients from the Electronic Medical Records (EMR) system. We constructed features using the diagnosis within the EMR, then built a clustering-oriented loss function directly on embedded features of the deep autoencoder to jointly perform feature refinement and cluster assignment. Lastly, we performed statistical analysis on the identified clusters to characterize the chronic cough patients compared to the non-chronic cough patients. Results: The experimental results show that the DCAE model generated three chronic cough clusters and one non-chronic cough patient cluster. We found various diagnoses, medications, and lab tests highly associated with chronic cough patients by comparing the chronic cough cluster with the non-chronic cough cluster. Comparison of chronic cough clusters demonstrated that certain combinations of medications and diagnoses characterize some chronic cough clusters. Conclusions: To the best of our knowledge, this study is the first to test the potential of unsupervised deep learning methods for chronic cough investigation, which also shows a great advantage over existing algorithms for patient data clustering.Item DEEPEN: A negation detection system for clinical text incorporating dependency relation into NegEx(Elsevier, 2015-04) Mehrabi, Saeed; Krishnan, Krishnan; Sohn, Sunghwan; Roch, Alexandra M; Schmidt, Heidi; Kesterson, Joe; Beesley, Chris; Dexter, Paul; Schmidt, C. Max; Liu, Hongfang; Palakal, Mathew; Surgery, School of MedicineIn Electronic Health Records (EHRs), much of valuable information regarding patients’ conditions is embedded in free text format. Natural language processing (NLP) techniques have been developed to extract clinical information from free text. One challenge faced in clinical NLP is that the meaning of clinical entities is heavily affected by modifiers such as negation. A negation detection algorithm, NegEx, applies a simplistic approach that has been shown to be powerful in clinical NLP. However, due to the failure to consider the contextual relationship between words within a sentence, NegEx fails to correctly capture the negation status of concepts in complex sentences. Incorrect negation assignment could cause inaccurate diagnosis of patients’ condition or contaminated study cohorts. We developed a negation algorithm called DEEPEN to decrease NegEx’s false positives by taking into account the dependency relationship between negation words and concepts within a sentence using Stanford dependency parser. The system was developed and tested using EHR data from Indiana University (IU) and it was further evaluated on Mayo Clinic dataset to assess its generalizability. The evaluation results demonstrate DEEPEN, which incorporates dependency parsing into NegEx, can reduce the number of incorrect negation assignment for patients with positive findings, and therefore improve the identification of patients with the target clinical findings in EHRs.Item Development and Temporal Validation of an Electronic Medical Record-Based Insomnia Prediction Model Using Data from a Statewide Health Information Exchange(MDPI, 2023-05-05) Holler, Emma; Chekani, Farid; Ai, Jizhou; Meng, Weilin; Khandker, Rezaul Karim; Ben Miled, Zina; Owora, Arthur; Dexter, Paul; Campbell, Noll; Solid, Craig; Boustani, Malaz; Electrical and Computer Engineering, School of Engineering and TechnologyThis study aimed to develop and temporally validate an electronic medical record (EMR)-based insomnia prediction model. In this nested case-control study, we analyzed EMR data from 2011–2018 obtained from a statewide health information exchange. The study sample included 19,843 insomnia cases and 19,843 controls matched by age, sex, and race. Models using different ML techniques were trained to predict insomnia using demographics, diagnosis, and medication order data from two surveillance periods: −1 to −365 days and −180 to −365 days before the first documentation of insomnia. Separate models were also trained with patient data from three time periods (2011–2013, 2011–2015, and 2011–2017). After selecting the best model, predictive performance was evaluated on holdout patients as well as patients from subsequent years to assess the temporal validity of the models. An extreme gradient boosting (XGBoost) model outperformed all other classifiers. XGboost models trained on 2011–2017 data from −1 to −365 and −180 to −365 days before index had AUCs of 0.80 (SD 0.005) and 0.70 (SD 0.006), respectively, on the holdout set. On patients with data from subsequent years, a drop of at most 4% in AUC is observed for all models, even when there is a five-year difference between the collection period of the training and the temporal validation data. The proposed EMR-based prediction models can be used to identify insomnia up to six months before clinical detection. These models may provide an inexpensive, scalable, and longitudinally viable method to screen for individuals at high risk of insomnia.Item Enrollment of Diverse Populations in the INGENIOUS Pharmacogenetics Clinical Trial(Frontiers, 2020-06) Shah-Williams, Ebony; Levy, Kenneth D.; Zang, Yong; Holmes, Ann M.; Stoughton, Christa; Dexter, Paul; Skaar, Todd C.; Medicine, School of MedicineRecruitment of diverse populations and subjects living in Medically Underserved Areas and Populations (MUA/P’s) into clinical trials is a considerable challenge. Likewise, representation of African-Americans in pharmacogenetic trials is often inadequate, but critical for identifying genetic variation within and between populations. To identify enrollment patterns and variables that predict enrollment in a diverse underserved population, we analyzed data from the INGENIOUS (Indiana GENomics Implementation and Opportunity for the UnderServed), pharmacogenomics implementation clinical trial conducted at a community hospital for underserved subjects (Safety net hospital), and a statewide healthcare system (Academic hospital). We used a logistic regression model to identify patient variables that predicted successful enrollment after subjects were contacted and evaluated the reasons that clinical trial eligible subjects refused enrollment. In both healthcare systems, African-Americans were less likely to refuse the study than non-Hispanic Whites (Safety net, OR = 0.68, and p < 0.002; Academic hospital, OR = 0.64, and p < 0.001). At the Safety net hospital, other minorities were more likely to refuse the study than non-Hispanic Whites (OR = 1.58, p < 0.04). The odds of refusing the study once contacted increased with patient age (Safety net hospital, OR = 1.02, p < 0.001, Academic hospital, OR = 1.02, and p < 0.001). At the Academic hospital, females were less likely to refuse the study than males (OR = 0.81, p = 0.01) and those not living in MUA/P’s were less likely to refuse the study than those living in MUA/P’s (OR = 0.81, p = 0.007). The most frequent barriers to enrollment included not being interested, being too busy, transportation, and illness. A lack of trust was reported less frequently. In conclusion, African-Americans can be readily recruited to pharmacogenetic clinical trials once contact has been successfully initiated. However, health care initiatives and increased recruitment efforts of subjects living in MUA/Ps are needed. Enrollment could be further enhanced by improving research awareness and knowledge of clinical trials, reducing time needed for participation, and compensating for travel.Item Identification of Patients with Family History of Pancreatic Cancer - Investigation of an NLP System Portability(IOS, 2015) Mehrabi, Saeed; Krishnan, Anand; Roch, Alexandra M.; Schmidt, Heidi; Li, DingCheng; Kesterson, Joe; Beesley, Chris; Dexter, Paul; Schmidt, Max; Palakal, Mathew; Liu, Hongfang; Department of BioHealth Informatics, School of Informatics and ComputingIn this study we have developed a rule-based natural language processing (NLP) system to identify patients with family history of pancreatic cancer. The algorithm was developed in a Unstructured Information Management Architecture (UIMA) framework and consisted of section segmentation, relation discovery, and negation detection. The system was evaluated on data from two institutions. The family history identification precision was consistent across the institutions shifting from 88.9% on Indiana University (IU) dataset to 87.8% on Mayo Clinic dataset. Customizing the algorithm on the the Mayo Clinic data, increased its precision to 88.1%. The family member relation discovery achieved precision, recall, and F-measure of 75.3%, 91.6% and 82.6% respectively. Negation detection resulted in precision of 99.1%. The results show that rule-based NLP approaches for specific information extraction tasks are portable across institutions; however customization of the algorithm on the new dataset improves its performance.Item Integrating Clinical Decision Support into Workflow(2011) Doebbeling, Bradley N.; Saleem, Jason; Haggstrom, David; Militello, Laura; Flanagan, Mindy; Arbuckle, Nicole; Kiess, Chris; Hoke, Shawn; Dexter, Paul; Linder, Jeff; Sarbah, Steedman; Burgo, LucillePurpose: The aims were to (1) identify barriers and facilitators related to integration of clinical decision support (CDS) into workflow and (2) develop and test CDS design alternatives. Scope: To better understand CDS integration, we studied its use in practice, focusing on CDS for colorectal cancer (CRC) screening and followup. Phase 1 involved outpatient clinics of four different systems—120 clinic staff and providers and 118 patients were observed. In Phase 2, prototyped design enhancements to the Veterans Administration’s CRC screening reminder were compared against its current reminder in a simulation experiment. Twelve providers participated. Methods: Phase 1 was a qualitative project, using key informant interviews, direct observation, opportunistic interviews, and focus groups. All data were analyzed using a coding template, based on the sociotechnical systems theory, which was modified as coding proceeded and themes emerged. Phase 2 consisted of rapid prototyping of CDS design alternatives based on Phase 1 findings and a simulation experiment to test these design changes in a within-subject comparison. Results: Very different CDS types existed across sites, yet there are common barriers: (1) lack of coordination of “outside” results and between primary and specialty care; (2) suboptimal data organization and presentation; (3) needed provider and patient education; (4) needed interface flexibility; (5) needed technological enhancements; (6) unclear role assignments; (7) organizational issues; and (8) disconnect with quality reporting. Design enhancements positively impacted usability and workflow integration but not workload. Conclusions: Effective CDS design and integration requires: (1) organizational and workflow integration; (2) integrating outside results; (3) improving data organization and presentation in a flexible interface; and (4) providing just-in time education, cognitive support, and quality reporting.Item Management of Chronic Cough in Adult Primary Care: A Qualitative Study(Springer, 2021-09) Gowan, Tayler M.; Huffman, Monica; Weiner, Michael; Talib, Tasneem L.; Schelfhout, Jonathan; Weaver, Jessica; Griffith, Ashley; Doshi, Ishita; Dexter, Paul; Bali, Vishal; Medicine, School of MedicineThis study is the first to describe, qualitatively, PCPs’ experiences evaluating and treating CC in adults. By interviewing clinicians, we sought to understand reasons for referrals, accessibility and use of clinical guidelines, confidence in evaluation and treatment, perceptions and attitudes, and desired resources. Findings may help in elucidating clinical decision-making and could indicate areas for improvement in dissemination and use of guidelines.Item Medication adherence and tolerability of Alzheimer’s disease medications: study protocol for a randomized controlled trial(BMC, 2013-05-04) Campbell, Noll L.; Dexter, Paul; Perkins, Anthony J.; Gao, Sujuan; Li, Lang; Skaar, Todd C.; Frame, Amie; Hendrie, Hugh C.; Callahan, Chris M.; Boustani, Malaz A.Background: The class of acetylcholinesterase inhibitors (ChEI), including donepezil, rivastigmine, and galantamine, have similar efficacy profiles in patients with mild to moderate Alzheimer's disease (AD). However, few studies have evaluated adherence to these agents. We sought to prospectively capture the rates and reasons for nonadherence to ChEI and determine factors influencing tolerability and adherence. Methods/design: We designed a pragmatic randomized clinical trial to evaluate the adherence to ChEIs among older adults with AD. Participants include AD patients receiving care within memory care practices in the greater Indianapolis area. Participants will be followed at 6-week intervals up to 18 weeks to measure the primary outcome of ChEI discontinuation and adherence rates and secondary outcomes of behavioral and psychological symptoms of dementia. The primary outcome will be assessed through two methods, a telephone interview of an informal caregiver and electronic medical record data captured from each healthcare system through a regional health information exchange. The secondary outcome will be measured by the Healthy Aging Brain Care Monitor and the Neuropsychiatric Inventory. In addition, the trial will conduct an exploratory evaluation of the pharmacogenomic signatures for the efficacy and the adverse effect responses to ChEIs. We hypothesized that patient-specific factors, including pharmacogenomics and pharmacokinetic characteristics, may influence the study outcomes. Discussion: This pragmatic trial will engage a diverse population from multiple memory care practices to evaluate the adherence to and tolerability of ChEIs in a real world setting. Engaging participants from multiple healthcare systems connected through a health information exchange will capture valuable clinical and non-clinical influences on the patterns of utilization and tolerability of a class of medications with a high rate of discontinuation.Item Modeling acute care utilization: practical implications for insomnia patients(Springer Nature, 2023-02-07) Chekani, Farid; Zhu, Zitong; Khandker, Rezaul Karim; Ai, Jizhou; Meng, Weilin; Holler, Emma; Dexter, Paul; Boustani, Malaz; Ben Miled, Zina; Medicine, School of MedicineMachine learning models can help improve health care services. However, they need to be practical to gain wide-adoption. In this study, we investigate the practical utility of different data modalities and cohort segmentation strategies when designing models for emergency department (ED) and inpatient hospital (IH) visits. The data modalities include socio-demographics, diagnosis and medications. Segmentation compares a cohort of insomnia patients to a cohort of general non-insomnia patients under varying age and disease severity criteria. Transfer testing between the two cohorts is introduced to demonstrate that an insomnia-specific model is not necessary when predicting future ED visits, but may have merit when predicting IH visits especially for patients with an insomnia diagnosis. The results also indicate that using both diagnosis and medications as a source of data does not generally improve model performance and may increase its overhead. Based on these findings, the proposed evaluation methodologies are recommended to ascertain the utility of disease-specific models in addition to the traditional intra-cohort testing.