IU Indianapolis ScholarWorks :: Browsing by Subject "ChatGPT"

Browsing by Subject "ChatGPT"

Now showing 1 - 10 of 22

A comparative study of English and Japanese ChatGPT responses to anaesthesia-related medical questions
(Elsevier, 2024-06-14) Ando, Kazuo; Sato, Masaki; Wakatsuki, Shin; Nagai, Ryotaro; Chino, Kumiko; Kai, Hinata; Sasaki, Tomomi; Kato, Rie; Phuongtram Nguyen, Teresa; Guo, Nan; Sultan, Pervez; Anesthesia, School of Medicine
Background: The expansion of artificial intelligence (AI) within large language models (LLMs) has the potential to streamline healthcare delivery. Despite the increased use of LLMs, disparities in their performance particularly in different languages, remain underexplored. This study examines the quality of ChatGPT responses in English and Japanese, specifically to questions related to anaesthesiology. Methods: Anaesthesiologists proficient in both languages were recruited as experts in this study. Ten frequently asked questions in anaesthesia were selected and translated for evaluation. Three non-sequential responses from ChatGPT were assessed for content quality (accuracy, comprehensiveness, and safety) and communication quality (understanding, empathy/tone, and ethics) by expert evaluators. Results: Eight anaesthesiologists evaluated English and Japanese LLM responses. The overall quality for all questions combined was higher in English compared with Japanese responses. Content and communication quality were significantly higher in English compared with Japanese LLMs responses (both P<0.001) in all three responses. Comprehensiveness, safety, and understanding were higher scores in English LLM responses. In all three responses, more than half of the evaluators marked overall English responses as better than Japanese responses. Conclusions: English LLM responses to anaesthesia-related frequently asked questions were superior in quality to Japanese responses when assessed by bilingual anaesthesia experts in this report. This study highlights the potential for language-related disparities in healthcare information and the need to improve the quality of AI responses in underrepresented languages. Future studies are needed to explore these disparities in other commonly spoken languages and to compare the performance of different LLMs.
Accuracy of a Commercial Large Language Model (ChatGPT) to Perform Disaster Triage of Simulated Patients Using the Simple Triage and Rapid Treatment (START) Protocol: Gage Repeatability and Reproducibility Study
(JMIR, 2024-09-30) Franc, Jeffrey Micheal; Hertelendy, Attila Julius; Cheng, Lenard; Hata, Ryan; Verde, Manuela; Emergency Medicine, School of Medicine
Background: The release of ChatGPT (OpenAI) in November 2022 drastically reduced the barrier to using artificial intelligence by allowing a simple web-based text interface to a large language model (LLM). One use case where ChatGPT could be useful is in triaging patients at the site of a disaster using the Simple Triage and Rapid Treatment (START) protocol. However, LLMs experience several common errors including hallucinations (also called confabulations) and prompt dependency. Objective: This study addresses the research problem: "Can ChatGPT adequately triage simulated disaster patients using the START protocol?" by measuring three outcomes: repeatability, reproducibility, and accuracy. Methods: Nine prompts were developed by 5 disaster medicine physicians. A Python script queried ChatGPT Version 4 for each prompt combined with 391 validated simulated patient vignettes. Ten repetitions of each combination were performed for a total of 35,190 simulated triages. A reference standard START triage code for each simulated case was assigned by 2 disaster medicine specialists (JMF and MV), with a third specialist (LC) added if the first two did not agree. Results were evaluated using a gage repeatability and reproducibility study (gage R and R). Repeatability was defined as variation due to repeated use of the same prompt. Reproducibility was defined as variation due to the use of different prompts on the same patient vignette. Accuracy was defined as agreement with the reference standard. Results: Although 35,102 (99.7%) queries returned a valid START score, there was considerable variability. Repeatability (use of the same prompt repeatedly) was 14% of the overall variation. Reproducibility (use of different prompts) was 4.1% of the overall variation. The accuracy of ChatGPT for START was 63.9% with a 32.9% overtriage rate and a 3.1% undertriage rate. Accuracy varied by prompt with a maximum of 71.8% and a minimum of 46.7%. Conclusions: This study indicates that ChatGPT version 4 is insufficient to triage simulated disaster patients via the START protocol. It demonstrated suboptimal repeatability and reproducibility. The overall accuracy of triage was only 63.9%. Health care professionals are advised to exercise caution while using commercial LLMs for vital medical determinations, given that these tools may commonly produce inaccurate data, colloquially referred to as hallucinations or confabulations. Artificial intelligence-guided tools should undergo rigorous statistical evaluation-using methods such as gage R and R-before implementation into clinical settings.
Beyond Clinical Accuracy: Considerations for the use of Generative AI Models in Gastrointestinal Care
(AGA, 2023-08) Feldman, Keith; Nehme, Fredy; Medicine, School of Medicine
Can ChatGPT 'Think Like a Lawyer?' A Socratic Dialogue
(2023-01-26) Bishop, Lea
A witty socratic dialogue with a language-generation model, exploring the aims of legal education in the new era of machine writing.
ChatGPT and the Future of Digital Health: A Study on Healthcare Workers' Perceptions and Expectations
(MDPI, 2023-06-21) Temsah, Mohamad-Hani; Aljamaan, Fadi; Malki, Khalid H.; Alhasan, Khalid; Altamimi, Ibraheem; Aljarbou, Razan; Bazuhair, Faisal; Alsubaihin, Abdulmajeed; Abdulmajeed, Naif; Alshahrani, Fatimah S.; Temsah, Reem; Alshahrani, Turki; Al-Eyadhy, Lama; Alkhateeb, Serin Mohammed; Saddik, Basema; Halwani, Rabih; Jamal, Amr; Al-Tawfiq, Jaffar A.; Al-Eyadhy, Ayman; Medicine, School of Medicine
This study aimed to assess the knowledge, attitudes, and intended practices of healthcare workers (HCWs) in Saudi Arabia towards ChatGPT, an artificial intelligence (AI) Chatbot, within the first three months after its launch. We also aimed to identify potential barriers to AI Chatbot adoption among healthcare professionals. A cross-sectional survey was conducted among 1057 HCWs in Saudi Arabia, distributed electronically via social media channels from 21 February to 6 March 2023. The survey evaluated HCWs' familiarity with ChatGPT-3.5, their satisfaction, intended future use, and perceived usefulness in healthcare practice. Of the respondents, 18.4% had used ChatGPT for healthcare purposes, while 84.1% of non-users expressed interest in utilizing AI Chatbots in the future. Most participants (75.1%) were comfortable with incorporating ChatGPT into their healthcare practice. HCWs perceived the Chatbot to be useful in various aspects of healthcare, such as medical decision-making (39.5%), patient and family support (44.7%), medical literature appraisal (48.5%), and medical research assistance (65.9%). A majority (76.7%) believed ChatGPT could positively impact the future of healthcare systems. Nevertheless, concerns about credibility and the source of information provided by AI Chatbots (46.9%) were identified as the main barriers. Although HCWs recognize ChatGPT as a valuable addition to digital health in the early stages of adoption, addressing concerns regarding accuracy, reliability, and medicolegal implications is crucial. Therefore, due to their unreliability, the current forms of ChatGPT and other Chatbots should not be used for diagnostic or treatment purposes without human expert oversight. Ensuring the trustworthiness and dependability of AI Chatbots is essential for successful implementation in healthcare settings. Future research should focus on evaluating the clinical outcomes of ChatGPT and benchmarking its performance against other AI Chatbots.
ChatGPT-3.5 System Usability Scale early assessment among Healthcare Workers: Horizons of adoption in medical practice
(Elsevier, 2024-04-07) Aljamaan, Fadi; Malki, Khalid H.; Alhasan, Khalid; Jamal, Amr; Altamimi, Ibraheem; Khayat, Afnan; Alhaboob, Ali; Abdulmajeed, Naif; Alshahrani, Fatimah S.; Saad, Khaled; Al-Eyadhy, Ayman; Al-Tawfiq, Jaffar A.; Temsah, Mohamad-Hani; Medicine, School of Medicine
Artificial intelligence (AI) chatbots, such as ChatGPT, have widely invaded all domains of human life. They have the potential to transform healthcare future. However, their effective implementation hinges on healthcare workers' (HCWs) adoption and perceptions. This study aimed to evaluate HCWs usability of ChatGPT three months post-launch in Saudi Arabia using the System Usability Scale (SUS). A total of 194 HCWs participated in the survey. Forty-seven percent were satisfied with their usage, 57 % expressed moderate to high trust in its ability to generate medical decisions. 58 % expected ChatGPT would improve patients' outcomes, even though 84 % were optimistic of its potential to improve the future of healthcare practice. They expressed possible concerns like recommending harmful medical decisions and medicolegal implications. The overall mean SUS score was 64.52, equivalent to 50 % percentile rank, indicating high marginal acceptability of the system. The strongest positive predictors of high SUS scores were participants' belief in AI chatbot's benefits in medical research, self-rated familiarity with ChatGPT and self-rated computer skills proficiency. Participants' learnability and ease of use score correlated positively but weakly. On the other hand, medical students and interns had significantly high learnability scores compared to others, while ease of use scores correlated very strongly with participants' perception of positive impact of ChatGPT on the future of healthcare practice. Our findings highlight the HCWs' perceived marginal acceptance of ChatGPT at the current stage and their optimism of its potential in supporting them in future practice, especially in the research domain, in addition to humble ambition of its potential to improve patients' outcomes particularly in regard of medical decisions. On the other end, it underscores the need for ongoing efforts to build trust and address ethical and legal concerns of AI implications in healthcare. The study contributes to the growing body of literature on AI chatbots in healthcare, especially addressing its future improvement strategies and provides insights for policymakers and healthcare providers about the potential benefits and challenges of implementing them in their practice.
A Computer Wrote this Paper: What ChatGPT Means for Education, Research, and Writing
(2023-01-26) Bishop, Lea
Of particular interest to educators, an exploration of what new language-generation software does (and does not) do well. Argues that the new language-generation models make instruction in writing mechanics irrelevant, and that educators should shift to teaching only the more advanced writing skills that reflect and advance critical thinking. The difference between mechanical and advanced writing is illustrated through a "Socratic Dialogue" with ChatGPT. Appropriate for classroom discussion at High School, College, Professional, and PhD levels.
COVID-19 and Bone Loss: A Review of Risk Factors, Mechanisms, and Future Directions
(Springer, 2024) Creecy, Amy; Awosanya, Olatundun D.; Harris, Alexander; Qiao, Xian; Ozanne, Marie; Toepp, Angela J.; Kacena, Melissa A.; McCune, Thomas; Orthopaedic Surgery, School of Medicine
Purpose of review: SARS-CoV-2 drove the catastrophic global phenomenon of the COVID-19 pandemic resulting in a multitude of systemic health issues, including bone loss. The purpose of this review is to summarize recent findings related to bone loss and potential mechanisms. Recent findings: The early clinical evidence indicates an increase in vertebral fractures, hypocalcemia, vitamin D deficiencies, and a loss in BMD among COVID-19 patients. Additionally, lower BMD is associated with more severe SARS-CoV-2 infection. Preclinical models have shown bone loss and increased osteoclastogenesis. The bone loss associated with SARS-CoV-2 infection could be the result of many factors that directly affect the bone such as higher inflammation, activation of the NLRP3 inflammasome, recruitment of Th17 cells, the hypoxic environment, and changes in RANKL/OPG signaling. Additionally, SARS-CoV-2 infection can exert indirect effects on the skeleton, as mechanical unloading may occur with severe disease (e.g., bed rest) or with BMI loss and muscle wasting that has also been shown to occur with SARS-CoV-2 infection. Muscle wasting can also cause systemic issues that may influence the bone. Medications used to treat SARS-CoV-2 infection also have a negative effect on the bone. Lastly, SARS-CoV-2 infection may also worsen conditions such as diabetes and negatively affect kidney function, all of which could contribute to bone loss and increased fracture risk. SARS-CoV-2 can negatively affect the bone through multiple direct and indirect mechanisms. Future work will be needed to determine what patient populations are at risk of COVID-19-related increases in fracture risk, the mechanisms behind bone loss, and therapeutic options. This review article is part of a series of multiple manuscripts designed to determine the utility of using artificial intelligence for writing scientific reviews.
Cracking the Code: The Role of Peripheral Nervous System Signaling in Fracture Repair
(Springer, 2024) Morris, Ashlyn J.; Parker, Reginald S.; Nazzal, Murad K.; Natoli, Roman M.; Fehrenbacher, Jill C.; Kacena, Melissa A.; White, Fletcher A.; Orthopaedic Surgery, School of Medicine
Purpose of review: The traditionally understated role of neural regulation in fracture healing is gaining prominence, as recent findings underscore the peripheral nervous system's critical contribution to bone repair. Indeed, it is becoming more evident that the nervous system modulates every stage of fracture healing, from the onset of inflammation to repair and eventual remodeling. Recent findings: Essential to this process are neurotrophins and neuropeptides, such as substance P, calcitonin gene-related peptide, and neuropeptide Y. These molecules fulfill key roles in promoting osteogenesis, influencing inflammation, and mediating pain. The sympathetic nervous system also plays an important role in the healing process: while local sympathectomies may improve fracture healing, systemic sympathetic denervation impairs fracture healing. Furthermore, chronic activation of the sympathetic nervous system, often triggered by stress, is a potential impediment to effective fracture healing, marking an important area for further investigation. The potential to manipulate aspects of the nervous system offers promising therapeutic possibilities for improving outcomes in fracture healing. This review article is part of a series of multiple manuscripts designed to determine the utility of using artificial intelligence for writing scientific reviews.
Do Not Lose Your Nerve, Be Callus: Insights Into Neural Regulation of Fracture Healing
(Springer, 2024) Nazzal, Murad K.; Morris, Ashlyn J.; Parker, Reginald S.; White, Fletcher A.; Natoli, Roman M.; Kacena, Melissa A.; Fehrenbacher, Jill C.; Orthopaedic Surgery, School of Medicine
Purpose of review: Fractures are a prominent form of traumatic injury and shall continue to be for the foreseeable future. While the inflammatory response and the cells of the bone marrow microenvironment play significant roles in fracture healing, the nervous system is also an important player in regulating bone healing. Recent findings: Considerable evidence demonstrates a role for nervous system regulation of fracture healing in a setting of traumatic injury to the brain. Although many of the impacts of the nervous system on fracture healing are positive, pain mediated by the nervous system can have detrimental effects on mobilization and quality of life. Understanding the role the nervous system plays in fracture healing is vital to understanding fracture healing as a whole and improving quality of life post-injury. This review article is part of a series of multiple manuscripts designed to determine the utility of using artificial intelligence for writing scientific reviews.

Browsing by Subject "ChatGPT"

Results Per Page

Sort Options