Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency

dc.contributor.authorShyr, Cathy
dc.contributor.authorGrout, Randall W.
dc.contributor.authorKennedy, Nan
dc.contributor.authorAkdas, Yasemin
dc.contributor.authorTischbein, Maeve
dc.contributor.authorMilford, Joshua
dc.contributor.authorTan, Jason
dc.contributor.authorQuarles, Kaysi
dc.contributor.authorEdwards, Terri L.
dc.contributor.authorNovak, Laurie L.
dc.contributor.authorWhite, Jules
dc.contributor.authorWilkins, Consuelo H.
dc.contributor.authorHarris, Paul A.
dc.contributor.departmentPediatrics, School of Medicine
dc.date.accessioned2024-10-29T09:35:23Z
dc.date.available2024-10-29T09:35:23Z
dc.date.issued2024
dc.description.abstractObjective: Returning aggregate study results is an important ethical responsibility to promote trust and inform decision making, but the practice of providing results to a lay audience is not widely adopted. Barriers include significant cost and time required to develop lay summaries and scarce infrastructure necessary for returning them to the public. Our study aims to generate, evaluate, and implement ChatGPT 4 lay summaries of scientific abstracts on a national clinical study recruitment platform, ResearchMatch, to facilitate timely and cost-effective return of study results at scale. Materials and methods: We engineered prompts to summarize abstracts at a literacy level accessible to the public, prioritizing succinctness, clarity, and practical relevance. Researchers and volunteers assessed ChatGPT-generated lay summaries across five dimensions: accuracy, relevance, accessibility, transparency, and harmfulness. We used precision analysis and adaptive random sampling to determine the optimal number of summaries for evaluation, ensuring high statistical precision. Results: ChatGPT achieved 95.9% (95% CI, 92.1-97.9) accuracy and 96.2% (92.4-98.1) relevance across 192 summary sentences from 33 abstracts based on researcher review. 85.3% (69.9-93.6) of 34 volunteers perceived ChatGPT-generated summaries as more accessible and 73.5% (56.9-85.4) more transparent than the original abstract. None of the summaries were deemed harmful. We expanded ResearchMatch's technical infrastructure to automatically generate and display lay summaries for over 750 published studies that resulted from the platform's recruitment mechanism. Discussion and conclusion: Implementing AI-generated lay summaries on ResearchMatch demonstrates the potential of a scalable framework generalizable to broader platforms for enhancing research accessibility and transparency.
dc.eprint.versionFinal published version
dc.identifier.citationShyr C, Grout RW, Kennedy N, et al. Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency. J Am Med Inform Assoc. 2024;31(10):2294-2303. doi:10.1093/jamia/ocae186
dc.identifier.urihttps://hdl.handle.net/1805/44294
dc.language.isoen_US
dc.publisherOxford University Press
dc.relation.isversionof10.1093/jamia/ocae186
dc.relation.journalJournal of the American Medical Informatics Association
dc.rightsAttribution-NonCommercial-NoDerivatives 4.0 Internationalen
dc.rights.urihttps://creativecommons.org/licenses/by-nc-nd/4.0
dc.sourcePMC
dc.subjectResearchMatch
dc.subjectArtificial intelligence
dc.subjectLarge language model
dc.subjectReturn of study results
dc.subjectText summarization
dc.titleLeveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency
dc.typeArticle
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Shyr2024Leveraging-CCBYNCND.pdf
Size:
1.45 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
2.04 KB
Format:
Item-specific license agreed upon to submission
Description: