Scientific Evidence for Clinical Text Summarization Using Large Language Models : Scoping Review

Article Article scientifique Article Scientific Evidence for Clinical Text Summarization Using Large Language Models : Scoping Review eng 2025-05-15 author Bednarczyk Lydie 0009-0000-2602-2927 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland. 841550 author Reichenpfader Daniel 0000-0002-8052-3359 Institute for Patient-centered Digital Health, Bern University of Applied Sciences, Biel, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 413384 author Gaudet-Blavignac Christophe 0000-0001-6527-5898 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland. 645533 author Etté Amon Kenna 0000-0001-7637-0057 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 881554 author Zaghir Jamil 0000-0002-8209-6098 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 956784 Département de radiologie et informatique médicale (RADIO) author Zheng Yuanyuan 0009-0008-7608-2571 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 406172 Département de radiologie et informatique médicale (RADIO) author Bensalah Talet Adel Fouad 0009-0001-8144-2331 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 420891 author Bjelogrlic Mina 0000-0002-6922-3283 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 918489 Département de radiologie et informatique médicale (RADIO) author Lovis Christian 0000-0002-2681-8076 Division of Medical Information Sciences, University Hospital of Geneva, Geneva, Switzerland.,Faculty of Medicine, University of Geneva, Geneva, Switzerland. 29016 Département de radiologie et informatique médicale (RADIO) e68998 10.2196/68998 40371947 1438-8871 PMC12123242 https://www.jmir.org/2025/1/e68998 Background: Information overload in electronic health records requires effective solutions to alleviate clinicians' administrative tasks. Automatically summarizing clinical text has gained significant attention with the rise of large language models. While individual studies show optimism, a structured overview of the research landscape is lacking. Objective: This study aims to present the current state of the art on clinical text summarization using large language models, evaluate the level of evidence in existing research and assess the applicability of performance findings in clinical settings. Methods: This scoping review complied with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines. Literature published between January 1, 2019, and June 18, 2024, was identified from 5 databases: PubMed, Embase, Web of Science, IEEE Xplore, and ACM Digital Library. Studies were excluded if they did not describe transformer-based models, did not focus on clinical text summarization, did not engage with free-text data, were not original research, were nonretrievable, were not peer-reviewed, or were not in English, French, Spanish, or German. Data related to study context and characteristics, scope of research, and evaluation methodologies were systematically collected and analyzed by 3 authors independently. Results: A total of 30 original studies were included in the analysis. All used observational retrospective designs, mainly using real patient data (n=28, 93%). The research landscape demonstrated a narrow research focus, often centered on summarizing radiology reports (n=17, 57%), primarily involving data from the intensive care unit (n=15, 50%) of US-based institutions (n=19, 73%), in English (n=26, 87%). This focus aligned with the frequent reliance on the open-source Medical Information Mart for Intensive Care dataset (n=15, 50%). Summarization methodologies predominantly involved abstractive approaches (n=17, 57%) on single-document inputs (n=4, 13%) with unstructured data (n=13, 43%), yet reporting on methodological details remained inconsistent across studies. Model selection involved both open-source models (n=26, 87%) and proprietary models (n=7, 23%). Evaluation frameworks were highly heterogeneous. All studies conducted internal validation, but external validation (n=2, 7%), failure analysis (n=6, 20%), and patient safety risks analysis (n=1, 3%) were infrequent, and none reported bias assessment. Most studies used both automated metrics and human evaluation (n=16, 53%), while 10 (33%) used only automated metrics, and 4 (13%) only human evaluation. Conclusions: Key barriers hinder the translation of current research into trustworthy, clinically valid applications. Current research remains exploratory and limited in scope, with many applications yet to be explored. Performance assessments often lack reliability, and clinical impact evaluations are insufficient raising concerns about model utility, safety, fairness, and data privacy. Advancing the field requires more robust evaluation frameworks, a broader research scope, and a stronger focus on real-world applicability. Artificial intelligence Electronic health records Health care Large language models Natural language processing Scoping review Summarization Translational research Dewey 616.0757 62 Département de radiologie et informatique médicale 227 false 344 Interfaces Homme-machine en milieu clinique 610 JMIR. Journal of medical internet research 27 e924b16f-1569-4ff8-80e2-23acedbe4de4 Article (Published version) Public jmir-2025-1-e68998.pdf 916115 application/pdf CC BY-4.0 b13720dd47dc5f90f52b184ab1cb100b6f6442e99540d526cb4269bd5e82dd5b 2760b67e-b924-4d47-9bef-d450f618147c Supplemental data Public jmir_v27i1e68998_app1.pdf 413273 application/pdf fda35ae46eb9105a82a39067c6765f973def48da5c80040262372f70610a5800 2e45fbd3-b14c-4a74-b0f0-93c13590c67b Supplemental data Public jmir_v27i1e68998_app2.docx 15711 application/vnd.openxmlformats-officedocument.wordprocessingml.document 0bdd304c2ac1f7758ab7c7c47c2445be0f159a5074105531657b8570149cd3bf 554acce8-1512-40c4-b5dc-3c5e64659ade Supplemental data Public jmir_v27i1e68998_app3.xlsx 194286 application/vnd.openxmlformats-officedocument.spreadsheetml.sheet 5fc1e6f33cb9af6840cb9b715a044a9c06993fb27e29c50374aa861bd4203138 bd100ecc-7b29-4fd3-b6c7-e9590834d3fd Supplemental data Public jmir_v27i1e68998_app4.docx 54659 application/vnd.openxmlformats-officedocument.wordprocessingml.document 54cc1a38887ac5706a5278b1275f73571c6ed4943d29bb2fd38f4321c5181c18 2bbc8718-6318-4389-bb8b-6ddf3bca01d2 Supplemental data Public jmir_v27i1e68998_app5.docx 24869 application/vnd.openxmlformats-officedocument.wordprocessingml.document b2b13e5cb59b96455f9e5f431678ec0218a38e9a1ac862e9726fa06a27c0e57d openAccess Faculté de médecine / Section de médecine clinique / Département de radiologie et informatique médicale

Title	Published in	Access level	OA Policy	Year	Views	Downloads

Scientific Evidence for Clinical Text Summarization Using Large Language Models : Scoping Review	JMIR. Journal of medical internet research			2025	4	3

Archive ouverte UNIGE

Reichenpfader, Daniel