Human-Level Extraction of Modified Rankin Scale Scores from Real-World Neurosurgical Clinical Notes Using a Locally Deployed, Quantized Large Language Model

Article Article scientifique Article Human-Level Extraction of Modified Rankin Scale Scores from Real-World Neurosurgical Clinical Notes Using a Locally Deployed, Quantized Large Language Model eng 2026-08-16 author Bornet Alban 0000-0002-7266-627X 402515 Département de radiologie et informatique médicale (RADIO) author Sandralegar Abiram 0009-0007-2724-6632 907767 Département des neurosciences cliniques (OTONO) author Yazdani Anthony 0000-0003-3309-6128 838059 Département de radiologie et informatique médicale (RADIO) author Benguettat Elias Adam 873810 author Righini Michael Francis 907050 author Ravarelli Feres author Constanthin Paul 0000-0003-1914-8876 670446 author Lave Alexandre 807585 author Haemmerli Julien 0000-0001-5998-8541 659636 Département des neurosciences cliniques (OTONO) author Janssen Insa 948830 author Issa Jelia 0009-0004-4151-3483 991085 Structure enseignement (ENSI) author Qaderdan Elham 0009-0004-8583-0333 984128 author Godin Ethan Guillaume 414967 author Bouhalassa Nasser 0009-0001-4112-6019 969133 Structure enseignement (ENSI) author Schaller Karl Lothard 0009-0001-8477-5553 658606 Département des neurosciences cliniques (OTONO) author Bijlenga Philippe Alexandre Pierre 0000-0002-3586-2757 104899 Département des neurosciences cliniques (OTONO) author Teodoro Douglas 0000-0001-6238-4503 682380 Département de radiologie et informatique médicale (RADIO) Douglas Henrique Teodoro 248 10.3390/make8080248 2504-4990 https://www.mdpi.com/2504-4990/8/8/248 <p>This study conducts a clinical evaluation of a secure, locally deployed, quantized large language model (LLM) for automating modified Rankin Scale (mRS) score extraction from unstructured neurosurgical notes. We retrospectively selected 103 authentic clinical letters (2007–2025) from aneurysm patients at a tertiary neurosurgical centre. To comply with data privacy constraints, an open-source reasoning LLM (Qwen3-32B) with 4-bit quantization was deployed entirely on-premises. The LLM extracted mRS scores using a zero-shot approach with custom logits processors to enforce strict JSON formatting. Performance was compared to a reference standard (attending neurosurgeons’ consensus) and parallel scoring by medical residents and students. The LLM achieved excellent agreement with the attending consensus (QWK 0.95), matching the reliability of medical students (QWK 0.95) and residents (QWK 0.93). Exact agreement was 75%, and agreement within ±1 mRS point was 96%. Bayesian analysis strongly supported statistical equivalence between the model and human raters. The computationally optimized LLM demonstrated human-level classification reliability without task-specific fine-tuning. This approach successfully addresses key patient data privacy barriers and the formatting inconsistencies typical of open-ended generative models. Securely deploying a general-purpose, quantized LLM provides a scalable pathway for extracting functional outcomes and supports FAIR-aligned data systems.</p> Artificial intelligence Large language models Information extraction Neurosurgery Modified Rankin Scale Dewey 616.8 Dewey 616.0757 Dewey 616 Swiss National Science Foundation AIIDKIT: Artificial Intelligence for Improved Infectious Diseases Outcomes in Kidney Transplant Recipients 229203 CH European Commission Towards GEMINI: A Generation of Multi-scale Digital Twins of Ischaemic and Haemorrhagic Stroke Patients 101136438 GEMINI EU HE 61 Département des neurosciences cliniques 226 62 Département de radiologie et informatique médicale 227 56 Département de médecine 221 false 1379 DS4DH - Data Science for Digital Health 1035 513 Groupe Schaller Karl Lothard (neurochirurgie) 851 https://huggingface.co/unsloth/Qwen3-32B-GGUF https://github.com/ds4dh/gemini Machine learning and knowledge extraction 8 8 7cdde392-a1f1-47ad-8235-cc3022ceae4a Article (Published version) Public make-08-00248.pdf 3534030 application/pdf CC BY-4.0 ada204fa9ada1acf633f3db117f009d6d424a2c69972c328a92112f995ea75fb f171ab12-9cfd-4aae-b57c-b7145df1a949 Supplemental data Public make-4444871-supplementary.pdf 128700 application/pdf e6a5b00ac6e12281d540ae68fe0a720a90efc1308f054c44a7d0691f5c3f1531 openAccess Faculté de médecine / Section de médecine clinique / Département des neurosciences cliniques Faculté de médecine / Section de médecine clinique / Département de radiologie et informatique médicale Faculté de médecine / Section de médecine clinique / Département de médecine

Title	Published in	Access level	OA Policy	Year	Views	Downloads

Human-Level Extraction of Modified Rankin Scale Scores from Real-World Neurosurgical Clinical Notes Using a Locally Deployed, Quantized Large Language Model	Machine learning and knowledge extraction			2026	1	0

Archive ouverte UNIGE

Qaderdan, Elham