Proceedings chapter
Open access

BioBERTpt - A Portuguese Neural Language Model for Clinical Named Entity Recognition

Presented at Online, 19 November 2020
PublisherAssociation for Computational Linguistics
  • ClinicalNLP | EMNLP
Publication date2020-11

With the growing number of electronic health record data, clinical NLP tasks have become increasingly relevant to unlock valuable information from unstructured clinical text. Although the performance of downstream NLP tasks, such as named-entity recognition (NER), in English corpus has recently improved by contextualised language models, less research is available for clinical texts in low resource languages. Our goal is to assess a deep contextual embedding model for Portuguese, so called BioBERTpt, to support clinical and biomedical NER. We transfer learned information encoded in a multilingual-BERT model to a corpora of clinical narratives and biomedical-scientific papers in Brazilian Portuguese. To evaluate the performance of BioBERTpt, we ran NER experiments on two annotated corpora containing clinical narratives and compared the results with existing BERT models. Our in-domain model outperformed the baseline model in F1-score by 2.72%, achieving higher performance in 11 out of 13 assessed entities. We demonstrate that enriching contextual embedding models with domain literature can play an important role in improving performance for specific NLP tasks. The transfer learning process enhanced the Portuguese biomedical NER model by reducing the necessity of labeled data and the demand for retraining a whole new model.

Affiliation Not a UNIGE publication
Citation (ISO format)
TERUMI RUBEL SCHNEIDER, Elisa et al. BioBERTpt - A Portuguese Neural Language Model for Clinical Named Entity Recognition. In: Proceedings of the 3rd Clinical Natural Language Processing Workshop. Online. [s.l.] : Association for Computational Linguistics, 2020. p. 65–72. (ClinicalNLP | EMNLP) doi: 10.18653/v1/2020.clinicalnlp-1.7
Main files (1)
Proceedings chapter (Published version)

Technical informations

Creation02/03/2022 10:40:00 AM
First validation02/03/2022 10:40:00 AM
Update time03/16/2023 2:52:11 AM
Status update03/16/2023 2:52:10 AM
Last indexation05/06/2024 10:23:29 AM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack