Proceedings chapter
OA Policy
English

Behavioral Data Categorization for Transformers-based Models in Digital Health

Presented atBiomedical and Health Informatics Conference (IEEE BHI 2022) and International Conference on Wearable and Implantable Body Sensor Networks (IEEE BSN 2022), Ioannina, Greece, 27-30 Sept. 2022
PublisherPiscataway, NJ : IEEE
Publication date2022
Abstract

Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While their potential was already demonstrated in the natural language processing (NLP) domain, emerging research shows transformers can also be an adequate modeling approach to relate longitudinal multi-featured continuous behavioral data to future health outcomes. As transformers-based predictions are based on a domain lexicon, the use of categories, commonly used in specialized areas to cluster values, is the likely way to compose lexica. However, the number of categories may influence the transformer prediction accuracy, mainly when the categorization process creates imbalanced datasets, or the search space is very restricted to generate optimal feasible solutions. This paper analyzes the relationship between models’ accuracy and the sparsity of behavioral data categories that compose the lexicon. This analysis relies on a case example that uses mQoL- Transformer to model the influence of physical activity behavior on sleep health. Results show that the number of categories shall be treated as a further transformer’s hyperparameter, which can balance the literature-based categorization and optimization aspects. Thus, DL processes could also obtain similar accuracies compared to traditional approaches, such as long short-term memory, when used to process short behavioral data sequence

Keywords
  • Deep learning
  • Transformers
  • Human behavior
  • Recommendations
  • Behavior informatics
  • Digital biomarkers
Citation (ISO format)
DE ALBUQUERQUE SIEBRA, Clauirton, ALMEIDA MATIAS, Igor Alexandre, WAC, Katarzyna. Behavioral Data Categorization for Transformers-based Models in Digital Health. In: BHI-BSN 2002 Symposium Proceedings. Ioannina, Greece. Piscataway, NJ : IEEE, 2022. doi: 10.1109/bhi56158.2022.9926938
Main files (1)
Proceedings chapter (Published version)
accessLevelPublic
Identifiers
Additional URL for this publicationhttps://ieeexplore.ieee.org/document/9926938/
ISBN978-1-6654-8791-7
230views
258downloads

Technical informations

Creation06/01/2023 16:45:00
First validation06/01/2023 16:45:00
Update time29/04/2025 09:21:45
Status update29/04/2025 09:21:45
Last indexation29/04/2025 09:21:46
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack