Conference proceedings
Open access

Behavioral Data Categorization for Transformers-based Models in Digital Health

Number of pages4
Publication date2022-09-27

Transformers are recent deep learning (DL) models used to capture the dependence between parts of sequential data. While their potential was already demonstrated in the natural language processing (NLP) domain, emerging research shows transformers can also be an adequate modeling approach to relate longitudinal multi-featured continuous behavioral data to future health outcomes. As transformers-based predictions are based on a domain lexicon, the use of categories, commonly used in specialized areas to cluster values, is the likely way to compose lexica. However, the number of categories may influence the transformer prediction accuracy, mainly when the categorization process creates imbalanced datasets, or the search space is very restricted to generate optimal feasible solutions. This paper analyzes the relationship between models’ accuracy and the sparsity of behavioral data categories that compose the lexicon. This analysis relies on a case example that uses mQoL- Transformer to model the influence of physical activity behavior on sleep health. Results show that the number of categories shall be treated as a further transformer’s hyperparameter, which can balance the literature-based categorization and optimization aspects. Thus, DL processes could also obtain similar accuracies compared to traditional approaches, such as long short-term memory, when used to process short behavioral data sequence

  • Deep learning
  • Transformers
  • Human behavior
  • Recommendations
  • Behavior informatics
  • Digital biomarkers
Citation (ISO format)
DE ALBUQUERQUE SIEBRA, Clauirton, ALMEIDA MATIAS, Igor Alexandre, WAC, Katarzyna. Behavioral Data Categorization for Transformers-based Models in Digital Health. [s.l.] : IEEE, 2022. doi: 10.1109/bhi56158.2022.9926938
Main files (1)
Proceedings (Published version)

Technical informations

Creation01/06/2023 4:45:00 PM
First validation01/06/2023 4:45:00 PM
Update time03/16/2023 10:20:36 AM
Status update03/16/2023 10:20:36 AM
Last indexation08/31/2023 10:21:24 AM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack