Doctoral thesis

Imagined speech decoding and training a brain computer interface

ContributorsBhadra, Kinkini
Number of pages117
Imprimatur date2023
Defense date2023

Disruption in speech production can have a devastating effect on patients and their caregivers in terms of quality of life. John Fiske claims that without communication, no culture could survive. In the same line, socio-linguists highlight communication's role in building relationships, sharing emotions, and expressing individual identity. Consequently, language disorders affecting speech can harm patients in two main ways: eroding existing relationships due to limited communication, and fostering self-esteem related withdrawal from social connections. Brain-computer interfaces (BCI) have the potential to bring back communication in patients with language disorders by providing alternative communication channels, e.g. real-time decoding of speech directly from the remaining intact brain areas, or by rehabilitation solutions e.g. exploiting neural plasticity mechanisms using BCI-feedback. Although recent studies confirm the possibility of decoding imagined speech from pre-recorded intracranial neurophysiological signals, current efforts focus on collecting vast amounts of data to train classifiers, rather than exploring how the brain can adapt to improve BCI control. In addition, invasive studies done so far to decode covert speech involve patients with epilepsy who are implanted with electrodes as a part of a pre-surgical procedure. To be able to initiate invasive protocols for speech rehabilitation, it is important to identify a potential candidate who can benefit the most from this technology as BCI feedback control suffers from high inter-individual variabilities. In the research work presented here we addressed speech-BCI controllability from a neurophysiological point of view by training 15 healthy participants to operate a BCI based on electroencephalography (EEG) signals during a binary syllable imagery task for 5 consecutive days. Alongside we also collected behavioral data related to mental imagery abilities and sustained attention to find out the possible correlation of the same with BCI performance. In the BCI task, we investigated whether controlling a BCI feedback can be improved with training and we characterized the evolution of the underlying neural mechanisms, both in terms of changes in EEG power during the BCI feedback control while imagining the syllables and in the neural features used for real-time classification. We found a significant linear improvement in BCI control performance across the days of training. Related neural and classifier features reorganize throughout this training period as we found the dynamic involvement of both low and high-frequency activity alongside spatial tuning. In the two behavioral tasks, we investigated if attention and mental imagery skills contribute to and correlate with BCI performance. Results show a significant impact of training over the 5 days, however, this effect is not significantly correlated or can predict BCI performance. Overall, we found that neural features can adapt to improve covert-speech BCI performance and that mental imagery skills and attention are necessary to carry out this task, but are not directly able to predict BCI performance. Future BCI applications will require a combination of machine and human learning to reach optimal controllability. Improvements on the user side could effectively compensate for the limited success in accurately decoding imagined speech as compared to attempted speech.

  • Brain-computer interface
  • Speech decoding
  • BCI training
  • Covert speech
  • SART
  • Mental chronometry
Citation (ISO format)
BHADRA, Kinkini. Imagined speech decoding and training a brain computer interface. 2023. doi: 10.13097/archive-ouverte/unige:175761
Main files (1)
Secondary files (1)

Technical informations

Creation03/18/2024 7:02:33 PM
First validation03/19/2024 2:46:50 PM
Update time03/19/2024 2:46:50 PM
Status update03/19/2024 2:46:50 PM
Last indexation03/19/2024 2:47:09 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack