We first worked on identifying factors (e.g. users’ personality, cognitive abilities or neurophysiological patterns) related to BCI user performance and learning. We have proposed new ways to measure users’ BCI skills, independently of how good the machine is. We have actually identified different types of BCI user learning, associated to different changes in users’ brain activity. We also identified that BCI experimenters, who train BCI users, actually do influence how users learn and perform. Finally, by using Artificial Intelligence (AI) techniques, we could reveal how some users’ personality traits, notably how anxious they are, or their brain activity patterns at rest, could predict how well they will control a BCI. We have identified new such patterns and proposed AI models using them to accurately predict users’ future BCI control performance.
To refine our models, we also worked on estimating users’ mental states (e.g. their mental efforts) during training. We thus designed new AI algorithms to estimate users’ cognitive, affective and motivational states from their brain (electroencephalography – EEG) and physiological (e.g. heart rate) signals. Such algorithms could recognize low or high mental efforts, and negative or positive emotions from EEG, with a better reliability than existing methods. We also conducted experiments to induce various types of attention, e.g. sustained attention or split attention, which our algorithms could recognize from EEG signals. Finally, we also studied curiosity, a mental state that is key to ensure learning. With a new experiment inducing users into various curiosity states (e.g. bored versus curious) and the AI algorithms above, we could discriminate low versus high curiosity from both EEG and physiological signals.
We also worked on optimizing BCI user training. We worked on user feedback, i.e. the information provided about what the BCI has recognized, so that users can learn better. We proposed new BCI feedbacks, including a feedback based on vibrotactile and realistic visual feedback, and a social feedback. For the latter, we designed the first artificial learning companion for BCI, which provides users with support or advices depending on their performance and learning. It can improve performance for the users who prefer to work in group. Finally, we explored biased feedback (making users believe they performed better or worse than what they really did), and showed that it could improve BCI performances and learning if the bias is personalized to each user’s traits, states and skills. We thus proposed an algorithm to do that automatically.
We also designed new robust and adaptive AI tools to deal with the changing users’ EEG signals. Such tools are robust to noise, can identify EEG sensors providing the most stable signals or update their parameters as new data become available. Our studies showed the gains they all offered, notably when we trained a tetraplegic user to control a BCI over 3 months. They indeed enabled him to increase his BCI control performance dramaticall.
Overall, this work led to the first theory and principles of BCI user training, able to explain who can use such BCIs, what kind of learning is involved, or how to optimize this training.
This work was disseminated with scientific publications (20+ journals and 25+ conferences) and talks (40+). Moreover, many of the designed AI tools and BCI feedbacks were shared open-source, as part of the OpenViBE and BioPyC software. EEG data collected during the project were also shared as open data.
