Interpretation of a deep analysis of speech imagery features extracted by a capsule neural network

Speech imagery has been successfully employed in developing Brain-Computer Interfaces because it is a novel mental strategy that generates brain activity more intuitively than evoked potentials or motor imagery. There are many methods to analyze speech imagery signals, but those based on deep neural networks achieve the best results. However, more research is necessary to understand the properties and features that describe imagined phonemes and words. In this paper, we analyze the statistical properties of speech imagery EEG signals from the KaraOne dataset to design a method that classifies imagined phonemes and words. With this analysis, we propose a Capsule Neural Network that categorizes speech imagery patterns into bilabial, nasal, consonant-vocal, and vowels/iy/ and/uw/. The method is called Capsules for Speech Imagery Analysis (CapsK-SI). The input of CapsK-SI is a set of statistical features of EEG speech imagery signals. The architecture of the Capsule Neural Network is composed of a convolution layer, a primary capsule layer, and a class capsule layer. The average accuracy reached is 90.88%+/- 7 for bilabial, 90.15%+/- 8 for nasal, 94.02%+/- 6 for consonant- vowel, 89.70%+/- 8 for word-phoneme, 94.33%+/- for/iy/ vowel and, 94.21%+/- 3 for/uw/ vowel detection. Finally, with the activity vectors of the CapsK-SI capsules, we generated brain maps to represent brain activity in the production of bilabial, nasal, and consonant-vocal signals.

Automated summary

In this paper, a method called CapsK-SI, which uses statistical features and a Capsule Neural Network, is proposed to classify imagined phonemes and words in speech imagery signals. The accuracy of the classification is high for different categories, reaching over 90% for most cases. Furthermore, brain maps are generated to represent brain activity in the production of certain speech signals.