Recognition of patterns of motor activity by a neural network based on continuous optical tomography fNIRS data
DOI:
https://doi.org/10.52575/2687-0932-2021-48-4-735-746Keywords:
brain-computer interface, residual neural network, optical tomography, recurrent neural network, convolutional neural network, long short-term memory neural networkAbstract
The article is devoted to the development and testing of the architecture of a neural network for the classification of patterns of motor activity according to the input data from an optical tomograph. The aim of this work is to create a neural network capable of searching for patterns of motor activity in a continuously arriving signal from equipment. The work analyzed three types of neural network architectures NN_LSTM, NN_ConvLST, NN_ResNet, each of which represents an original approach for finding logic in time series data. The dataset of neurophysiological signals obtained from an optical tomograph was prepared for approbation and qualitative assessment of neural networks were carried out. The plan of the experiment was developed taking into account the specifics of the physical foundations of the received signal, for example, the lag and inertia of oxy- and deoxy-hemoglobin in the blood. The experiment timing allows to unambiguously identify events during the experiment in order to identify the fact of the execution of target commands by the experiment subject. The training of neural network models was carried out in two target classes (compress and unclench the hand). The next stage was the training of models in three classes (a class of other motor activity was added). As a result, the best accuracy was achieved for the NN_ResNet model (accuracy 91%). In this way, obtained the deep learning neural network model capable of identifying motor patterns of brain activity according to fNIRS-data, in which an external signal is recorded in addition to target commands.
Acknowledgements: research is supported by the RFBR grant 20-08-01178.
Downloads
References
Гагарин А.П. 1988. Бугера – Ламберта – Бера закон. В кн.: Физическая энциклопедия (в 5 т.). М., Советская энциклопедия: 232–233.
Добро пожаловать в Colaboratory! 2021. URL: https://colab.research.google.com/?utm_ source=scs-index (дата обращения 13.10.2021).
He K., Zhang X., Ren S., Sun J. 2015. Deep Residual Learning for Image Recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI:10.1109/cvpr.2016.90: 12p.
Ma T, Chen W, Li X, Xia Y, Zhu X, He S. 2021. fNIRS Signal Classification Based on Deep Learning in Rock-Paper-Scissors Imagery Task. Applied Sciences, 11(11): 18p.
Naseer N., Hong K-S. 2015. fNIRS-based brain-computer interfaces: a review. Frontiers in Human Neuroscience, 9: 15p.
NIRx fNIRS Technology & Service Overview. 2015. URL: https://nirx.net (дата обращения 13.10.2021).
S. Hochreiter; J. Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8): 1735-1780.
Shi X., Chen Z., Wang H., Yeung D.-Y., Wong W., Woo W. 2015. Convolutional LSTM Network: a machine learning approach for precipitation nowcasting. NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems, 1: 802–810.
Sung C. Jun. 2018. A Ternary Hybrid EEG-NIRS Brain-Computer Interface for the Classification of Brain Activation Patterns during Mental Arithmetic, Motor Imagery, and Idle State. Frontiers in Neuroinformatics, 12A(5): 9p.
TensorFlow Federated: машинное обучение на децентрализованных данных. 2020. URL: https://www.tensorflow.org/federated (дата обращения 14.10.2021).
Tomita Y., Vialatte F. B., Dreyfus G., Mitsukura Y., Bakardjian H., Cichocki A. 2014. Bimodal BCI using simultaneously NIRS and EEG. IEEE Trans. Biomed Eng., 61: 1274–1284.
Wang Y., WuReal D. 2018. Real-time fMRI-based Brain Computer Interface: A Review. IEEE. Computer Science, 1: 10p.
Abstract views: 104
Share
Published
How to Cite
Issue
Section
Copyright (c) 2021 ECONOMICS. INFORMATION TECHNOLOGIES
This work is licensed under a Creative Commons Attribution 4.0 International License.
