Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel_Journal of Biomedical Engineering

Authors：

XU Liukai ^1,2 , ZHANG Keqin ³ , XU Zhaohong ³ ,  YANG Genke ^1,2

1. Ningbo Artificial Intelligence Institute of Shanghai Jiao Tong University, Ningbo, Zhejiang 315000, P.R.China;
2. Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, P.R.China;
3. Ningbo Industrial Internet Institute, Ningbo, Zhejiang 315000, P.R.China;

Corresponding?author：

YANG Genke, Email: gkyang@sjtu.edu.cn

Keywords：

surface electromyography; gesture recognition; energy kernel; convolutional neural network

DOI：

10.7507/1001-5515.202010080

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Surface electromyography (sEMG) is a weak signal which is non-stationary and non-periodic. The sEMG classification methods based on time domain and frequency domain features have low recognition rate and poor stability. Based on the modeling and analysis of sEMG energy kernel, this paper proposes a new method to recognize human gestures utilizing convolutional neural network (CNN) and phase portrait of sEMG energy kernel. Firstly, the matrix counting method is used to process the sEMG energy kernel phase portrait into a grayscale image. Secondly, the grayscale image is preprocessed by moving average method. Finally, CNN is used to recognize sEMG of gestures. Experiments on gesture sEMG signal data set show that the effectiveness of the recognition framework and the recognition method of CNN combined with the energy kernel phase portrait have obvious advantages in recognition accuracy and computational efficiency over the area extraction methods. The algorithm in this paper provides a new feasible method for sEMG signal modeling analysis and real-time identification.

Citation： XU Liukai, ZHANG Keqin, XU Zhaohong, YANG Genke. Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel. Journal of Biomedical Engineering, 2021, 38(4): 621-629. doi: 10.7507/1001-5515.202010080 Copy

1.	丁其川, 熊安斌, 趙新剛, 等. 基于表面肌電的運動意圖識別方法研究及應用綜述. 自動化學報, 2016, 42(1): 13-25.
2.	Meng Qingyun, Meng Qiaoling, Yu Hongliu, et al. A survey on sEMG control strategies of wearable hand exoskeleton for rehabilitation//2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), Wuhan: IEEE, 2017: 165-169.
3.	Phukpattaranont P, Thongpanja S, Anam K, et al. Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal. Med Biol Eng Comput, 2018, 56(12): 2259-2271.
4.	丁帥, 王亮. 基于塊稀疏貝葉斯學習的肌電信號特征提取. 儀器儀表學報, 2014, 35(12): 2731-2738.
5.	Chen X, Yin Y, Fan Y. EMG oscillator model-based energy kernel method for haracterizing muscle intrinsic property under isometric contraction. Chin Sci Bull, 2014, 59(14): 1556-1567.
6.	Chen X, Zeng Y, Yin Y. Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 577-588.
7.	Zeng Y, Yang J, Peng C, et al. Evolving gaussian process autoregression based learning of human motion intent using improved energy kernel method of EMG. IEEE Trans Biomed Eng, 2019, 66(9): 2556-2565.
8.	石欣, 朱家慶, 秦鵬杰, 等. 基于改進能量核的下肢表面肌電信號特征提取方法. 儀器儀表學報, 2020, 41(1): 121-128.
9.	Yang Kuo, Zhang Zhen. Real-time pattern recognition for hand gesture based on ANN and surface EMG//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 799-802.
10.	Abdel-Hamid O, Mohamed A, Jiang H, et al. Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Language Process, 2014, 22(10): 1533-1545.
11.	Pinzón-Arenas J O, Jiménez-Moreno R, Herrera-Benavides J E. Convolutional neural network for hand gesture recognition using 8 different EMG signals//2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA). Bucaramanga: IEEE, 2019: 1-5.
12.	Atzori M, Cognolato M, Müller H. Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front Neurorobot, 2016, 10: 9.
13.	Geng W, Du Y, Jin W, et al. Gesture recognition by instantaneous surface EMG images. Sci Rep, 2016, 6: 36571.
14.	Wei W, Wong Y, Du Y, et al. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit Lett, 2019, 119: 131-138.
15.	Zhai X, Jelfs B, Chan R M, et al. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci, 2017, 11: 379.
16.	He Yunan, Fukuda O, Bu Nan, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu: IEEE, 2018: 5636-5639.
17.	Wu Yuheng, Zheng Bin, Zhao Yongting. Dynamic gesture recognition based on LSTM-CNN//2018 Chinese Automation Congress (CAC), Xi’an: IEEE, 2018: 2446-2450.
18.	Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation--a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
19.	Du Y C, Lin C H, Shyu L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis. Expert Syst Appl, 2010, 37(6): 4283-4291.
20.	McComas A J, Mrozek K. The electrical properties of muscle fiber membranes in dystrophia myotonica and myotonia congenita. J Neurol Neurosurg Psychiatry, 1968, 31(5): 441-447.
21.	Trajano G S, Nosaka K, Blazevich A J. Neurophysiological mechanisms underpinning stretch-induced force loss. Sports Med, 2017, 47(8): 1531-1541.
22.	李偉, 楊向東, 陳懇. 基于CNN和RNN聯合網絡的心音自動分類. 計算機工程與設計, 2020, 41(1): 46-51.
23.	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.

1. 丁其川, 熊安斌, 趙新剛, 等. 基于表面肌電的運動意圖識別方法研究及應用綜述. 自動化學報, 2016, 42(1): 13-25.
2. Meng Qingyun, Meng Qiaoling, Yu Hongliu, et al. A survey on sEMG control strategies of wearable hand exoskeleton for rehabilitation//2017 2nd Asia-Pacific Conference on Intelligent Robot Systems (ACIRS), Wuhan: IEEE, 2017: 165-169.
3. Phukpattaranont P, Thongpanja S, Anam K, et al. Evaluation of feature extraction techniques and classifiers for finger movement recognition using surface electromyography signal. Med Biol Eng Comput, 2018, 56(12): 2259-2271.
4. 丁帥, 王亮. 基于塊稀疏貝葉斯學習的肌電信號特征提取. 儀器儀表學報, 2014, 35(12): 2731-2738.
5. Chen X, Yin Y, Fan Y. EMG oscillator model-based energy kernel method for haracterizing muscle intrinsic property under isometric contraction. Chin Sci Bull, 2014, 59(14): 1556-1567.
6. Chen X, Zeng Y, Yin Y. Improving the transparency of an exoskeleton knee joint based on the understanding of motor intent using energy kernel method of EMG. IEEE Trans Neural Syst Rehabil Eng, 2017, 25(6): 577-588.
7. Zeng Y, Yang J, Peng C, et al. Evolving gaussian process autoregression based learning of human motion intent using improved energy kernel method of EMG. IEEE Trans Biomed Eng, 2019, 66(9): 2556-2565.
8. 石欣, 朱家慶, 秦鵬杰, 等. 基于改進能量核的下肢表面肌電信號特征提取方法. 儀器儀表學報, 2020, 41(1): 121-128.
9. Yang Kuo, Zhang Zhen. Real-time pattern recognition for hand gesture based on ANN and surface EMG//2019 IEEE 8th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), Chongqing: IEEE, 2019: 799-802.
10. Abdel-Hamid O, Mohamed A, Jiang H, et al. Convolutional neural networks for speech recognition. IEEE/ACM Trans Audio, Speech, Language Process, 2014, 22(10): 1533-1545.
11. Pinzón-Arenas J O, Jiménez-Moreno R, Herrera-Benavides J E. Convolutional neural network for hand gesture recognition using 8 different EMG signals//2019 XXII Symposium on Image, Signal Processing and Artificial Vision (STSIVA). Bucaramanga: IEEE, 2019: 1-5.
12. Atzori M, Cognolato M, Müller H. Deep learning with convolutional neural networks applied to electromyography data: a resource for the classification of movements for prosthetic hands. Front Neurorobot, 2016, 10: 9.
13. Geng W, Du Y, Jin W, et al. Gesture recognition by instantaneous surface EMG images. Sci Rep, 2016, 6: 36571.
14. Wei W, Wong Y, Du Y, et al. A multi-stream convolutional neural network for sEMG-based gesture recognition in muscle-computer interface. Pattern Recognit Lett, 2019, 119: 131-138.
15. Zhai X, Jelfs B, Chan R M, et al. Self-recalibrating surface EMG pattern recognition for neuroprosthesis control based on convolutional neural network. Front Neurosci, 2017, 11: 379.
16. He Yunan, Fukuda O, Bu Nan, et al. Surface EMG pattern recognition using long short-term memory combined with multilayer perceptron//2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu: IEEE, 2018: 5636-5639.
17. Wu Yuheng, Zheng Bin, Zhao Yongting. Dynamic gesture recognition based on LSTM-CNN//2018 Chinese Automation Congress (CAC), Xi’an: IEEE, 2018: 2446-2450.
18. Staudenmann D, Roeleveld K, Stegeman D F, et al. Methodological aspects of SEMG recordings for force estimation--a tutorial and review. J Electromyogr Kinesiol, 2010, 20(3): 375-387.
19. Du Y C, Lin C H, Shyu L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis. Expert Syst Appl, 2010, 37(6): 4283-4291.
20. McComas A J, Mrozek K. The electrical properties of muscle fiber membranes in dystrophia myotonica and myotonia congenita. J Neurol Neurosurg Psychiatry, 1968, 31(5): 441-447.
21. Trajano G S, Nosaka K, Blazevich A J. Neurophysiological mechanisms underpinning stretch-induced force loss. Sports Med, 2017, 47(8): 1531-1541.
22. 李偉, 楊向東, 陳懇. 基于CNN和RNN聯合網絡的心音自動分類. 計算機工程與設計, 2020, 41(1): 46-51.
23. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. Commun ACM, 2017, 60(6): 84-90.

Next Article
Research on enhancement of mental rotation ability based on transcranial direct current stimulation

Journal of Biomedical Engineering

Convolutional neural network human gesture recognition algorithm based on phase portrait of surface electromyography energy kernel

Abstract Full text Figures/Tables Video References Cited by

Next Article

Format

Content