Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet_Journal of Biomedical Engineering

Authors：

 FENG Guohong , ZHENG Xiao , ZHANG Bin , WANG Hongen

College of Mechanical and Electrical Engineering, Northeast Forestry University, Harbin 150040, P. R. China;

Corresponding?author：

FENG Guohong, Email: fgh_1980@126.com

Keywords：

Electroencephalogram; Emotion recognition; Independent component analysis; Recurrence plot; EfficientNet; Attention mechanism

DOI：

10.7507/1001-5515.202406029

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

To accurately capture and effectively integrate the spatiotemporal features of electroencephalogram (EEG) signals for the purpose of improving the accuracy of EEG-based emotion recognition, this paper proposes a new method combining independent component analysis-recurrence plot with an improved EfficientNet version 2 (EfficientNetV2). First, independent component analysis is used to extract independent components containing spatial information from key channels of the EEG signals. These components are then converted into two-dimensional images using recurrence plot to better extract emotional features from the temporal information. Finally, the two-dimensional images are input into an improved EfficientNetV2, which incorporates a global attention mechanism and a triplet attention mechanism, and the emotion classification is output by the fully connected layer. To validate the effectiveness of the proposed method, this study conducts comparative experiments, channel selection experiments and ablation experiments based on the Shanghai Jiao Tong University Emotion Electroencephalogram Dataset (SEED). The results demonstrate that the average recognition accuracy of our method is 96.77%, which is significantly superior to existing methods, offering a novel perspective for research on EEG-based emotion recognition.

Citation： FENG Guohong, ZHENG Xiao, ZHANG Bin, WANG Hongen. Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet. Journal of Biomedical Engineering, 2024, 41(6): 1103-1109. doi: 10.7507/1001-5515.202406029 Copy

1.	Hamada M, Zaidan B B, Zaidan A A. A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. Journal of Medical Systems, 2018, 42(9): 162.
2.	Liu S, Wang Z, An Y, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network. Knowledge-Based Systems, 2023, 265: 110372.
3.	Li M, Xu H, Liu X, et al. Emotion recognition from multichannel EEG signals using K-nearest neighbor classification. Technology and Health Care, 2018, 26(S1): 509-519.
4.	Yan C, Chang X, Li Z, et al. ZeroNAS: Differentiable generative adversarial networks search for zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9733-9740.
5.	Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, 2014, 5(3): 327-339.
6.	Duan R N, Zhu J Y, Lu B L. Differential entropy feature for EEG-based emotion classification//2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego: IEEE, 2013: 81-84.
7.	Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation, 2018, 10: 368-380.
8.	Cheng C, Yu Z, Zhang Y, et al. Hybrid network using dynamic graph convolution and temporal self-attention for EEG-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems, 2023: 2162-2388.
9.	Rahman M A, Anjum A, Milu M M H, et al. Emotion recognition from EEG-based relative power spectral topography using convolutional neural network. Array, 2021, 11: 100072.
10.	Wang Z, Wang Y, Zhang J, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
11.	王偉, 周建華, 劉紫恒, 等. 基于時空Inception殘差注意力網絡的腦電情緒識別. 重慶郵電大學學報(自然科學版), 2024, 36(1): 68-75.
12.	崔發昌. 時-頻-空多維融合的腦電情緒分類方法. 南京: 南京郵電大學, 2023.
13.	宋昊, 徐頌, 劉國明, 等. 基于獨立成分分析的非侵入式腦-機接口眼電偽跡自動去除算法. 生物醫學工程學雜志, 2022, 39(6): 1074-1081.
14.	Jiang W, Zhang D, Ling L, et al. Time series classification based on image transformation using feature fusion strategy. Neural Processing Letters, 2022, 54(5): 3727-3748.
15.	馮國紅, 王宏恩, 刁鵬飛, 等. 小樣本下基于遞歸圖和遷移學習的軸承故障診斷. 機床與液壓, 2024, 52(16): 240-248.
16.	Al Moteri M, Mahesh T R, Thakur A, et al. Enhancing accessibility for improved diagnosis with modified EfficientNetV2-S and cyclic learning rate strategy in women with disabilities and breast cancer. Frontiers in Medicine, 2024, 11: 1373244.
17.	Mouhcine K, Zrira N, Elafi I, et al. STEFF: spatio-temporal EfficientNet for dynamic texture classification in outdoor scenes. Heliyon, 2024, 10(3): e25360.
18.	Zhao Y, Ju Z, Sun T, et al. TGC-YOLOv5: an enhanced YOLOv5 drone detection model based on transformer, GAM & CA attention mechanism. Drones, 2023, 7(7): 446.
19.	仲崢迪, 屠良平, 馮雪琦, 等. EfficientNetV2-S-Triplet 7: 一種改進的星系形態學分類算法. 天文學報, 2024, 65(2): 55-68.
20.	Cui L, Dong Z, Xu H, et al. Triplet attention-enhanced residual tree-inspired decision network: a hierarchical fault diagnosis model for unbalanced bearing datasets. Advanced Engineering Informatics, 2024, 59: 102322.
21.	聶聃, 王曉韡, 段若男, 等. 基于腦電的情緒識別研究綜述. 中國生物醫學工程學報, 2012, 31(4): 595-606.
22.	Zheng W L, Lu B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
23.	焦凱強, 王湖斐, 郭茂田. 腦電信號中的頻譜不對稱指數特征與情緒識別. 科學技術與工程, 2018, 18(17): 145-149.
24.	王雪蒙, 郭濱, 馬欣. 基于優化變分模態分解的腦電情緒識別. 計算機應用與軟件, 2024, 41(2): 80-85,177.
25.	Zhang G, Davoodnia V, Sepas-Moghaddam A, et al. Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sensors Journal, 2019, 20(6): 3113-3122.
26.	張冰雪, 李文楷. 少量通道腦電信號的實時情緒分類模型. 小型微型計算機系統, 2024, 45(2): 271-277.
27.	戴紫玉,馬玉良, 高云園, 等. 基于多尺度卷積核CNN的腦電情緒識別. 傳感技術學報, 2021, 34(4): 496-503.
28.	Yang C J, Li W C, Wan M T, et al. Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network//2021 IEEE International Symposium on Circuits and Systems (ISCAS). Daegu: IEEE, 2021: 1-5.
29.	谷學靜, 劉佳, 郭宇承, 等. 采用多尺度多路混合注意力機制的腦電情緒識別方法. 計算機工程與應用, 2024, 60(19): 130-138.
30.	劉柯, 張孝, 李沛洋, 等. 基于腦功能網絡和共空間模式分析的腦電情緒識別. 計算機應用研究, 2021, 38(5): 1344-1349.
31.	高越, 傅湘玲, 歐陽天雄, 等. 基于時空自適應圖卷積神經網絡的腦電信號情緒識別. 計算機科學, 2022, 49(4): 30-36.
32.	李奇, 常立娜, 武巖, 等. 基于深層圖卷積的EEG情緒識別方法研究. 電子測量技術, 2024, 47(4): 18-22.
33.	謝松云, 雷凌俊, 孫江, 等. 基于IWOA-ELM算法的腦電情緒識別方法研究. 生物醫學工程學雜志, 2024, 41(1): 1-8.

1. Hamada M, Zaidan B B, Zaidan A A. A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. Journal of Medical Systems, 2018, 42(9): 162.
2. Liu S, Wang Z, An Y, et al. EEG emotion recognition based on the attention mechanism and pre-trained convolution capsule network. Knowledge-Based Systems, 2023, 265: 110372.
3. Li M, Xu H, Liu X, et al. Emotion recognition from multichannel EEG signals using K-nearest neighbor classification. Technology and Health Care, 2018, 26(S1): 509-519.
4. Yan C, Chang X, Li Z, et al. ZeroNAS: Differentiable generative adversarial networks search for zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(12): 9733-9740.
5. Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective Computing, 2014, 5(3): 327-339.
6. Duan R N, Zhu J Y, Lu B L. Differential entropy feature for EEG-based emotion classification//2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), San Diego: IEEE, 2013: 81-84.
7. Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation, 2018, 10: 368-380.
8. Cheng C, Yu Z, Zhang Y, et al. Hybrid network using dynamic graph convolution and temporal self-attention for EEG-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems, 2023: 2162-2388.
9. Rahman M A, Anjum A, Milu M M H, et al. Emotion recognition from EEG-based relative power spectral topography using convolutional neural network. Array, 2021, 11: 100072.
10. Wang Z, Wang Y, Zhang J, et al. Spatial-temporal feature fusion neural network for EEG-based emotion recognition. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
11. 王偉, 周建華, 劉紫恒, 等. 基于時空Inception殘差注意力網絡的腦電情緒識別. 重慶郵電大學學報(自然科學版), 2024, 36(1): 68-75.
12. 崔發昌. 時-頻-空多維融合的腦電情緒分類方法. 南京: 南京郵電大學, 2023.
13. 宋昊, 徐頌, 劉國明, 等. 基于獨立成分分析的非侵入式腦-機接口眼電偽跡自動去除算法. 生物醫學工程學雜志, 2022, 39(6): 1074-1081.
14. Jiang W, Zhang D, Ling L, et al. Time series classification based on image transformation using feature fusion strategy. Neural Processing Letters, 2022, 54(5): 3727-3748.
15. 馮國紅, 王宏恩, 刁鵬飛, 等. 小樣本下基于遞歸圖和遷移學習的軸承故障診斷. 機床與液壓, 2024, 52(16): 240-248.
16. Al Moteri M, Mahesh T R, Thakur A, et al. Enhancing accessibility for improved diagnosis with modified EfficientNetV2-S and cyclic learning rate strategy in women with disabilities and breast cancer. Frontiers in Medicine, 2024, 11: 1373244.
17. Mouhcine K, Zrira N, Elafi I, et al. STEFF: spatio-temporal EfficientNet for dynamic texture classification in outdoor scenes. Heliyon, 2024, 10(3): e25360.
18. Zhao Y, Ju Z, Sun T, et al. TGC-YOLOv5: an enhanced YOLOv5 drone detection model based on transformer, GAM & CA attention mechanism. Drones, 2023, 7(7): 446.
19. 仲崢迪, 屠良平, 馮雪琦, 等. EfficientNetV2-S-Triplet 7: 一種改進的星系形態學分類算法. 天文學報, 2024, 65(2): 55-68.
20. Cui L, Dong Z, Xu H, et al. Triplet attention-enhanced residual tree-inspired decision network: a hierarchical fault diagnosis model for unbalanced bearing datasets. Advanced Engineering Informatics, 2024, 59: 102322.
21. 聶聃, 王曉韡, 段若男, 等. 基于腦電的情緒識別研究綜述. 中國生物醫學工程學報, 2012, 31(4): 595-606.
22. Zheng W L, Lu B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Transactions on Autonomous Mental Development, 2015, 7(3): 162-175.
23. 焦凱強, 王湖斐, 郭茂田. 腦電信號中的頻譜不對稱指數特征與情緒識別. 科學技術與工程, 2018, 18(17): 145-149.
24. 王雪蒙, 郭濱, 馬欣. 基于優化變分模態分解的腦電情緒識別. 計算機應用與軟件, 2024, 41(2): 80-85,177.
25. Zhang G, Davoodnia V, Sepas-Moghaddam A, et al. Classification of hand movements from EEG using a deep attention-based LSTM network. IEEE Sensors Journal, 2019, 20(6): 3113-3122.
26. 張冰雪, 李文楷. 少量通道腦電信號的實時情緒分類模型. 小型微型計算機系統, 2024, 45(2): 271-277.
27. 戴紫玉,馬玉良, 高云園, 等. 基于多尺度卷積核CNN的腦電情緒識別. 傳感技術學報, 2021, 34(4): 496-503.
28. Yang C J, Li W C, Wan M T, et al. Real-time EEG-based affective computing using on-chip learning long-term recurrent convolutional network//2021 IEEE International Symposium on Circuits and Systems (ISCAS). Daegu: IEEE, 2021: 1-5.
29. 谷學靜, 劉佳, 郭宇承, 等. 采用多尺度多路混合注意力機制的腦電情緒識別方法. 計算機工程與應用, 2024, 60(19): 130-138.
30. 劉柯, 張孝, 李沛洋, 等. 基于腦功能網絡和共空間模式分析的腦電情緒識別. 計算機應用研究, 2021, 38(5): 1344-1349.
31. 高越, 傅湘玲, 歐陽天雄, 等. 基于時空自適應圖卷積神經網絡的腦電信號情緒識別. 計算機科學, 2022, 49(4): 30-36.
32. 李奇, 常立娜, 武巖, 等. 基于深層圖卷積的EEG情緒識別方法研究. 電子測量技術, 2024, 47(4): 18-22.
33. 謝松云, 雷凌俊, 孫江, 等. 基于IWOA-ELM算法的腦電情緒識別方法研究. 生物醫學工程學雜志, 2024, 41(1): 1-8.

Journal of Biomedical Engineering

Research on emotion recognition in electroencephalogram based on independent component analysis-recurrence plot and improved EfficientNet

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content