Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films_Journal of Biomedical Engineering

Authors：

TIAN Feng ^1,2 , ZHANG Wenrui ¹ ,  LI Yingjie ^3,4

1. Shanghai Film Academy, Shanghai University, Shanghai 200072, P. R. China;
2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai University, Shanghai 200072, P. R. China;
3. College of International Education, Shanghai University, Shanghai 200444, P. R. China;
4. Institute of Biomedical Engineering, School of Life Science, Shanghai University, Shanghai 200444, P. R. China;

Corresponding?author：

LI Yingjie, Email: liyj@i.shu.edu.cn

Keywords：

Non-stereoscopic virtual reality; Stereoscopic virtual reality; Continuous display; Emotional arousal; Electroencephalogram; Neurocinematics

DOI：

10.7507/1001-5515.202101010

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

There are two modes to display panoramic movies in virtual reality (VR) environment: non-stereoscopic mode (2D) and stereoscopic mode (3D). It has not been fully studied whether there are differences in the activation effect between these two continuous display modes on emotional arousal and what characteristics of the related neural activity are. In this paper, we designed a cognitive psychology experiment in order to compare the effects of VR-2D and VR-3D on emotional arousal by analyzing synchronously collected scalp electroencephalogram signals. We used support vector machine (SVM) to verify the neurophysiological differences between the two modes in VR environment. The results showed that compared with VR-2D films, VR-3D films evoked significantly higher electroencephalogram (EEG) power (mainly reflected in α and β activities). The significantly improved β wave power in VR-3D mode showed that 3D vision brought more intense cortical activity, which might lead to higher arousal. At the same time, the more intense α activity in the occipital region of the brain also suggested that VR-3D films might cause higher visual fatigue. By the means of neurocinematics, this paper demonstrates that EEG activity can well reflect the effects of different vision modes on the characteristics of the viewers’ neural activities. The current study provides theoretical support not only for the future exploration of the image language under the VR perspective, but for future VR film shooting methods and human emotion research.

Citation： TIAN Feng, ZHANG Wenrui, LI Yingjie. Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films. Journal of Biomedical Engineering, 2022, 39(1): 56-66. doi: 10.7507/1001-5515.202101010 Copy

1.	趙沁平. 虛擬現實綜述. 中國科學: 信息科學, 2009, 39(1): 2-46.
2.	Bartsch A, Viehoff R. The use of media entertainment and emotional gratification. Procedia Soc Behav Sci, 2010, 5: 2247-2255.
3.	Tussyadiah I P, Wang D, Jung T H, et al. Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tourism Manage, 2018, 66: 140-154.
4.	Diemer J, Alpers G W, Peperkorn H M, et al. The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front Psychol, 2015, 6: 26.
5.	Chao C J, Wu S Y, Yau Y J, et al. Effects of three-dimensional virtual reality and traditional training methods on mental workload and training performance. Hum Factor Ergon Man, 2017, 27(4): 187-196.
6.	徐小萍, 呂健, 金昱潼, 等. 用戶認知驅動的VR自然交互認知負荷研究. 計算機應用研究, 2020, 37(7): 1958-1963.
7.	Montana J I, Matamala-Gomez M, Maisto M, et al. The benefits of emotion regulation interventions in virtual reality for the improvement of wellbeing in adults and older adults: a systematic review. J Clin Med, 2020, 9(2): 500.
8.	Li B J, Bailenson J N, Pines A, et al. A public database of immersive VR videos with corresponding ratings of arousal, valence, and correlations between head movements and self report measures. Front Psychol, 2017, 8: 2116.
9.	Hasson U, Landesman O, Knappmeyer B, et al. Neurocinematics: The neuroscience of film. Projections, 2008, 2(1): 1-26.
10.	Schubring D, Kraus M, Stolz C, et al. Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci, 2020, 10(8): 537.
11.	He L, Li H, Xue T, et al. Am I in the theater?: usability study of live performance based virtual reality// The 24th ACM Symposium. Tokyo: ACM, 2018: 1-11.
12.	Bilgin P, Agres K, Robinson N, et al. A comparative study of mental states in 2D and 3D virtual environments using EEG// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 2833-2838.
13.	Kweon S H, Kweon H J, Kim S, et al. A brain wave research on VR (virtual reality) usage: comparison between VR and 2D video in EEG measurement// International Conference on Applied Human Factors and Ergonomics. Los Angeles: Springer, Cham, 2017: 194-203.
14.	Kober S E, Kurzmann J, Neuper C. Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. Int J Psychophysiol, 2011, 83(3): 365-374.
15.	Slobounov S M, Ray W, Johnson B, et al. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int J Psychophysiol, 2015, 95(3): 254-260.
16.	Tian Feng, Hua Minlei, Zhang Wenrui, et al. Emotional arousal in 2D versus 3D virtual reality environments. PLoS One, 2021, 16(9): e0256211.
17.	Bradley M M, Lang P J. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry, 1994, 25(1): 49-59.
18.	Yao D, Qin Y, Hu S, et al. Which reference should we use for EEG and ERP practice?. Brain Topogr, 2019, 32(4): 530-549.
19.	劉沖, 趙海濱, 李春勝, 等. 腦電信號頻帶能量特征的提取方法及分類研究. 系統仿真學報, 2012, 24(12): 2496-2499.
20.	奉國和. SVM分類核函數及參數選擇比較. 計算機工程與應用, 2011, 47(3): 123-124, 128.
21.	章悅, 夏春明, 謝佳智, 等. 特征與分類算法在基于肌音信號的頭部運動分類中的對比研究. 數據采集與處理, 2020, 35(4): 711-719.
22.	侯曉菲. 面向影視的2D與VR正負性情緒比較研究. 上海: 上海大學, 2020.
23.	Watson D, Clark L A, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol, 1988, 54(6): 1063-1070.
24.	Kunaharan S, Halpin S, Sitharthan T, et al. Do EEG and startle reflex modulation vary with self-reported aggression in response to violent images?. Brain Sci, 2019, 9(11): 298.
25.	Perrett D I, Mistlin A J, Chitty A J. Visual neurones responsive to faces. Trends Neurosci, 1987, 10(9): 358-364.
26.	Roy J P, Wurtz R H. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature, 1990, 348(6297): 160-162.
27.	劉燁, 付秋芳, 傅小蘭. 認知與情緒的交互作用. 科學通報, 2009, 54(18): 2783-2796.
28.	George M S, Ketter T A, Parekh P I, et al. Brain activity during transient sadness and happiness in healthy women. Am J Psychiat, 1995, 152(3): 341-351.
29.	Lane R D, Reiman E M, Ahern G L, et al. Neuroanatomical correlates of happiness, sadness, and disgust. Am J Psychiat, 1997, 154(7): 926-933.
30.	Lane R D, Reiman E M, Axelrod B, et al. Neural correlates of levels of emotional awareness: Evidence of an interaction between emotion and attention in the anterior cingulate cortex. J Cogn Neurosci, 1998, 10(4): 525-535.
31.	Damasio A R, Grabowski T J, Bechara A, et al. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat Neurosci, 2000, 3(10): 1049-1056.
32.	Rooney B, Hennessy E. Actually in the cinema: A field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychol, 2013, 16(4): 441-460.
33.	Tan E S H. Entertainment is emotion: The functional architecture of the entertainment experience. Media Psychol, 2008, 11(1): 28-51.
34.	Dennis T A, Solomon B. Frontal EEG and emotion regulation: Electrocortical activity in response to emotional film clips is associated with reduced mood induction and attention interference effects. Biol Psychol, 2010, 85(3): 456-464.
35.	陳曾, 劉光遠. 腦電信號在情感識別中的應用. 計算機工程, 2010, 36(9): 168-170.
36.	Jurewicz K, Paluch K, Kublik E, et al. EEG-neurofeedback training of beta band (12-22Hz) affects alpha and beta frequencies - A controlled study of a healthy population. Neuropsychologia, 2017, 108: 13-24.
37.	陳成明, 王慧, 周健, 等. VDT視覺疲勞及其測量方法綜述. 人類工效學, 2013, 19(2): 92-95.
38.	Zou B, Liu Y, Guo M, et al. EEG-based assessment of stereoscopic 3D visual fatigue caused by vergence-accommodation conflict. J Disp Technol, 2015, 11(12): 1076-1083.
39.	王嘉輝, 程義, 李焜陽, 等. 3D顯示方式與視差對視覺疲勞的影響研究. 中山大學學報(自然科學版), 2013, 52(5): 1-5.
40.	趙彥富, 隨力, 李月如. 基于腦電圖的腦疲勞檢測研究進展. 中國醫學物理學雜志, 2019, 36(11): 1312-1316.

1. 趙沁平. 虛擬現實綜述. 中國科學: 信息科學, 2009, 39(1): 2-46.
2. Bartsch A, Viehoff R. The use of media entertainment and emotional gratification. Procedia Soc Behav Sci, 2010, 5: 2247-2255.
3. Tussyadiah I P, Wang D, Jung T H, et al. Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tourism Manage, 2018, 66: 140-154.
4. Diemer J, Alpers G W, Peperkorn H M, et al. The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front Psychol, 2015, 6: 26.
5. Chao C J, Wu S Y, Yau Y J, et al. Effects of three-dimensional virtual reality and traditional training methods on mental workload and training performance. Hum Factor Ergon Man, 2017, 27(4): 187-196.
6. 徐小萍, 呂健, 金昱潼, 等. 用戶認知驅動的VR自然交互認知負荷研究. 計算機應用研究, 2020, 37(7): 1958-1963.
7. Montana J I, Matamala-Gomez M, Maisto M, et al. The benefits of emotion regulation interventions in virtual reality for the improvement of wellbeing in adults and older adults: a systematic review. J Clin Med, 2020, 9(2): 500.
8. Li B J, Bailenson J N, Pines A, et al. A public database of immersive VR videos with corresponding ratings of arousal, valence, and correlations between head movements and self report measures. Front Psychol, 2017, 8: 2116.
9. Hasson U, Landesman O, Knappmeyer B, et al. Neurocinematics: The neuroscience of film. Projections, 2008, 2(1): 1-26.
10. Schubring D, Kraus M, Stolz C, et al. Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci, 2020, 10(8): 537.
11. He L, Li H, Xue T, et al. Am I in the theater?: usability study of live performance based virtual reality// The 24th ACM Symposium. Tokyo: ACM, 2018: 1-11.
12. Bilgin P, Agres K, Robinson N, et al. A comparative study of mental states in 2D and 3D virtual environments using EEG// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 2833-2838.
13. Kweon S H, Kweon H J, Kim S, et al. A brain wave research on VR (virtual reality) usage: comparison between VR and 2D video in EEG measurement// International Conference on Applied Human Factors and Ergonomics. Los Angeles: Springer, Cham, 2017: 194-203.
14. Kober S E, Kurzmann J, Neuper C. Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. Int J Psychophysiol, 2011, 83(3): 365-374.
15. Slobounov S M, Ray W, Johnson B, et al. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int J Psychophysiol, 2015, 95(3): 254-260.
16. Tian Feng, Hua Minlei, Zhang Wenrui, et al. Emotional arousal in 2D versus 3D virtual reality environments. PLoS One, 2021, 16(9): e0256211.
17. Bradley M M, Lang P J. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry, 1994, 25(1): 49-59.
18. Yao D, Qin Y, Hu S, et al. Which reference should we use for EEG and ERP practice?. Brain Topogr, 2019, 32(4): 530-549.
19. 劉沖, 趙海濱, 李春勝, 等. 腦電信號頻帶能量特征的提取方法及分類研究. 系統仿真學報, 2012, 24(12): 2496-2499.
20. 奉國和. SVM分類核函數及參數選擇比較. 計算機工程與應用, 2011, 47(3): 123-124, 128.
21. 章悅, 夏春明, 謝佳智, 等. 特征與分類算法在基于肌音信號的頭部運動分類中的對比研究. 數據采集與處理, 2020, 35(4): 711-719.
22. 侯曉菲. 面向影視的2D與VR正負性情緒比較研究. 上海: 上海大學, 2020.
23. Watson D, Clark L A, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol, 1988, 54(6): 1063-1070.
24. Kunaharan S, Halpin S, Sitharthan T, et al. Do EEG and startle reflex modulation vary with self-reported aggression in response to violent images?. Brain Sci, 2019, 9(11): 298.
25. Perrett D I, Mistlin A J, Chitty A J. Visual neurones responsive to faces. Trends Neurosci, 1987, 10(9): 358-364.
26. Roy J P, Wurtz R H. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature, 1990, 348(6297): 160-162.
27. 劉燁, 付秋芳, 傅小蘭. 認知與情緒的交互作用. 科學通報, 2009, 54(18): 2783-2796.
28. George M S, Ketter T A, Parekh P I, et al. Brain activity during transient sadness and happiness in healthy women. Am J Psychiat, 1995, 152(3): 341-351.
29. Lane R D, Reiman E M, Ahern G L, et al. Neuroanatomical correlates of happiness, sadness, and disgust. Am J Psychiat, 1997, 154(7): 926-933.
30. Lane R D, Reiman E M, Axelrod B, et al. Neural correlates of levels of emotional awareness: Evidence of an interaction between emotion and attention in the anterior cingulate cortex. J Cogn Neurosci, 1998, 10(4): 525-535.
31. Damasio A R, Grabowski T J, Bechara A, et al. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat Neurosci, 2000, 3(10): 1049-1056.
32. Rooney B, Hennessy E. Actually in the cinema: A field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychol, 2013, 16(4): 441-460.
33. Tan E S H. Entertainment is emotion: The functional architecture of the entertainment experience. Media Psychol, 2008, 11(1): 28-51.
34. Dennis T A, Solomon B. Frontal EEG and emotion regulation: Electrocortical activity in response to emotional film clips is associated with reduced mood induction and attention interference effects. Biol Psychol, 2010, 85(3): 456-464.
35. 陳曾, 劉光遠. 腦電信號在情感識別中的應用. 計算機工程, 2010, 36(9): 168-170.
36. Jurewicz K, Paluch K, Kublik E, et al. EEG-neurofeedback training of beta band (12-22Hz) affects alpha and beta frequencies - A controlled study of a healthy population. Neuropsychologia, 2017, 108: 13-24.
37. 陳成明, 王慧, 周健, 等. VDT視覺疲勞及其測量方法綜述. 人類工效學, 2013, 19(2): 92-95.
38. Zou B, Liu Y, Guo M, et al. EEG-based assessment of stereoscopic 3D visual fatigue caused by vergence-accommodation conflict. J Disp Technol, 2015, 11(12): 1076-1083.
39. 王嘉輝, 程義, 李焜陽, 等. 3D顯示方式與視差對視覺疲勞的影響研究. 中山大學學報(自然科學版), 2013, 52(5): 1-5.
40. 趙彥富, 隨力, 李月如. 基于腦電圖的腦疲勞檢測研究進展. 中國醫學物理學雜志, 2019, 36(11): 1312-1316.

Previous Article
Research on depression recognition based on brain function network
Next Article
Effect of lower limb amputation level on aortic hemodynamics: a numerical study

Journal of Biomedical Engineering

Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content