Application progress of mixed reality technology in hepatobiliary and pancreatic fields_CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Authors：

CAO Yuanyinuo ¹ , LILI Shiluo ² , XUANYUAN Yirui ¹ ,  CHEN Yonghua ^1,3

1. West China Hospital/West China Clinical Medical College, Sichuan University, Chengdu 610041, P. R. China;
2. West China Fourth Hospital/West China School of Public Health, Sichuan University, Chengdu 610041, P. R. China;
3. Division of Pancreatic Surgery, Department of General Surgery, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;

Corresponding?author：

CHEN Yonghua, Email: chenyonghua2007@163.com

Keywords：

mixed reality technology; liver neoplasm; pancreatic neoplasm; surgical navigation; teaching practice

DOI：

10.7507/1007-9424.202212022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To investigate the application progress of mixed reality (MR) technology in hepatobiliary and pancreatic fields. Method The relevant literatures on the application of MR technology of the hepatobiliary and pancreatic field in recent years at home and abroad were reviewed. Results MR technology had been widely used in the hepatobiliary and pancreatic field, including preoperative diagnosis and evaluation, surgical plan formulation, doctor-patient communication, intraoperative navigation precision surgery, teaching practice and many other aspects, which had the advantages of shortening the operation time, reducing the difficulty of surgery and improving the success rate of surgery. To some extent, it had promoted the innovation of clinical diagnosis and treatment in the field of liver, gallbladder and pancreas. Conclusions The application and development of MR related techniques are of great significance to the operation and teaching in the hepatobiliary and pancreatic field. With the development and progress of MR technology and modern medicine, MR technology will give full play to its advantages in intelligent real-time navigation hepatobiliary and pancreatic surgery system and promote the further development of hepatobiliary and pancreatic surgery.

Citation： CAO Yuanyinuo, LILI Shiluo, XUANYUAN Yirui, CHEN Yonghua. Application progress of mixed reality technology in hepatobiliary and pancreatic fields. CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY, 2023, 30(5): 613-619. doi: 10.7507/1007-9424.202212022 Copy

1.	中華醫學會外科學分會肝臟外科學組, 中華醫學會數字醫學分會, 中國醫師協會肝癌專業委員會, 等. 增強與混合現實技術在腹腔鏡復雜性肝切除術中應用的中國專家共識. 中華外科雜志, 2022, 60(6): 517-523.
2.	羅青林, 李生偉. 機器人手術系統在國內肝膽外科應用的歷史、現狀和展望. 中國普外基礎與臨床雜志, 2017, 24(11): 1419-1422.
3.	侯建存, 張雅敏. 混合現實技術在肝膽外科領域中的應用. 中華外科雜志, 2022, 60(1): 17-21.
4.	陳哲, 姜維良. 混合現實技術在血管外科中的應用進展. 中國普外基礎與臨床雜志, 2021, 28(6): 837-840.
5.	閆曉冬, 侯建存, 田青, 等. 混合現實技術在外科領域的應用研究進展. 山東醫藥, 2020, 60(35): 112-114.
6.	劉帆, 昃躍峰. 混合現實技術在教育領域的應用. 新課程教學(電子版). 2021, 5: 140-141.
7.	梅昆有, 李治, 王罡, 等. 混合現實技術在肝膽腫瘤手術中的應用. 中國介入影像與治療學, 2019, 16(12): 745-748.
8.	楊威, 湯睿, 童翾, 等. 混合現實技術在精準肝膽外科手術中應用研究. 中國實用外科雜志, 2021, 41(3): 298-305.
9.	張隆鑫, 陳曉鑫, 姜開文, 等. ALPPS聯合混合現實全息影像成像治療巨塊型肝癌1例. 中國普外基礎與臨床雜志, 2020, 27(10): 1284-1289.
10.	祝文, 曾小軍, 項楠, 等. 增強現實與混合現實導航技術在三維腹腔鏡縮小右半肝切除術中的應用. 中華外科雜志, 2022, 60(3): 249-256.
11.	冷超, 朱鵬, 張必翔. 混合現實全息影像重建技術在肝切除術中的初步應用. 腹部外科, 2020, 33(4): 311-313, 248.
12.	Andolfi C, Plana A, Kania P, et al. Usefulness of three-dimensional modeling in surgical planning, resident training, and patient education. J Laparoendosc Adv Surg Tech A, 2017, 27(5): 512-515.
13.	劉剛, 李曉莉, 龔連生, 等. 混合現實技術輔助全腹腔鏡下胰十二指腸切除術. 中國內鏡雜志, 2020, 26(7): 72-75.
14.	Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
15.	劉文瑛, 歐陽再興, 朱劍華, 等. 多模態影像技術在巨塊型肝癌精準手術治療中的應用. 中國普外基礎與臨床雜志, 2022, 29(9): 1207-1212.
16.	Simone M, Galati R, Barile G, et al. Remote mentoring in laparotomic and laparoscopic cancer surgery during Covid-19 pandemic: an experimental setup based on mixed reality. Med Educ Online, 2021, 26(1): 1996923. doi: 10.1080/10872981.2021.1996923.
17.	Oldhafer KJ, Peterhans M, Kantas A, et al. Navigated liver surgery: current state and importance in the future. Chirurg, 2018, 89(10): 769-776.
18.	Heiselman JS, Clements LW, Collins JA, et al. Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J Med Imaging (Bellingham), 2018, 5(2): 021203. doi: 10.1117/1.JMI.5.2.021203.
19.	Aoki T, Koizumi T, Sugimoto M, et al. Holography-guided percutaneous puncture technique for selective near-infrared fluorescence-guided laparoscopic liver resection using mixed-reality wearable spatial computer. Surg Oncol, 2020, 35: 476-477.
20.	馬春陽, 周益龍, 李懷亮, 等. 混合現實術中導航技術聯合三維可視化技術在復雜性肝癌切除術中的應用. 中華醫學雜志, 2019, 99(4): 279-283.
21.	Saito Y, Sugimoto M, Morine Y, et al. Intraoperative support with three-dimensional holographic cholangiography in hepatobiliary surgery. Langenbecks Arch Surg, 2022, 407(3): 1285-1289.
22.	祝文, 曾小軍, 胡浩宇, 等. 應用增強現實與混合現實導航預防腹腔鏡肝切除術中出血價值研究. 中國實用外科雜志, 2022, 42(3): 298-302,308.
23.	Zhu LY, Hou JC, Yang L, et al. Application value of mixed reality in hepatectomy for hepatocellular carcinoma. World J Gastrointest Surg, 2022, 14(1): 36-45.
24.	Balci D, Kirimker EO, Raptis DA, et al. Uses of a dedicated 3D reconstruction software with augmented and mixed reality in planning and performing advanced liver surgery and living donor liver transplantation (with videos). Hepatobiliary Pancreat Dis Int, 2022, 21(5): 455-461.
25.	王峻峰, 贊左, 宋正己, 等. 基于3D打印/混合現實技術在胰周感染性積液治療的應用. 國際臨床研究雜志, 2021, 5(1): 24-28.
26.	Wierzbicki R, Paw?owicz M, Job J, et al. 3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases. J Cancer Res Clin Oncol, 2022, 148(1): 237-243.
27.	Saito Y, Sugimoto M, Imura S, et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg, 2020, 271(1): e4-e7. doi: 10.1097/SLA.0000000000003552.
28.	Strickland A, Fairhurst K, Lauder C, et al. Development of an ex vivo simulated training model for laparoscopic liver resection. Surg Endosc, 2011, 25(5): 1677-1682.
29.	李鵬超, 袁文博, 萬芬, 等. 混合現實技術在住院醫師規范化培訓外科手術教學中的應用研究. 創新創業理論研究與實踐, 2022, 5(22): 179-184.
30.	呂秀琴, 張生海. 基于HoloLens2的DEM空間分析實驗教學混合現實環境研究. 實驗技術與管理. 2021, 38(10): 124-129.
31.	Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol, 2020, 29(3): 154-160.
32.	景濤, 邱陽, 李靜堯, 等. 混合現實技術在住院醫師規范化培訓臨床手術能力教學中的應用. 重慶醫學, 2021, 50(8): 1435-1437.
33.	李忠海, 徐鋼, 高淑賢, 等. 混合現實技術在外科臨床教學中的應用及效果評價. 中國高等醫學教育, 2022, (9): 67-69.
34.	王宏, 歐晏嬌, 胡鵬, 等. 混合現實技術在肝膽外科手術教學中的應用效果. 中華醫學教育探索雜志, 2019, 18(12): 1230-1234.
35.	Bala L, Kinross J, Martin G, et al. A remote access mixed reality teaching ward round. Clin Teach, 2021, 18(4): 386-390.
36.	石小強, 馬艷飛, 梁挺, 等. 混合現實技術在專科醫師培訓中輔助腹膜后腫瘤手術臨床教學的應用研究. 中國醫學教育技術, 2020, 34(4): 504-507.
37.	Liu S, Wang H, Wang C, et al. Intra-operative 3D hologram support with mixed reality technique based on CT-MRI fusion images: operation guidance for brain brachytherapy. J Contemp Brachytherapy, 2021, 13(2): 205-210.
38.	Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
39.	van den Berg NS, Engelen T, Brouwer OR, et al. A pilot study of SPECT/CT-based mixed-reality navigation towards the sentinel node in patients with melanoma or Merkel cell carcinoma of a lower extremity. Nucl Med Commun, 2016, 37(8): 812-817.
40.	Davrieux CF, Giménez ME, González CA, et al. Mixed reality navigation system for ultrasound-guided percutaneous punctures: a pre-clinical evaluation. Surg Endosc, 2020, 34(1): 226-230.
41.	Tu P, Qin C, Guo Y, et al. Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng, 2022, 69(8): 2593-2603.
42.	Li R, Pan J, Yang Y, et al. Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images. Comput Med Imaging Graph, 2021, 94: 102007. doi: 10.1016/j.compmedimag.2021.102007.
43.	Bloom D, Colombo JN, Miller N, et al. Early preclinical experience of a mixed reality ultrasound system with active GUIDance for NEedle-based interventions: The GUIDE study. Cardiovasc Digit Health J, 2022, 3(5): 232-240.

1. 中華醫學會外科學分會肝臟外科學組, 中華醫學會數字醫學分會, 中國醫師協會肝癌專業委員會, 等. 增強與混合現實技術在腹腔鏡復雜性肝切除術中應用的中國專家共識. 中華外科雜志, 2022, 60(6): 517-523.
2. 羅青林, 李生偉. 機器人手術系統在國內肝膽外科應用的歷史、現狀和展望. 中國普外基礎與臨床雜志, 2017, 24(11): 1419-1422.
3. 侯建存, 張雅敏. 混合現實技術在肝膽外科領域中的應用. 中華外科雜志, 2022, 60(1): 17-21.
4. 陳哲, 姜維良. 混合現實技術在血管外科中的應用進展. 中國普外基礎與臨床雜志, 2021, 28(6): 837-840.
5. 閆曉冬, 侯建存, 田青, 等. 混合現實技術在外科領域的應用研究進展. 山東醫藥, 2020, 60(35): 112-114.
6. 劉帆, 昃躍峰. 混合現實技術在教育領域的應用. 新課程教學(電子版). 2021, 5: 140-141.
7. 梅昆有, 李治, 王罡, 等. 混合現實技術在肝膽腫瘤手術中的應用. 中國介入影像與治療學, 2019, 16(12): 745-748.
8. 楊威, 湯睿, 童翾, 等. 混合現實技術在精準肝膽外科手術中應用研究. 中國實用外科雜志, 2021, 41(3): 298-305.
9. 張隆鑫, 陳曉鑫, 姜開文, 等. ALPPS聯合混合現實全息影像成像治療巨塊型肝癌1例. 中國普外基礎與臨床雜志, 2020, 27(10): 1284-1289.
10. 祝文, 曾小軍, 項楠, 等. 增強現實與混合現實導航技術在三維腹腔鏡縮小右半肝切除術中的應用. 中華外科雜志, 2022, 60(3): 249-256.
11. 冷超, 朱鵬, 張必翔. 混合現實全息影像重建技術在肝切除術中的初步應用. 腹部外科, 2020, 33(4): 311-313, 248.
12. Andolfi C, Plana A, Kania P, et al. Usefulness of three-dimensional modeling in surgical planning, resident training, and patient education. J Laparoendosc Adv Surg Tech A, 2017, 27(5): 512-515.
13. 劉剛, 李曉莉, 龔連生, 等. 混合現實技術輔助全腹腔鏡下胰十二指腸切除術. 中國內鏡雜志, 2020, 26(7): 72-75.
14. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
15. 劉文瑛, 歐陽再興, 朱劍華, 等. 多模態影像技術在巨塊型肝癌精準手術治療中的應用. 中國普外基礎與臨床雜志, 2022, 29(9): 1207-1212.
16. Simone M, Galati R, Barile G, et al. Remote mentoring in laparotomic and laparoscopic cancer surgery during Covid-19 pandemic: an experimental setup based on mixed reality. Med Educ Online, 2021, 26(1): 1996923. doi: 10.1080/10872981.2021.1996923.
17. Oldhafer KJ, Peterhans M, Kantas A, et al. Navigated liver surgery: current state and importance in the future. Chirurg, 2018, 89(10): 769-776.
18. Heiselman JS, Clements LW, Collins JA, et al. Characterization and correction of intraoperative soft tissue deformation in image-guided laparoscopic liver surgery. J Med Imaging (Bellingham), 2018, 5(2): 021203. doi: 10.1117/1.JMI.5.2.021203.
19. Aoki T, Koizumi T, Sugimoto M, et al. Holography-guided percutaneous puncture technique for selective near-infrared fluorescence-guided laparoscopic liver resection using mixed-reality wearable spatial computer. Surg Oncol, 2020, 35: 476-477.
20. 馬春陽, 周益龍, 李懷亮, 等. 混合現實術中導航技術聯合三維可視化技術在復雜性肝癌切除術中的應用. 中華醫學雜志, 2019, 99(4): 279-283.
21. Saito Y, Sugimoto M, Morine Y, et al. Intraoperative support with three-dimensional holographic cholangiography in hepatobiliary surgery. Langenbecks Arch Surg, 2022, 407(3): 1285-1289.
22. 祝文, 曾小軍, 胡浩宇, 等. 應用增強現實與混合現實導航預防腹腔鏡肝切除術中出血價值研究. 中國實用外科雜志, 2022, 42(3): 298-302,308.
23. Zhu LY, Hou JC, Yang L, et al. Application value of mixed reality in hepatectomy for hepatocellular carcinoma. World J Gastrointest Surg, 2022, 14(1): 36-45.
24. Balci D, Kirimker EO, Raptis DA, et al. Uses of a dedicated 3D reconstruction software with augmented and mixed reality in planning and performing advanced liver surgery and living donor liver transplantation (with videos). Hepatobiliary Pancreat Dis Int, 2022, 21(5): 455-461.
25. 王峻峰, 贊左, 宋正己, 等. 基于3D打印/混合現實技術在胰周感染性積液治療的應用. 國際臨床研究雜志, 2021, 5(1): 24-28.
26. Wierzbicki R, Paw?owicz M, Job J, et al. 3D mixed-reality visualization of medical imaging data as a supporting tool for innovative, minimally invasive surgery for gastrointestinal tumors and systemic treatment as a new path in personalized treatment of advanced cancer diseases. J Cancer Res Clin Oncol, 2022, 148(1): 237-243.
27. Saito Y, Sugimoto M, Imura S, et al. Intraoperative 3D hologram support with mixed reality techniques in liver surgery. Ann Surg, 2020, 271(1): e4-e7. doi: 10.1097/SLA.0000000000003552.
28. Strickland A, Fairhurst K, Lauder C, et al. Development of an ex vivo simulated training model for laparoscopic liver resection. Surg Endosc, 2011, 25(5): 1677-1682.
29. 李鵬超, 袁文博, 萬芬, 等. 混合現實技術在住院醫師規范化培訓外科手術教學中的應用研究. 創新創業理論研究與實踐, 2022, 5(22): 179-184.
30. 呂秀琴, 張生海. 基于HoloLens2的DEM空間分析實驗教學混合現實環境研究. 實驗技術與管理. 2021, 38(10): 124-129.
31. Pelanis E, Kumar RP, Aghayan DL, et al. Use of mixed reality for improved spatial understanding of liver anatomy. Minim Invasive Ther Allied Technol, 2020, 29(3): 154-160.
32. 景濤, 邱陽, 李靜堯, 等. 混合現實技術在住院醫師規范化培訓臨床手術能力教學中的應用. 重慶醫學, 2021, 50(8): 1435-1437.
33. 李忠海, 徐鋼, 高淑賢, 等. 混合現實技術在外科臨床教學中的應用及效果評價. 中國高等醫學教育, 2022, (9): 67-69.
34. 王宏, 歐晏嬌, 胡鵬, 等. 混合現實技術在肝膽外科手術教學中的應用效果. 中華醫學教育探索雜志, 2019, 18(12): 1230-1234.
35. Bala L, Kinross J, Martin G, et al. A remote access mixed reality teaching ward round. Clin Teach, 2021, 18(4): 386-390.
36. 石小強, 馬艷飛, 梁挺, 等. 混合現實技術在專科醫師培訓中輔助腹膜后腫瘤手術臨床教學的應用研究. 中國醫學教育技術, 2020, 34(4): 504-507.
37. Liu S, Wang H, Wang C, et al. Intra-operative 3D hologram support with mixed reality technique based on CT-MRI fusion images: operation guidance for brain brachytherapy. J Contemp Brachytherapy, 2021, 13(2): 205-210.
38. Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
39. van den Berg NS, Engelen T, Brouwer OR, et al. A pilot study of SPECT/CT-based mixed-reality navigation towards the sentinel node in patients with melanoma or Merkel cell carcinoma of a lower extremity. Nucl Med Commun, 2016, 37(8): 812-817.
40. Davrieux CF, Giménez ME, González CA, et al. Mixed reality navigation system for ultrasound-guided percutaneous punctures: a pre-clinical evaluation. Surg Endosc, 2020, 34(1): 226-230.
41. Tu P, Qin C, Guo Y, et al. Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng, 2022, 69(8): 2593-2603.
42. Li R, Pan J, Yang Y, et al. Accurate and robust feature description and dense point-wise matching based on feature fusion for endoscopic images. Comput Med Imaging Graph, 2021, 94: 102007. doi: 10.1016/j.compmedimag.2021.102007.
43. Bloom D, Colombo JN, Miller N, et al. Early preclinical experience of a mixed reality ultrasound system with active GUIDance for NEedle-based interventions: The GUIDE study. Cardiovasc Digit Health J, 2022, 3(5): 232-240.

CHINESE JOURNAL OF BASES AND CLINICS IN GENERAL SURGERY

Application progress of mixed reality technology in hepatobiliary and pancreatic fields

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content