Application of mixed reality technology in orthopedic field_West China Medical Journal

Authors：

CHEN Lin , LIU Bailian , PU Qi , WANG Daxing ,  YANG Huiqin

Department of Orthopedics, Yan’an Hospital Affiliated to Kunming Medical University, Kunming, Yunnan 650051, P. R. China;

Corresponding?author：

YANG Huiqin, Email: doctoryhq@163.com

Keywords：

Orthopedics; Mixed reality; Precision medicine; Digital orthopedics

DOI：

10.7507/1002-0179.202005101

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Mixed reality technology is new digital holographic imaging technology that generates three-dimensional simulation images through computers and anchors the virtual images to the real world. Compared with traditional imaging diagnosis and treatment methods, mixed reality technology is more conducive to the advantages of precision medicine, helps to promote the development of medical clinical application, teaching and scientific research in the field of orthopedics, and will further promote the progress of clinical orthopedics toward standardization, digitization and precision. This article briefly introduces the mixed reality technology, reviews its application in the perioperative period, teaching and diagnosis and treatment standardization and dataization in the field of orthopedics, and discusses its technical advantages, aiming to provide a reference for the better use of mixed reality technology in orthopedics.

Citation： CHEN Lin, LIU Bailian, PU Qi, WANG Daxing, YANG Huiqin. Application of mixed reality technology in orthopedic field. West China Medical Journal, 2021, 36(8): 1126-1131. doi: 10.7507/1002-0179.202005101 Copy

1.	Hauze SW, Hoyt HH, Frazee JP, et al. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol, 2019, 1120: 1-13.
2.	方馳華, 陳康, 張鵬. 智能化診療技術在普通外科中應用現狀及前景. 中華外科雜志, 2019, 57(1): 1-5.
3.	Halic T, Kockara S, Bayrak C, et al. Mixed reality simulation of rasping procedure in artificial cervical disc replacement (ACDR) surgery. BMC Bioinformatics, 2010, 11(Suppl 6): S11.
4.	Gao Y, Tan K, Sun J, et al. Application of mixed reality technology in visualization of medical operations. Chin Med Sci J, 2019, 34(2): 103-109.
5.	Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
6.	Goo HW, Park SJ, Yoo SJ. Advanced medical use of three-dimensional imaging in congenital heart disease: augmented reality, mixed reality, virtual reality, and three-dimensional printing. Korean J Radiol, 2020, 21(2): 133-145.
7.	葉哲偉, 吳星火. 混合現實技術在骨科的最新應用進展. 臨床外科雜志, 2018, 26(1): 13-14.
8.	Tuladhar S, AlSallami N, Alsadoon A, et al. A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality. Int J Med Robot, 2020, 16(5): 1-22.
9.	Salmas M, Chronopoulos E, Chytas D. Comment on: “A novel evaluation model for a mixed-reality surgical navigation system: where microsoft hololens meets the operating room”. Surg Innov, 2020, 27(6): 702-703.
10.	Checcucci E, Amparore D, Pecoraro A, et al. 3D mixed reality holograms for preoperative surgical planning of nephron-sparing surgery: evaluation of surgeons’ perception. Minerva Urol Nephrol, 2021, 73(3): 367-375.
11.	萬磊, 庾偉中, 王建華, 等. 基于混合現實的三維測量技術輔助診斷顱底凹陷癥. 中國骨科臨床與基礎研究雜志, 2018, 10(5): 280-284.
12.	Chakravarthy V, Sheikh S, Schmidt E, et al. Imaging technologies in spine surgery. Neurosurg Clin N Am, 2020, 31(1): 93-101.
13.	Wu X, Liu R, Yu J, et al. Mixed reality technology launches in orthopedic surgery for comprehensive preoperative management of complicated cervical fractures. Surg Innov, 2018, 25(4): 421-422.
14.	Yu H, Zhou Z, Lei X, et al. Mixed reality-based preoperative planning for training of percutaneous transforaminal endoscopic discectomy: a feasibility study. World Neurosurg, 2019, 129: e767-e775.
15.	Wu X, Liu R, Yu J, et al. Mixed reality technology-assisted orthopedics surgery navigation. Surg Innov, 2018, 25(3): 304-305.
16.	Chu Y, Li X, Yang X, et al. Perception enhancement using importance-driven hybrid rendering for augmented reality based endoscopic surgical navigation. Biomed Opt Express, 2018, 9(11): 5205-5226.
17.	Wei P, Yao Q, Xu Y, et al. Percutaneous kyphoplasty assisted with/without mixed reality technology in treatment of OVCF with IVC: a prospective study. J Orthop Surg Res, 2019, 14(1): 255.
18.	陶星光. 計算機輔助技術在脛骨平臺骨折分期治療中的應用研究. 江蘇: 南京醫科大學, 2019: 26-39.
19.	Wu X, Liu R, Xu S, et al. Feasibility of mixed reality-based intraoperative three-dimensional image-guided navigation for atlanto-axial pedicle screw placement. Proc Inst Mech Eng H, 2019, 233(12): 1310-1317.
20.	萬磊, 梁篤, 李保林, 等. 基于 CAD-MR 技術混合現實重建骨盆與髖臼骨折及血管網輔助手術臨床研究. 實用醫學雜志, 2019, 35(7): 1019-1022, 1027.
21.	Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
22.	Lindeman DA, Kim KK, Gladstone C, et al. Technology and caregiving: emerging interventions and directions for research. Gerontologist, 2020, 60(Suppl 1): S41-S49.
23.	王慧文, 高春紅, 胡甜. 股骨頸骨折青少年患者混合現實技術引導下微創手術的快速康復護理. 護理學雜志, 2019, 34(10): 23-25.
24.	Negrillo-Cárdenas J, Jiménez-Pérez JR, Feito FR. The role of virtual and augmented reality in orthopedic trauma surgery: from diagnosis to rehabilitation. Comput Methods Programs Biomed, 2020, 191: 105407.
25.	沈劍輝, 高興蓮, 鄢利芳, 等. 混合現實技術引導下行復雜骨折手術的護理配合. 中華護理雜志, 2018, 53(9): 1050-1054.
26.	Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ, 2020, 11: 1-18.
27.	Flores A, Linehan MM, Todd SR, et al. The use of virtual reality to facilitate mindfulness skills training in dialectical behavioral therapy for spinal cord injury: a case study. Front Psychol, 2018, 9: 531.
28.	Stefan P, Habert S, Winkler A, et al. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg, 2018, 13(9): 1335-1344.
29.	McJunkin JL, Jiramongkolchai P, Chung W, et al. Development of a mixed reality platform for lateral skull base anatomy. Otol Neurotol, 2018, 39(10): e1137-e1142.
30.	Wainman B, Pukas G, Wolak L, et al. The critical role of stereopsis in virtual and mixed reality learning environments. Anat Sci Educ, 2020, 13(3): 401-412.
31.	Ruthberg JS, Tingle G, Tan L, et al. Mixed reality as a time-efficient alternative to cadaveric dissection. Med Teach, 2020, 42(8): 896-901.
32.	張里程, 雷明星, 張浩, 等. 基于混合現實技術骨科教學模式的構建及應用. 中國繼續醫學教育, 2018, 10(36): 13-16.
33.	Moro C, Gregory S. Utilising anatomical and physiological visualisations to enhance the face-to-face student learning experience in biomedical sciences and medicine. Adv Exp Med Biol, 2019, 1156: 41-48.
34.	Sappenfield JW, Smith WB, Cooper LA, et al. Visualization improves supraclavicular access to the subclavian vein in a mixed reality simulator. Anesth Analg, 2018, 127(1): 83-89.
35.	Hochman JB, Sepehri N, Rampersad V, et al. Mixed reality temporal bone surgical dissector: mechanical design. J Otolaryngol Head Neck Surg, 2014, 43(1): 23.
36.	Maniam P, Schnell P, Dan L, et al. Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med, 2020, 43(1): 17-26.
37.	Condino S, Turini G, Parchi PD, et al. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the microsoft HoloLens. J Healthc Eng, 2018, 2018: 5435097.
38.	Coelho G, Defino HLA. The role of mixed reality simulation for surgical training in spine: phase 1 validation. Spine (Phila Pa 1976), 2018, 43(22): 1609-1616.
39.	Stefan P, Pfandler M, Wucherer P, et al. Team training and assessment in mixed reality-based simulated operating room: current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project. Unfallchirurg, 2018, 121(4): 271-277.
40.	馬春陽, 周益龍, 李懷亮, 等. 混合現實術中導航技術聯合三維可視化技術在復雜性肝癌切除術中的應用. 中華醫學雜志, 2019(4): 279-283.
41.	Kanevsky J, Safran T, Zammit D, et al. Making augmented and virtual reality work for the plastic surgeon. Ann Plast Surg, 2019, 82(4): 363-368.
42.	朱捷, 沈誕, 劉啟明, 等. 混合現實平臺遠程協作機器人微創手術 1 例報告. 微創泌尿外科雜志, 2018, 7(4): 278-281.
43.	Rojas-Mu?oz E, Cabrera ME, Lin C, et al. Telementoring in leg fasciotomies via mixed-reality: clinical evaluation of the STAR platform. Mil Med, 2020, 185(Suppl 1): 513-520.
44.	Wang C, Daniel BK, Asil M, et al. A randomised control trial and comparative analysis of multi-dimensional learning tools in anatomy. Sci Rep, 2020, 10(1): 6120.
45.	He Y, Yang G, Yang J, et al. Dense biased networks with deep priori anatomy and hard region adaptation: semi-supervised learning for fine renal artery segmentation. Med Image Anal, 2020, 63: 101722.
46.	Lei PF, Su SL, Kong LY, et al. Mixed reality combined with three-dimensional printing technology in total hip arthroplasty: an updated review with a preliminary case presentation. Orthop Surg, 2019, 11(5): 914-920.
47.	Talaat S, Ghoneima A, Kaboudan A, et al. Three-dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device. Orthod Craniofac Res, 2019, 22(Suppl 1): 62-68.
48.	Perkins SL, Krajancich B, Yang CJ, et al. A patient-specific mixed-reality visualization tool for thoracic surgical planning. Ann Thorac Surg, 2020, 110(1): 290-295.
49.	Coelho G, Figueiredo EG, Rabelo NN, et al. Development and evaluation of pediatric mixed-reality model for neuroendoscopic surgical training. World Neurosurg, 2020, 139: e189-e202.
50.	Huber T, Hadzijusufovic E, Hansen C, et al. Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision. Dis Colon Rectum, 2019, 62(2): 258-261.
51.	Vávra P, Roman J, Zon? a P, et al. Recent development of augmented reality in surgery: a review. J Healthc Eng, 2017, 2017: 4574172.

1. Hauze SW, Hoyt HH, Frazee JP, et al. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol, 2019, 1120: 1-13.
2. 方馳華, 陳康, 張鵬. 智能化診療技術在普通外科中應用現狀及前景. 中華外科雜志, 2019, 57(1): 1-5.
3. Halic T, Kockara S, Bayrak C, et al. Mixed reality simulation of rasping procedure in artificial cervical disc replacement (ACDR) surgery. BMC Bioinformatics, 2010, 11(Suppl 6): S11.
4. Gao Y, Tan K, Sun J, et al. Application of mixed reality technology in visualization of medical operations. Chin Med Sci J, 2019, 34(2): 103-109.
5. Sauer IM, Queisner M, Tang P, et al. Mixed reality in visceral surgery: development of a suitable workflow and evaluation of intraoperative use-cases. Ann Surg, 2017, 266(5): 706-712.
6. Goo HW, Park SJ, Yoo SJ. Advanced medical use of three-dimensional imaging in congenital heart disease: augmented reality, mixed reality, virtual reality, and three-dimensional printing. Korean J Radiol, 2020, 21(2): 133-145.
7. 葉哲偉, 吳星火. 混合現實技術在骨科的最新應用進展. 臨床外科雜志, 2018, 26(1): 13-14.
8. Tuladhar S, AlSallami N, Alsadoon A, et al. A recent review and a taxonomy for hard and soft tissue visualization-based mixed reality. Int J Med Robot, 2020, 16(5): 1-22.
9. Salmas M, Chronopoulos E, Chytas D. Comment on: “A novel evaluation model for a mixed-reality surgical navigation system: where microsoft hololens meets the operating room”. Surg Innov, 2020, 27(6): 702-703.
10. Checcucci E, Amparore D, Pecoraro A, et al. 3D mixed reality holograms for preoperative surgical planning of nephron-sparing surgery: evaluation of surgeons’ perception. Minerva Urol Nephrol, 2021, 73(3): 367-375.
11. 萬磊, 庾偉中, 王建華, 等. 基于混合現實的三維測量技術輔助診斷顱底凹陷癥. 中國骨科臨床與基礎研究雜志, 2018, 10(5): 280-284.
12. Chakravarthy V, Sheikh S, Schmidt E, et al. Imaging technologies in spine surgery. Neurosurg Clin N Am, 2020, 31(1): 93-101.
13. Wu X, Liu R, Yu J, et al. Mixed reality technology launches in orthopedic surgery for comprehensive preoperative management of complicated cervical fractures. Surg Innov, 2018, 25(4): 421-422.
14. Yu H, Zhou Z, Lei X, et al. Mixed reality-based preoperative planning for training of percutaneous transforaminal endoscopic discectomy: a feasibility study. World Neurosurg, 2019, 129: e767-e775.
15. Wu X, Liu R, Yu J, et al. Mixed reality technology-assisted orthopedics surgery navigation. Surg Innov, 2018, 25(3): 304-305.
16. Chu Y, Li X, Yang X, et al. Perception enhancement using importance-driven hybrid rendering for augmented reality based endoscopic surgical navigation. Biomed Opt Express, 2018, 9(11): 5205-5226.
17. Wei P, Yao Q, Xu Y, et al. Percutaneous kyphoplasty assisted with/without mixed reality technology in treatment of OVCF with IVC: a prospective study. J Orthop Surg Res, 2019, 14(1): 255.
18. 陶星光. 計算機輔助技術在脛骨平臺骨折分期治療中的應用研究. 江蘇: 南京醫科大學, 2019: 26-39.
19. Wu X, Liu R, Xu S, et al. Feasibility of mixed reality-based intraoperative three-dimensional image-guided navigation for atlanto-axial pedicle screw placement. Proc Inst Mech Eng H, 2019, 233(12): 1310-1317.
20. 萬磊, 梁篤, 李保林, 等. 基于 CAD-MR 技術混合現實重建骨盆與髖臼骨折及血管網輔助手術臨床研究. 實用醫學雜志, 2019, 35(7): 1019-1022, 1027.
21. Zhou Z, Yang Z, Jiang S, et al. Design and validation of a surgical navigation system for brachytherapy based on mixed reality. Med Phys, 2019, 46(8): 3709-3718.
22. Lindeman DA, Kim KK, Gladstone C, et al. Technology and caregiving: emerging interventions and directions for research. Gerontologist, 2020, 60(Suppl 1): S41-S49.
23. 王慧文, 高春紅, 胡甜. 股骨頸骨折青少年患者混合現實技術引導下微創手術的快速康復護理. 護理學雜志, 2019, 34(10): 23-25.
24. Negrillo-Cárdenas J, Jiménez-Pérez JR, Feito FR. The role of virtual and augmented reality in orthopedic trauma surgery: from diagnosis to rehabilitation. Comput Methods Programs Biomed, 2020, 191: 105407.
25. 沈劍輝, 高興蓮, 鄢利芳, 等. 混合現實技術引導下行復雜骨折手術的護理配合. 中華護理雜志, 2018, 53(9): 1050-1054.
26. Gerup J, Soerensen CB, Dieckmann P. Augmented reality and mixed reality for healthcare education beyond surgery: an integrative review. Int J Med Educ, 2020, 11: 1-18.
27. Flores A, Linehan MM, Todd SR, et al. The use of virtual reality to facilitate mindfulness skills training in dialectical behavioral therapy for spinal cord injury: a case study. Front Psychol, 2018, 9: 531.
28. Stefan P, Habert S, Winkler A, et al. A radiation-free mixed-reality training environment and assessment concept for C-arm-based surgery. Int J Comput Assist Radiol Surg, 2018, 13(9): 1335-1344.
29. McJunkin JL, Jiramongkolchai P, Chung W, et al. Development of a mixed reality platform for lateral skull base anatomy. Otol Neurotol, 2018, 39(10): e1137-e1142.
30. Wainman B, Pukas G, Wolak L, et al. The critical role of stereopsis in virtual and mixed reality learning environments. Anat Sci Educ, 2020, 13(3): 401-412.
31. Ruthberg JS, Tingle G, Tan L, et al. Mixed reality as a time-efficient alternative to cadaveric dissection. Med Teach, 2020, 42(8): 896-901.
32. 張里程, 雷明星, 張浩, 等. 基于混合現實技術骨科教學模式的構建及應用. 中國繼續醫學教育, 2018, 10(36): 13-16.
33. Moro C, Gregory S. Utilising anatomical and physiological visualisations to enhance the face-to-face student learning experience in biomedical sciences and medicine. Adv Exp Med Biol, 2019, 1156: 41-48.
34. Sappenfield JW, Smith WB, Cooper LA, et al. Visualization improves supraclavicular access to the subclavian vein in a mixed reality simulator. Anesth Analg, 2018, 127(1): 83-89.
35. Hochman JB, Sepehri N, Rampersad V, et al. Mixed reality temporal bone surgical dissector: mechanical design. J Otolaryngol Head Neck Surg, 2014, 43(1): 23.
36. Maniam P, Schnell P, Dan L, et al. Exploration of temporal bone anatomy using mixed reality (HoloLens): development of a mixed reality anatomy teaching resource prototype. J Vis Commun Med, 2020, 43(1): 17-26.
37. Condino S, Turini G, Parchi PD, et al. How to build a patient-specific hybrid simulator for orthopaedic open surgery: benefits and limits of mixed-reality using the microsoft HoloLens. J Healthc Eng, 2018, 2018: 5435097.
38. Coelho G, Defino HLA. The role of mixed reality simulation for surgical training in spine: phase 1 validation. Spine (Phila Pa 1976), 2018, 43(22): 1609-1616.
39. Stefan P, Pfandler M, Wucherer P, et al. Team training and assessment in mixed reality-based simulated operating room: current state of research in the field of simulation in spine surgery exemplified by the ATMEOS project. Unfallchirurg, 2018, 121(4): 271-277.
40. 馬春陽, 周益龍, 李懷亮, 等. 混合現實術中導航技術聯合三維可視化技術在復雜性肝癌切除術中的應用. 中華醫學雜志, 2019(4): 279-283.
41. Kanevsky J, Safran T, Zammit D, et al. Making augmented and virtual reality work for the plastic surgeon. Ann Plast Surg, 2019, 82(4): 363-368.
42. 朱捷, 沈誕, 劉啟明, 等. 混合現實平臺遠程協作機器人微創手術 1 例報告. 微創泌尿外科雜志, 2018, 7(4): 278-281.
43. Rojas-Mu?oz E, Cabrera ME, Lin C, et al. Telementoring in leg fasciotomies via mixed-reality: clinical evaluation of the STAR platform. Mil Med, 2020, 185(Suppl 1): 513-520.
44. Wang C, Daniel BK, Asil M, et al. A randomised control trial and comparative analysis of multi-dimensional learning tools in anatomy. Sci Rep, 2020, 10(1): 6120.
45. He Y, Yang G, Yang J, et al. Dense biased networks with deep priori anatomy and hard region adaptation: semi-supervised learning for fine renal artery segmentation. Med Image Anal, 2020, 63: 101722.
46. Lei PF, Su SL, Kong LY, et al. Mixed reality combined with three-dimensional printing technology in total hip arthroplasty: an updated review with a preliminary case presentation. Orthop Surg, 2019, 11(5): 914-920.
47. Talaat S, Ghoneima A, Kaboudan A, et al. Three-dimensional evaluation of the holographic projection in digital dental model superimposition using HoloLens device. Orthod Craniofac Res, 2019, 22(Suppl 1): 62-68.
48. Perkins SL, Krajancich B, Yang CJ, et al. A patient-specific mixed-reality visualization tool for thoracic surgical planning. Ann Thorac Surg, 2020, 110(1): 290-295.
49. Coelho G, Figueiredo EG, Rabelo NN, et al. Development and evaluation of pediatric mixed-reality model for neuroendoscopic surgical training. World Neurosurg, 2020, 139: e189-e202.
50. Huber T, Hadzijusufovic E, Hansen C, et al. Head-mounted mixed-reality technology during robotic-assisted transanal total mesorectal excision. Dis Colon Rectum, 2019, 62(2): 258-261.
51. Vávra P, Roman J, Zon? a P, et al. Recent development of augmented reality in surgery: a review. J Healthc Eng, 2017, 2017: 4574172.

Previous Article
Virtual reality for the upper limb motor rehabilitation after stroke
Next Article
The role of alternative splicing in autoimmune diseases

West China Medical Journal

Application of mixed reality technology in orthopedic field

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content