Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions_Journal of Biomedical Engineering

Authors：

SHI Tianlei ^1,2 , ZHANG Jiayi ^1,2,3 , BAO Yongyang ⁴ ,  GAO Xin ^2,3,5

1. School of Biomedical Engineering (Suzhou), Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei 230026, P. R. China;
2. Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences, Suzhou, Jiangsu 215163, P. R. China;
3. Jinan Guoke Medical Engineering and Technology Development Co., Ltd., Jinan 250101, P. R. China;
4. Shanghai Ninth People’s Hospital, Shanghai Jiaotong University School of Medicine, Shanghai 200011, P. R. China;
5. Jiangsu Province Engineering Research Center of Diagnosis and Treatment of Children’s Malignant Tumor, Suzhou, Jiangsu 215025, P. R. China;

Corresponding?author：

GAO Xin, Email: xingaosam@163.com

Keywords：

Melanocytic lesions; Whole slide images; Intelligent and precise diagnosis; Deep learning; Color normalization

DOI：

10.7507/1001-5515.202203080

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Melanocytic lesions occur on the surface of the skin, in which the malignant type is melanoma with a high fatality rate, seriously endangering human health. The histopathological analysis is the gold standard for diagnosis of melanocytic lesions. In this study, a fully automated intelligent diagnosis method based on deep learning was proposed to classify the pathological whole slide images (WSI) of melanocytic lesions. Firstly, the color normalization based on CycleGAN neural network was performed on multi-center pathological WSI; Secondly, ResNet-152 neural network-based deep convolutional network prediction model was built using 745 WSI; Then, a decision fusion model was cascaded, which calculates the average prediction probability of each WSI; Finally, the diagnostic performance of the proposed method was verified by internal and external test sets containing 182 and 54 WSI, respectively. Experimental results showed that the overall diagnostic accuracy of the proposed method reached 94.12% in the internal test set and exceeded 90% in the external test set. Furthermore, the color normalization method adopted was superior to the traditional color statistics-based and staining separation-based methods in terms of structure preservation and artifact suppression. The results demonstrate that the proposed method can achieve high precision and strong robustness in pathological WSI classification of melanocytic lesions, which has the potential in promoting the clinical application of computer-aided pathological diagnosis.

Citation： SHI Tianlei, ZHANG Jiayi, BAO Yongyang, GAO Xin. Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions. Journal of Biomedical Engineering, 2022, 39(5): 919-927. doi: 10.7507/1001-5515.202203080 Copy

1.	Swetter S M, Tsao H, Bichakjian C K, et al. Guidelines of care for the management of primary cutaneous melanoma. J Am Acad Dermatol, 2019, 80(1): 208-250.
2.	Schadendorf D, van Akkooi A C J, Berking C, et al. Melanoma. Lancet, 2018, 392(10151): 971-984.
3.	Piepkorn M W, Barnhill R L, Elder D E, et al. The MPATH-Dx reporting schema for melanocytic proliferations and melanoma. J Am Acad Dermatol, 2014, 70(1): 131-141.
4.	Davis L E, Shalin S C, Tackett A J. Current state of melanoma diagnosis and treatment. Cancer Biol Ther, 2019, 20(11): 1366-1379.
5.	The Skin Cancer Foundation. Skin cancer facts & statistics, 2021 (2022-05) [2022-07-04]. https: //www.skincancer.org/skin-cancer-information/skin-cancer-facts.
6.	Haggenmüller S, Maron R C, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer Care, 2021, 156: 202-216.
7.	Rocha L K F L, Vilain R E, Scolyer R A, et al. Confocal microscopy, dermoscopy, and histopathology features of atypical intraepidermal melanocytic proliferations associated with evolution to melanoma in?situ. Int J Dermatol, 2022, 61(2): 167-174.
8.	Jitian Mihulecea C-R, Fr??il? S, Rotaru M. Clinical-dermoscopic similarities between atypical nevi and early stage melanoma. Exp Ther Med, 2021, 22(2): 854-854.
9.	Lodha S, Saggar S, Celebi J T, et al. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting. J Cutan Pathol, 2008, 35(4): 349-352.
10.	Ronen S, Al-Rohil R N, Keiser E, et al. Discordance in diagnosis of melanocytic lesions and its impact on clinical management. Arch Pathol Lab Med, 2021, 145(12): 1505-1515.
11.	Tolkach Y, Dohmg?rgen T, Toma M, et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell, 2020, 2(7): 411-418.
12.	Coudray N, Ocampo P S, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med, 2018, 24(10): 1559-1567.
13.	Bera K, Schalper K A, Rimm D L, et al. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol, 2019, 16(11): 703-715.
14.	Rakhlin A, Shvets A, Iglovikov V, et al. Deep convolutional neural networks for breast cancer histology image analysis// Campilho A, Karray F, ter Haar Romeny B. Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science. Cham: Springer, 2018, 10882: 737-744.
15.	韓繼能, 謝嘉偉, 顧松, 等. 基于全景病理圖像細胞密度和異型特征的膠質瘤自動分級. 生物醫學工程學雜志, 2021, 38(6): 1062-1071.
16.	王荃, 沈勤, 張澤林, 等. 基于深度學習和組織形態分析的肺癌基因突變預測. 生物醫學工程學雜志, 2020, 37(1): 10-18.
17.	Hekler A, Utikal J S, Enk A H, et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer Care, 2019, 118: 91-96.
18.	Hekler A, Utikal J S, Enk A H, et al. Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur J Cancer Care, 2019, 115: 79-83.
19.	Brinker T J, Schmitt M, Krieghoff-Henning E I, et al. Diagnostic performance of artificial intelligence for histologic melanoma recognition compared to 18 international expert pathologists. J Am Acad Dermatol, 2022, 86(3): 640-642.
20.	Li T, Li F, Zuo K. Pathologist-Level classification of melanoma disease pathologies using a convolutional neural network: A retrospective study of Chinese// Yao J, Xiao Y, You P, et al. The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021). Lecture Notes in Electrical Engineering. Singapore: Springer, 2022, 813: 833-839.
21.	Bejnordi B E, Litjens G, Timofeeva N, et al. Stain specific standardization of whole-slide histopathological images. IEEE Trans on Med Imag, 2016, 35(2): 404-415.
22.	Howard F M, Dolezal J, Kochanny S, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun, 2021, 12(1): 4423.
23.	Stiff K M, Franklin M J, Zhou Y, et al. Artificial intelligence and melanoma: A comprehensive review of clinical, dermoscopic, and histologic applications. Pigment Cell Melanoma Res, 2022, 35(2): 203-211.
24.	Ianni J D, Soans R E, Sankarapandian S, et al. Tailored for real-world: A whole slide image classification system validated on uncurated multi-site data emulating the prospective pathology workload. Sci Rep, 2020, 10(1): 3217.
25.	Shaban M T, Baur C, Navab N, et al. Staingan: Stain style transfer for digital histological images// 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI). Venice: IEEE, 2019: 953-956.
26.	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2223-2232.
27.	Ruifrok A C, Johnston D A. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol, 2001, 23(4): 291-299.
28.	Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans on Med Imag, 2016, 35(8): 1962-1971.
29.	張術昌, 袁梓洋, 王紅霞, 等. 面向組織病理學圖像的顏色遷移算法. 計算機輔助設計與圖形學學報, 2020, 32(12): 1890-1897.
30.	卞殷旭, 邢濤, 鄧偉杰, 等. 基于深度學習的色彩遷移生物醫學成像技術. 紅外與激光工程, 2022, 51(2): 339-356.
31.	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
32.	周濤, 劉赟璨, 陸惠玲, 等. ResNet及其在醫學圖像處理領域的應用:研究進展與挑戰. 電子與信息學報, 2022, 44(1): 149-167.
33.	H?hn J, Krieghoff-Henning E, Jutzi T B, et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur J Cancer Care, 2021, 149: 94-101.
34.	Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21(5): 34-41.

1. Swetter S M, Tsao H, Bichakjian C K, et al. Guidelines of care for the management of primary cutaneous melanoma. J Am Acad Dermatol, 2019, 80(1): 208-250.
2. Schadendorf D, van Akkooi A C J, Berking C, et al. Melanoma. Lancet, 2018, 392(10151): 971-984.
3. Piepkorn M W, Barnhill R L, Elder D E, et al. The MPATH-Dx reporting schema for melanocytic proliferations and melanoma. J Am Acad Dermatol, 2014, 70(1): 131-141.
4. Davis L E, Shalin S C, Tackett A J. Current state of melanoma diagnosis and treatment. Cancer Biol Ther, 2019, 20(11): 1366-1379.
5. The Skin Cancer Foundation. Skin cancer facts & statistics, 2021 (2022-05) [2022-07-04]. https: //www.skincancer.org/skin-cancer-information/skin-cancer-facts.
6. Haggenmüller S, Maron R C, Hekler A, et al. Skin cancer classification via convolutional neural networks: systematic review of studies involving human experts. Eur J Cancer Care, 2021, 156: 202-216.
7. Rocha L K F L, Vilain R E, Scolyer R A, et al. Confocal microscopy, dermoscopy, and histopathology features of atypical intraepidermal melanocytic proliferations associated with evolution to melanoma in?situ. Int J Dermatol, 2022, 61(2): 167-174.
8. Jitian Mihulecea C-R, Fr??il? S, Rotaru M. Clinical-dermoscopic similarities between atypical nevi and early stage melanoma. Exp Ther Med, 2021, 22(2): 854-854.
9. Lodha S, Saggar S, Celebi J T, et al. Discordance in the histopathologic diagnosis of difficult melanocytic neoplasms in the clinical setting. J Cutan Pathol, 2008, 35(4): 349-352.
10. Ronen S, Al-Rohil R N, Keiser E, et al. Discordance in diagnosis of melanocytic lesions and its impact on clinical management. Arch Pathol Lab Med, 2021, 145(12): 1505-1515.
11. Tolkach Y, Dohmg?rgen T, Toma M, et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell, 2020, 2(7): 411-418.
12. Coudray N, Ocampo P S, Sakellaropoulos T, et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med, 2018, 24(10): 1559-1567.
13. Bera K, Schalper K A, Rimm D L, et al. Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology. Nat Rev Clin Oncol, 2019, 16(11): 703-715.
14. Rakhlin A, Shvets A, Iglovikov V, et al. Deep convolutional neural networks for breast cancer histology image analysis// Campilho A, Karray F, ter Haar Romeny B. Image Analysis and Recognition. ICIAR 2018. Lecture Notes in Computer Science. Cham: Springer, 2018, 10882: 737-744.
15. 韓繼能, 謝嘉偉, 顧松, 等. 基于全景病理圖像細胞密度和異型特征的膠質瘤自動分級. 生物醫學工程學雜志, 2021, 38(6): 1062-1071.
16. 王荃, 沈勤, 張澤林, 等. 基于深度學習和組織形態分析的肺癌基因突變預測. 生物醫學工程學雜志, 2020, 37(1): 10-18.
17. Hekler A, Utikal J S, Enk A H, et al. Deep learning outperformed 11 pathologists in the classification of histopathological melanoma images. Eur J Cancer Care, 2019, 118: 91-96.
18. Hekler A, Utikal J S, Enk A H, et al. Pathologist-level classification of histopathological melanoma images with deep neural networks. Eur J Cancer Care, 2019, 115: 79-83.
19. Brinker T J, Schmitt M, Krieghoff-Henning E I, et al. Diagnostic performance of artificial intelligence for histologic melanoma recognition compared to 18 international expert pathologists. J Am Acad Dermatol, 2022, 86(3): 640-642.
20. Li T, Li F, Zuo K. Pathologist-Level classification of melanoma disease pathologies using a convolutional neural network: A retrospective study of Chinese// Yao J, Xiao Y, You P, et al. The International Conference on Image, Vision and Intelligent Systems (ICIVIS 2021). Lecture Notes in Electrical Engineering. Singapore: Springer, 2022, 813: 833-839.
21. Bejnordi B E, Litjens G, Timofeeva N, et al. Stain specific standardization of whole-slide histopathological images. IEEE Trans on Med Imag, 2016, 35(2): 404-415.
22. Howard F M, Dolezal J, Kochanny S, et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nat Commun, 2021, 12(1): 4423.
23. Stiff K M, Franklin M J, Zhou Y, et al. Artificial intelligence and melanoma: A comprehensive review of clinical, dermoscopic, and histologic applications. Pigment Cell Melanoma Res, 2022, 35(2): 203-211.
24. Ianni J D, Soans R E, Sankarapandian S, et al. Tailored for real-world: A whole slide image classification system validated on uncurated multi-site data emulating the prospective pathology workload. Sci Rep, 2020, 10(1): 3217.
25. Shaban M T, Baur C, Navab N, et al. Staingan: Stain style transfer for digital histological images// 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI). Venice: IEEE, 2019: 953-956.
26. Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 2223-2232.
27. Ruifrok A C, Johnston D A. Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol, 2001, 23(4): 291-299.
28. Vahadane A, Peng T, Sethi A, et al. Structure-preserving color normalization and sparse stain separation for histological images. IEEE Trans on Med Imag, 2016, 35(8): 1962-1971.
29. 張術昌, 袁梓洋, 王紅霞, 等. 面向組織病理學圖像的顏色遷移算法. 計算機輔助設計與圖形學學報, 2020, 32(12): 1890-1897.
30. 卞殷旭, 邢濤, 鄧偉杰, 等. 基于深度學習的色彩遷移生物醫學成像技術. 紅外與激光工程, 2022, 51(2): 339-356.
31. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 770-778.
32. 周濤, 劉赟璨, 陸惠玲, 等. ResNet及其在醫學圖像處理領域的應用:研究進展與挑戰. 電子與信息學報, 2022, 44(1): 149-167.
33. H?hn J, Krieghoff-Henning E, Jutzi T B, et al. Combining CNN-based histologic whole slide image analysis and patient data to improve skin cancer classification. Eur J Cancer Care, 2021, 149: 94-101.
34. Reinhard E, Adhikhmin M, Gooch B, et al. Color transfer between images. IEEE Comput Graph Appl, 2001, 21(5): 34-41.

Journal of Biomedical Engineering

Deep learning-based fully automated intelligent and precise diagnosis for melanocytic lesions

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content