Breast cancer lesion segmentation based on co-learning feature fusion and Transformer_Journal of Biomedical Engineering

Authors：

ZHAI Yuesong ¹ ,  CHEN Zhili ¹ ,  SHAO Dan ²

1. School of Computer Science and Engineering, Shenyang Jianzhu University, Shenyang 110168, P. R. China;
2. Department of Nuclear Medicine, Guangdong Academy of Medical Sciences, Guangdong Provincial People's Hospital, Guangzhou 519041, P. R. China;

Corresponding?author：

CHEN Zhili, Email: zzc@sjzu.edu.cn; SHAO Dan, Email: shaodan@gdph.org.cns

Keywords：

Positron emission tomography and computed tomography; Breast cancer lesion segmentation; Dual-path U-Net; Co-learning feature fusion; Transformer

DOI：

10.7507/1001-5515.202306063

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The PET/CT imaging technology combining positron emission tomography (PET) and computed tomography (CT) is the most advanced imaging examination method currently, and is mainly used for tumor screening, differential diagnosis of benign and malignant tumors, staging and grading. This paper proposes a method for breast cancer lesion segmentation based on PET/CT bimodal images, and designs a dual-path U-Net framework, which mainly includes three modules: encoder module, feature fusion module and decoder module. Among them, the encoder module uses traditional convolution for feature extraction of single mode image; The feature fusion module adopts collaborative learning feature fusion technology and uses Transformer to extract the global features of the fusion image; The decoder module mainly uses multi-layer perceptron to achieve lesion segmentation. This experiment uses actual clinical PET/CT data to evaluate the effectiveness of the algorithm. The experimental results show that the accuracy, recall and accuracy of breast cancer lesion segmentation are 95.67%, 97.58% and 96.16%, respectively, which are better than the baseline algorithm. Therefore, it proves the rationality of the single and bimodal feature extraction method combining convolution and Transformer in the experimental design of this article, and provides reference for feature extraction methods for tasks such as multimodal medical image segmentation or classification.

Citation： ZHAI Yuesong, CHEN Zhili, SHAO Dan. Breast cancer lesion segmentation based on co-learning feature fusion and Transformer. Journal of Biomedical Engineering, 2024, 41(2): 237-245. doi: 10.7507/1001-5515.202306063 Copy

1.	Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2.	Lei S, Zheng R, Zhang S, et al. Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med, 2021, 18(3): 900-909.
3.	Shi Z, Lin J, Wu Y, et al. Burden of cancer and changing cancer spectrum among older adults in China: Trends and projections to 2030. Cancer Epidemiol, 2022, 76: 102068.
4.	Zeng H, Chen W, Zheng R, et al. Changing cancer survival in China during 2003-15: a pooled analysis of 17 population-based cancer registries. Lancet Glob Health, 2018, 6(5): e555-e567.
5.	Hildebrandt M G, Naghavi-Behzad M, Vogsen M. A role of FDG-PET/CT for response evaluation in metastatic breast cancer?. Semin Nucl Med, 2022, 52(5): 520-530.
6.	Chen W. Clinical application of PET in pediatric brain tumors. PET Clin, 2008, 3(4): 517-529.
7.	辛陽. CT成像技術的發展及技術特點. 科技創新導報, 2018, 15(4): 132-133.
8.	Li Y, Dai Y, Guo Y, et al. Correlation analysis of sup18/sup F-FDG PET/CT for the staging and treatment effect assessment of breast cancer. Journal of X-Ray Science and Technology, 2019, 27(6): 1131-1144.
9.	Cruz-Roa A, Arevalo J, Judkins A, et al. A method for medulloblastoma tumor differentiation based on convolutional neural networks and transfer learning//International Symposium on Medical Information Processing and Analysis. International Society for Optics and Photonics, 2015, DOI: 10.1117/12.2208825.
10.	趙旭. 基于醫學先驗的多尺度乳腺超聲腫瘤實例分割方法. 哈爾濱: 哈爾濱工業大學, 2019.
11.	徐勝舟, 程時宇. 基于全卷積神經網絡遷移學習的乳腺腫塊圖像分割. 中南民族大學學報: 自然科學版, 2019, 38(2): 278-284.
12.	Al-Antari M A, Al-Masni M A, Choi M T, et al. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inform, 2018, 117: 44-54.
13.	Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015: 234–241.
14.	Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//European Conference on Computer Vision. Springer, Cham, 2018: 833-851.
15.	Zhuang J. LadderNet: multi-path networks based on U-Net for medical image segmentation. arXiv preprint, 2018, DOI: 10.48550/arXiv.1810.07810.
16.	Kumar A, Fulham M, Feng D, et al. Co-learning feature fusion maps from PET-CT images of lung cancer. IEEE Trans Med Imaging, 2019, 39(1): 204-217.
17.	Xiao X, Lian S, Luo Z, et al. Weighted Res-UNet for high-quality retina vessel segmentation//2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE Computer Society, 2018: 327-331.
18.	Guan S, Khan A A, Sikdar S, et al. Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform, 2020, 24(2): 568-576.
19.	Isensee F, J?ger P F, Kohl S A A, et al. Automated design of deep learning methods for biomedical image segmentation. arXiv preprint, 2019, arXiv: 1904.08128.
20.	Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, DOI: 10.48550/arXiv.2102.04306.
21.	Jain J, Li J, Chiu M T, et al. OneFormer: one transformer to rule universal image segmentation. arXiv preprint, 2022, DOI: 10.48550/arXiv.2211.06220.
22.	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint, 2020, DOI: 10.48550/arXiv.2010.11929.
23.	Xie E, Wang W, Yu Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv preprint, 2021, DOI: 10.48550/arXiv.2105.15203.
24.	Liu Z, Lin Y, Cao Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows. arXiv preprint, 2021. DOI: 10.48550/arXiv.2103.14030.
25.	Ji S, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell, 2013, 35(1): 221-231.
26.	Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. arXiv preprint, 2017. DOI: 10.48550/arXiv.1706.03762.
27.	余輝, 張書旭. 4DCT圖像二維配準與三維配準的區別. 醫療裝備, 2011, 24(7): 6-8.
28.	胡德文, 陳廣學, 朱劍銘, 等. 基于ROI的醫學圖像預處理技術研究. 醫學影像學雜志, 2018, 28(4): 585-589.
29.	Fukuda T, Fernandez R, Rosenberg A, et al. Data augmentation improves recognition of foreign accented speech. Interspeech, 2018, 18(9): 2409-2413.
30.	侯向丹, 李紫宇, 牛敬鈺, 等. 結合注意力機制和多路徑U-Net的視網膜血管分割. 計算機輔助設計與圖形學學報, 2023, 35(1): 55-65.

1. Sung H, Ferlay J, Siegel R L, et al. Global Cancer Statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2. Lei S, Zheng R, Zhang S, et al. Breast cancer incidence and mortality in women in China: temporal trends and projections to 2030. Cancer Biol Med, 2021, 18(3): 900-909.
3. Shi Z, Lin J, Wu Y, et al. Burden of cancer and changing cancer spectrum among older adults in China: Trends and projections to 2030. Cancer Epidemiol, 2022, 76: 102068.
4. Zeng H, Chen W, Zheng R, et al. Changing cancer survival in China during 2003-15: a pooled analysis of 17 population-based cancer registries. Lancet Glob Health, 2018, 6(5): e555-e567.
5. Hildebrandt M G, Naghavi-Behzad M, Vogsen M. A role of FDG-PET/CT for response evaluation in metastatic breast cancer?. Semin Nucl Med, 2022, 52(5): 520-530.
6. Chen W. Clinical application of PET in pediatric brain tumors. PET Clin, 2008, 3(4): 517-529.
7. 辛陽. CT成像技術的發展及技術特點. 科技創新導報, 2018, 15(4): 132-133.
8. Li Y, Dai Y, Guo Y, et al. Correlation analysis of sup18/sup F-FDG PET/CT for the staging and treatment effect assessment of breast cancer. Journal of X-Ray Science and Technology, 2019, 27(6): 1131-1144.
9. Cruz-Roa A, Arevalo J, Judkins A, et al. A method for medulloblastoma tumor differentiation based on convolutional neural networks and transfer learning//International Symposium on Medical Information Processing and Analysis. International Society for Optics and Photonics, 2015, DOI: 10.1117/12.2208825.
10. 趙旭. 基于醫學先驗的多尺度乳腺超聲腫瘤實例分割方法. 哈爾濱: 哈爾濱工業大學, 2019.
11. 徐勝舟, 程時宇. 基于全卷積神經網絡遷移學習的乳腺腫塊圖像分割. 中南民族大學學報: 自然科學版, 2019, 38(2): 278-284.
12. Al-Antari M A, Al-Masni M A, Choi M T, et al. A fully integrated computer-aided diagnosis system for digital X-ray mammograms via deep learning detection, segmentation, and classification. Int J Med Inform, 2018, 117: 44-54.
13. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2015: 234–241.
14. Chen L C, Zhu Y, Papandreou G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation//European Conference on Computer Vision. Springer, Cham, 2018: 833-851.
15. Zhuang J. LadderNet: multi-path networks based on U-Net for medical image segmentation. arXiv preprint, 2018, DOI: 10.48550/arXiv.1810.07810.
16. Kumar A, Fulham M, Feng D, et al. Co-learning feature fusion maps from PET-CT images of lung cancer. IEEE Trans Med Imaging, 2019, 39(1): 204-217.
17. Xiao X, Lian S, Luo Z, et al. Weighted Res-UNet for high-quality retina vessel segmentation//2018 9th International Conference on Information Technology in Medicine and Education (ITME). IEEE Computer Society, 2018: 327-331.
18. Guan S, Khan A A, Sikdar S, et al. Fully dense UNet for 2-D sparse photoacoustic tomography artifact removal. IEEE J Biomed Health Inform, 2020, 24(2): 568-576.
19. Isensee F, J?ger P F, Kohl S A A, et al. Automated design of deep learning methods for biomedical image segmentation. arXiv preprint, 2019, arXiv: 1904.08128.
20. Chen J, Lu Y, Yu Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation. arXiv preprint, 2021, DOI: 10.48550/arXiv.2102.04306.
21. Jain J, Li J, Chiu M T, et al. OneFormer: one transformer to rule universal image segmentation. arXiv preprint, 2022, DOI: 10.48550/arXiv.2211.06220.
22. Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint, 2020, DOI: 10.48550/arXiv.2010.11929.
23. Xie E, Wang W, Yu Z, et al. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv preprint, 2021, DOI: 10.48550/arXiv.2105.15203.
24. Liu Z, Lin Y, Cao Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows. arXiv preprint, 2021. DOI: 10.48550/arXiv.2103.14030.
25. Ji S, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Trans Pattern Anal Mach Intell, 2013, 35(1): 221-231.
26. Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need. arXiv preprint, 2017. DOI: 10.48550/arXiv.1706.03762.
27. 余輝, 張書旭. 4DCT圖像二維配準與三維配準的區別. 醫療裝備, 2011, 24(7): 6-8.
28. 胡德文, 陳廣學, 朱劍銘, 等. 基于ROI的醫學圖像預處理技術研究. 醫學影像學雜志, 2018, 28(4): 585-589.
29. Fukuda T, Fernandez R, Rosenberg A, et al. Data augmentation improves recognition of foreign accented speech. Interspeech, 2018, 18(9): 2409-2413.
30. 侯向丹, 李紫宇, 牛敬鈺, 等. 結合注意力機制和多路徑U-Net的視網膜血管分割. 計算機輔助設計與圖形學學報, 2023, 35(1): 55-65.

Journal of Biomedical Engineering

Breast cancer lesion segmentation based on co-learning feature fusion and Transformer

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content