Application of large language models in sarcopenia diagnosis and treatment: a comparative study with clinical decision-making by physicians_Chinese Journal of Evidence-Based Medicine

Authors：

MA Zebing ¹ , LIU Yibing ¹ , FAN Huayu ² , YANG Chaofan ² , KONG Liang ² , CHEN Rui ² ,  CAO Xiangyang ^2,3,4,5

1. Hunan University of Chinese Medicine, Changsha 410000, P. R. China;
2. Luoyang Orthopedic Hospital of Henan Province (Orthopedic Hospital of Henan Province), Zhengzhou 450000, P. R. China;
3. Institute of Intelligent Medical and Bioengineering, Henan Academy of Traditional Chinese Medicine Sciences, Zhengzhou 450000, P. R. China;
4. Henan Province Artificial Intelligence Engineering Research Center for Bone Injury Rehabilitation, Zhengzhou 450000, P. R. China;
5. Henan University of Chinese Medicine, Zhengzhou 450000, P. R. China;

Corresponding?author：

CAO Xiangyang, Email: Cxy1260@126.com

Keywords：

Sarcopenia; Artificial intelligence; Large language model; Clinicians; Diagnosis and treatment applications

DOI：

10.7507/1672-2531.202411134

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To evaluate the quality differences in recommendations generated by large language models (LLM) and clinical practitioners for sarcopenia-related questions. Methods A sarcopenia knowledge base was constructed based on the latest domestic and international research and consensus guidelines. Using the Python environment, a locally deployed and sarcopenia-focused hybrid vertical LLM (referred to as LC) was implemented via LangChain-LLM. Eight fixed questions covering etiology, diagnosis, and prevention were selected, along with eight virtual patient cases. The evaluation team assessed the quality of answers generated by LC and written by clinical practitioners. Quantitative analysis was performed on the precision, recall, and F₁ scores (harmonic mean of precision and recall) of treatment recommendations. Results The responses were generally perceived as "possibly written by humans or AI", with a stronger inclination toward being AI-generated, although the accuracy of such judgments was low. Regarding answer quality attributes, LC's responses were superior to those of clinical practitioners in guideline consistency (P<0.01), exhibited similar acceptability (P>0.05), showed better practicality (P<0.05), and had a lower proportion of "1–2 errors" (P<0.05). Quantitative analysis of treatment recommendations indicated that LC and GPT-4.0 outperformed clinical practitioners in recall and F₁ scores (P<0.05), with minimal differences between LC and GPT-4.0. Conclusion The locally deployed sarcopenia-focused hybrid vertical LLM demonstrates high accuracy and applicability in addressing sarcopenia-related issues, outperforming clinical practitioners and exhibiting strong clinical decision-support capabilities.

Citation： MA Zebing, LIU Yibing, FAN Huayu, YANG Chaofan, KONG Liang, CHEN Rui, CAO Xiangyang. Application of large language models in sarcopenia diagnosis and treatment: a comparative study with clinical decision-making by physicians. Chinese Journal of Evidence-Based Medicine, 2025, 25(7): 775-782. doi: 10.7507/1672-2531.202411134 Copy

1.	Cruz-Jentoft AJ, Sayer AA. Sarcopenia. Lancet, 2019, 393(10191): 2636-2646.
2.	Polyzos SA, Vachliotis ID, Mantzoros CS. Sarcopenia, sarcopenic obesity and nonalcoholic fatty liver disease. Metabolism, 2023 Oct: 147: 155676.
3.	No authors listed. Will ChatGPT transform healthcare. Nat Med, 2023, 29(3): 505-506.
4.	Moon JH, Lee H, Shin W, et al. Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training. IEEE J Biomed Health Inform, 2022, 26(12): 6070-6080.
5.	榮國偉, 孫寶, 王文婧, 等. 大語言模型在數學建模領域的應用探索. 科技創新與應用, 2024, 14(22): 18-21.
6.	Yaneva V, Baldwin P, Jurich DP, et al. Examining ChatGPT performance on USMLE sample items and implications for assessment. Acad Med, 2024, 99(2): 192-197.
7.	胡亦新, 譯. 克魯茲·詹托夫特美. 肌少癥: 原書第2版. 北京: 中國科學技術出版社, 2023.
8.	孫建琴. 肌不可失: 肌少癥防治全攻略. 上海: 上海科學普及出版社, 2022.
9.	閆炳霖. 骨骼肌減少癥. 北京: 科學技術文獻出版社, 2021.
10.	Benary M, Wang XD, Schmidt M, et al. Leveraging large language models for decision support in personalized oncology. JAMA Netw Open, 2023, 6(11): e2343689.
11.	邢丹, 王斌, 侯云飛, 等. 中國骨科臨床實踐指南的質量評價. 中國循證醫學雜志, 2017, 17(3): 347-356.
12.	Bernstein IA, Zhang YV, Govil D, et al. Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions. JAMA Netw Open, 2023, 6(8): e2330320.
13.	夏宇隆, 蔣理, 但煒, 等. 基于人工智能的高血壓性腦出血醫療文本信息自動識別系統. 重慶醫科大學學報, 2023, 48(9): 1122-1127.
14.	張芊, 陳攀峰, 馮林坤, 等. PeMeBench: 中文兒科醫療問答基準測試方法. 大數據, 2024, 10(5): 28-44.
15.	陳龍飛, 高鑫, 侯皓天, 等. 生成式大語言模型在中文放射醫學領域的應用研究. 計算機科學與探索, 2024, 18(9): 2337-2348.
16.	Zhang C, Liu S, Zhou X, et al. Examining the role of large language models in orthopedics: systematic review. J Med Internet Res, 2024, 26: e59607.
17.	Zhang F, Sun J. Multi-feature intelligent oral english error correction based on few-shot learning technology. Comput Intell Neurosci, 2022, 2022: 2501693.
18.	Gallifant J, Fiske A, Levites Strekalova YA, et al. Peer review of GPT-4 technical report and systems card. PLOS Digit Health, 2024, 3(1): e0000417.
19.	Taylor N, Ghose U, Rohanian O, et al. Efficiency at scale: investigating the performance of diminutive language models in clinical tasks. Artif Intell Med, 2024, 157: 103002.
20.	Li J, Deng Y, Sun Q, et al. Benchmarking large language models in evidence-based medicine. IEEE J Biomed Health Inform, 2024, 21: PP.
21.	Kanjee Z, Crowe B, Rodman A. Accuracy of a generative artificial intelligence model in a complex diagnostic challenge. JAMA, 2023, 330(1): 78-80.
22.	Su Z, Jin K, Wu H, et al. Assessment of large language models in cataract care information provision: a quantitative comparison. Ophthalmol Ther, 2025, 14(1): 103-116.
23.	Fong A, Adams KT, Boxley C, et al. Does one size fit all. Developing an evaluation strategy to assess large language models for patient safety event report analysis. JAMIA Open, 2024, 7(4): ooae128.
24.	Wilhelm TI, Roos J, Kaczmarczyk R. Large language models for therapy recommendations across 3 clinical specialties: comparative study. J Med Internet Res, 2023, 25: e49324.
25.	Teckwani SH, Wong AH, Luke NV, et al. Accuracy and reliability of large language models in assessing learning outcomes achievement across cognitive domains. Adv Physiol Educ, 2024, 48(4): 904-914.

1. Cruz-Jentoft AJ, Sayer AA. Sarcopenia. Lancet, 2019, 393(10191): 2636-2646.
2. Polyzos SA, Vachliotis ID, Mantzoros CS. Sarcopenia, sarcopenic obesity and nonalcoholic fatty liver disease. Metabolism, 2023 Oct: 147: 155676.
3. No authors listed. Will ChatGPT transform healthcare. Nat Med, 2023, 29(3): 505-506.
4. Moon JH, Lee H, Shin W, et al. Multi-Modal Understanding and Generation for Medical Images and Text via Vision-Language Pre-Training. IEEE J Biomed Health Inform, 2022, 26(12): 6070-6080.
5. 榮國偉, 孫寶, 王文婧, 等. 大語言模型在數學建模領域的應用探索. 科技創新與應用, 2024, 14(22): 18-21.
6. Yaneva V, Baldwin P, Jurich DP, et al. Examining ChatGPT performance on USMLE sample items and implications for assessment. Acad Med, 2024, 99(2): 192-197.
7. 胡亦新, 譯. 克魯茲·詹托夫特美. 肌少癥: 原書第2版. 北京: 中國科學技術出版社, 2023.
8. 孫建琴. 肌不可失: 肌少癥防治全攻略. 上海: 上海科學普及出版社, 2022.
9. 閆炳霖. 骨骼肌減少癥. 北京: 科學技術文獻出版社, 2021.
10. Benary M, Wang XD, Schmidt M, et al. Leveraging large language models for decision support in personalized oncology. JAMA Netw Open, 2023, 6(11): e2343689.
11. 邢丹, 王斌, 侯云飛, 等. 中國骨科臨床實踐指南的質量評價. 中國循證醫學雜志, 2017, 17(3): 347-356.
12. Bernstein IA, Zhang YV, Govil D, et al. Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions. JAMA Netw Open, 2023, 6(8): e2330320.
13. 夏宇隆, 蔣理, 但煒, 等. 基于人工智能的高血壓性腦出血醫療文本信息自動識別系統. 重慶醫科大學學報, 2023, 48(9): 1122-1127.
14. 張芊, 陳攀峰, 馮林坤, 等. PeMeBench: 中文兒科醫療問答基準測試方法. 大數據, 2024, 10(5): 28-44.
15. 陳龍飛, 高鑫, 侯皓天, 等. 生成式大語言模型在中文放射醫學領域的應用研究. 計算機科學與探索, 2024, 18(9): 2337-2348.
16. Zhang C, Liu S, Zhou X, et al. Examining the role of large language models in orthopedics: systematic review. J Med Internet Res, 2024, 26: e59607.
17. Zhang F, Sun J. Multi-feature intelligent oral english error correction based on few-shot learning technology. Comput Intell Neurosci, 2022, 2022: 2501693.
18. Gallifant J, Fiske A, Levites Strekalova YA, et al. Peer review of GPT-4 technical report and systems card. PLOS Digit Health, 2024, 3(1): e0000417.
19. Taylor N, Ghose U, Rohanian O, et al. Efficiency at scale: investigating the performance of diminutive language models in clinical tasks. Artif Intell Med, 2024, 157: 103002.
20. Li J, Deng Y, Sun Q, et al. Benchmarking large language models in evidence-based medicine. IEEE J Biomed Health Inform, 2024, 21: PP.
21. Kanjee Z, Crowe B, Rodman A. Accuracy of a generative artificial intelligence model in a complex diagnostic challenge. JAMA, 2023, 330(1): 78-80.
22. Su Z, Jin K, Wu H, et al. Assessment of large language models in cataract care information provision: a quantitative comparison. Ophthalmol Ther, 2025, 14(1): 103-116.
23. Fong A, Adams KT, Boxley C, et al. Does one size fit all. Developing an evaluation strategy to assess large language models for patient safety event report analysis. JAMIA Open, 2024, 7(4): ooae128.
24. Wilhelm TI, Roos J, Kaczmarczyk R. Large language models for therapy recommendations across 3 clinical specialties: comparative study. J Med Internet Res, 2023, 25: e49324.
25. Teckwani SH, Wong AH, Luke NV, et al. Accuracy and reliability of large language models in assessing learning outcomes achievement across cognitive domains. Adv Physiol Educ, 2024, 48(4): 904-914.

Chinese Journal of Evidence-Based Medicine

Application of large language models in sarcopenia diagnosis and treatment: a comparative study with clinical decision-making by physicians

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content