Constructing a real-world evidence evaluation system based on Delphi method_Chinese Journal of Evidence-Based Medicine

Authors：

YAO Zhenge ¹ , ZHOU Jiawei ¹ , LU Mengyi ^1,2 , YOU Dongfang ¹ ,  ZHAO Yang ^1,2,3

1. Department of Biostatistics, School of Public Health, Nanjing Medical University, Nanjing 211166, P. R. China;
2. National Vaccine Innovation Platform, Nanjing Medical University, Nanjing 211166, P. R. China;
3. Jiangsu Key Laboratory of Cancer Biomarkers, Prevention and Treatment, Collaborative Innovation Center for Cancer Personalized Medicine, Nanjing 211166, P. R. China;

Corresponding?author：

ZHAO Yang, Email: yzhao@njmu.edu.cn

Keywords：

Real-world evidence; Delphi; Quality evaluation; Identification of bias

DOI：

10.7507/1672-2531.202402025

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective We constructed a real-world evidence evaluation system to provide reference for obtaining high-quality evidence in evidence-based medicine.Methods Through the investigation and analysis of the key factors influencing the real-world research evidence, combined with domestic and foreign literature and evaluation tools, we preliminarily constructed the indicators of the real-world evidence evaluation system, then consulted experts in related fields by the Delphi method, modified and determined the final evaluation indicators. Results The indicators of the final real-world evidence evaluation system included 40 items. The recovery efficiencies of the two rounds of expert consultation were 88.2% and 100%; The expert coordination coefficients were 0.174 (P<0.001) and 0.189 (P<0.001). After the second round of consultation, the mean of Likert scale in the range of 3.73~4.93, and the coefficient of variation varied in the range of 0.05~0.21. Conclusion The real-world evidence evaluation system constructed in this study has certain reliability and scientificity, which can provide a basis and help for the transformation of real-world research into high-quality evidence.

Citation： YAO Zhenge, ZHOU Jiawei, LU Mengyi, YOU Dongfang, ZHAO Yang. Constructing a real-world evidence evaluation system based on Delphi method. Chinese Journal of Evidence-Based Medicine, 2024, 24(10): 1156-1161. doi: 10.7507/1672-2531.202402025 Copy

1.	國家藥品監督管理局. 真實世界證據支持藥物研發與審評的指導原則(試行). 2023.
2.	Wang SV, Schneeweiss S. Emulation of randomized clinical trials with nonrandomized database analyses: results of 32 clinical trials. JAMA, 2023, 329(16): 1376-1385.
3.	Sheldrick RC. Randomized trials vs real-world evidence: how can both inform decision-making. JAMA, 2023, 329(16): 1352-1353.
4.	Roberts MH, Ferguson GT. Real-world evidence: bridging gaps in evidence to guide payer decisions. Pharmacoecon Open, 2021, 5(1): 3-11.
5.	Scavone C, di Mauro G, Mascolo A, et al. The new paradigms in clinical research: from early access programs to the novel therapeutic approaches for unmet medical needs. Front Pharmacol, 2019, 10: 111.
6.	Morales DR, Arlett P. RCTs and real world evidence are complementary, not alternatives. BMJ, 2023, 381: 736.
7.	Purpura CA, Garry EM, Honig N, et al. The role of real-world evidence in fda-approved new drug and biologics license applications. Clin Pharmacol Ther, 2022, 111(1): 135-144.
8.	廖茜雯, 姚晨, 張軍, 等. 真實世界數據和證據在我國臨床決策中的應用現狀. 中國食品藥品監管, 2023, (10): 24-35.
9.	陳薇, 方賽男, 劉建平, 等. 國際循證醫學證據分級體系的發展與現狀. 中國中西醫結合雜志, 2017, 37(12): 1413-1419.
10.	Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary. Control Clin Trials, 1996, 17(1): 1-12.
11.	Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 2008, 336(7650): 924-926.
12.	曹雪, 孟祥然, 王馨, 等. 真實世界觀察性研究的質量評價工具ArRoWS解讀. 中國循證醫學雜志, 2023, 23(2): 227-232.
13.	Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 2016, 355: i4919.
14.	廖星, 章軼立, 謝雁鳴. 真實世界研究標準: RECORD清單和GRACE清單的解讀. 中國中藥雜志, 2015, 40(24): 4734-4738.
15.	陳英耀, 倪明, 胡獻之, 等. 公立醫療機構公益性評價指標篩選—基于德爾菲專家咨詢法. 中國衛生政策研究, 2012, 5(1): 6-10.
16.	葉瑩, 姬艷芳, 張璐, 等. 運用Likert 5級評分法對免疫規劃互聯網+培訓的效果評價. 河南預防醫學雜志, 2019, 30(9): 701-703.
17.	宋辰斐, 薛征, 吳淑艷, 等. 《藥物香佩療法預防小兒反復呼吸道感染治未病實踐指南》香佩藥物的德爾菲法結果分析. 中華中醫藥雜志, 2017, 32(1): 139-141.
18.	馬雪顏, 于河, 吳力群, 等. 基于德爾菲法的兒童胃腸積熱評價量表條目篩選. 中華中醫藥雜志, 2020, 35(2): 851-854.
19.	肖爽, 朱雪琦, 王彥, 等. 基于德爾菲法的臨床科研課題全過程質量控制指標體系構建研究. 中醫藥管理雜志, 2023, 31(21): 1-6.
20.	Schad F, Thronicke A. Real-world evidence-current developments and perspectives. Int J Environ Res Public Health, 2022, 19(16): 10159.
21.	艾飛玲, 胡葵茹, 石鈺霖, 等. 基于紐卡斯爾-渥太華量表對中國吸煙隊列研究文獻的質量評價. 中華疾病控制雜志, 2021, 25(6): 722-729.
22.	Radenkovic D, Keogh SB, Maruthappu M. Data science in modern evidence-based medicine. J R Soc Med, 2019, 112(12): 493-494.
23.	范美玉. 高質量循證醫學證據獲取與應用研究. 協和醫學雜志, 2023, 14(1): 39-43.
24.	Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence - what is it and what can it tell us. N Engl J Med, 2016, 375(23): 2293-2297.
25.	McNair D, Lumpkin M, Kern S, et al. Use of RWE to inform regulatory, public health policy, and intervention priorities for the developing world. Clin Pharmacol Ther, 2022, 111(1): 44-51.

1. 國家藥品監督管理局. 真實世界證據支持藥物研發與審評的指導原則(試行). 2023.
2. Wang SV, Schneeweiss S. Emulation of randomized clinical trials with nonrandomized database analyses: results of 32 clinical trials. JAMA, 2023, 329(16): 1376-1385.
3. Sheldrick RC. Randomized trials vs real-world evidence: how can both inform decision-making. JAMA, 2023, 329(16): 1352-1353.
4. Roberts MH, Ferguson GT. Real-world evidence: bridging gaps in evidence to guide payer decisions. Pharmacoecon Open, 2021, 5(1): 3-11.
5. Scavone C, di Mauro G, Mascolo A, et al. The new paradigms in clinical research: from early access programs to the novel therapeutic approaches for unmet medical needs. Front Pharmacol, 2019, 10: 111.
6. Morales DR, Arlett P. RCTs and real world evidence are complementary, not alternatives. BMJ, 2023, 381: 736.
7. Purpura CA, Garry EM, Honig N, et al. The role of real-world evidence in fda-approved new drug and biologics license applications. Clin Pharmacol Ther, 2022, 111(1): 135-144.
8. 廖茜雯, 姚晨, 張軍, 等. 真實世界數據和證據在我國臨床決策中的應用現狀. 中國食品藥品監管, 2023, (10): 24-35.
9. 陳薇, 方賽男, 劉建平, 等. 國際循證醫學證據分級體系的發展與現狀. 中國中西醫結合雜志, 2017, 37(12): 1413-1419.
10. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary. Control Clin Trials, 1996, 17(1): 1-12.
11. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ, 2008, 336(7650): 924-926.
12. 曹雪, 孟祥然, 王馨, 等. 真實世界觀察性研究的質量評價工具ArRoWS解讀. 中國循證醫學雜志, 2023, 23(2): 227-232.
13. Sterne JA, Hernán MA, Reeves BC, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 2016, 355: i4919.
14. 廖星, 章軼立, 謝雁鳴. 真實世界研究標準: RECORD清單和GRACE清單的解讀. 中國中藥雜志, 2015, 40(24): 4734-4738.
15. 陳英耀, 倪明, 胡獻之, 等. 公立醫療機構公益性評價指標篩選—基于德爾菲專家咨詢法. 中國衛生政策研究, 2012, 5(1): 6-10.
16. 葉瑩, 姬艷芳, 張璐, 等. 運用Likert 5級評分法對免疫規劃互聯網+培訓的效果評價. 河南預防醫學雜志, 2019, 30(9): 701-703.
17. 宋辰斐, 薛征, 吳淑艷, 等. 《藥物香佩療法預防小兒反復呼吸道感染治未病實踐指南》香佩藥物的德爾菲法結果分析. 中華中醫藥雜志, 2017, 32(1): 139-141.
18. 馬雪顏, 于河, 吳力群, 等. 基于德爾菲法的兒童胃腸積熱評價量表條目篩選. 中華中醫藥雜志, 2020, 35(2): 851-854.
19. 肖爽, 朱雪琦, 王彥, 等. 基于德爾菲法的臨床科研課題全過程質量控制指標體系構建研究. 中醫藥管理雜志, 2023, 31(21): 1-6.
20. Schad F, Thronicke A. Real-world evidence-current developments and perspectives. Int J Environ Res Public Health, 2022, 19(16): 10159.
21. 艾飛玲, 胡葵茹, 石鈺霖, 等. 基于紐卡斯爾-渥太華量表對中國吸煙隊列研究文獻的質量評價. 中華疾病控制雜志, 2021, 25(6): 722-729.
22. Radenkovic D, Keogh SB, Maruthappu M. Data science in modern evidence-based medicine. J R Soc Med, 2019, 112(12): 493-494.
23. 范美玉. 高質量循證醫學證據獲取與應用研究. 協和醫學雜志, 2023, 14(1): 39-43.
24. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence - what is it and what can it tell us. N Engl J Med, 2016, 375(23): 2293-2297.
25. McNair D, Lumpkin M, Kern S, et al. Use of RWE to inform regulatory, public health policy, and intervention priorities for the developing world. Clin Pharmacol Ther, 2022, 111(1): 44-51.

Chinese Journal of Evidence-Based Medicine

Constructing a real-world evidence evaluation system based on Delphi method

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content