Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST)_Chinese Journal of Evidence-Based Medicine

Authors：

YANG Nan ^1,2,3,4,5 , ZOU Kun ^2,3,4,5 , HE Siyi ^1,2,3,4,5 , ZENG Linan ^2,3,4,5 , LI Hailong ^2,3,4,5 , HUANG Liang ^2,3,4,5 , YI Qiusha ^2,3,4,5 , ZHANG Mingyue ⁶ , HUANG Chao ⁷ ,  WANG Qiang ⁷ ,  ZHANG Lingli ^2,3,4,5,8

1. West China School of Pharmacy, Sichuan University, Chengdu 610041, P. R. China;
2. Department of Pharmacy, West China Second University Hospital, Sichuan University, Chengdu 610041, P. R. China;
3. Evidence-based Pharmacy Center, West China Second University Hospital, Sichuan University, Chengdu 610041, P. R. China;
4. NMPA Key Laboratory for Technical Research on Drug Products in vitro and in vivo Correlation, Chengdu 610041, P. R. China;
5. Key Laboratory of Birth Defects and Related Diseases of Women and Children, Ministry of Education, Chengdu 610041, P. R. China;
6. School of Stomatology, Chinese Medical University, Shenyang 110002, P. R. China;
7. Medical Management Service Guidance Center, National Health Commission of the People’s Republic of China, Beijing 100044, P. R. China;
8. Chinese Evidence-based Medicine Center, West China Hospital, Sichuan University, Chengdu 610041, P. R. China;

Corresponding?author：

WANG Qiang, Email: jason2019@sina.cn; ZHANG Lingli, Email: zhanglingli@scu.edu.cn

Keywords：

Guideline Implementation Success Assessment Tool; Reliability; Validity

DOI：

10.7507/1672-2531.202308066

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Objective To evaluate the reliability and validity of Guideline Implementation Success Assessment Tool (A-GIST). Methods With the guideline for the prevention and treatment of type 2 diabetes mellitus in China (2020 edition) as the target guideline, health care providers and patients from different hospitals across the country were investigated by questionnaire using A-GIST. Spearman-Brown coefficient and Cronbach's α coefficient were used to evaluate the split-half reliability and internal consistency reliability, while the structural validity, convergent validity and discriminant validity were investigated by confirmatory factor analysis based on structural equation. Results The internal consistency reliability and split half reliability coefficients of the whole tool and each dimension ranged from 0.650 to 0.986. The scale-level content validity index (S-CVI/Ave) of content validity was 0.846. Confirmatory factor analysis showed that, the χ²/df of two sections of the tool were 8.695 and 6.123, respectively. The root mean square residual (RMR), the standard root mean square residual (SRMR) and the root mean square error of approximation (RMSEA) were under or almost under the threshold. Besides, the goodness-of-fit index (GFI) of them were 0.901 and 0.822, the adjusted goodness-of-fit index (AGFI) were 0.836 and 0.787, and the parsimonious normed fit index (PNFI) were 0.545 and 0.788, respectively. Conclusion Guideline Implementation Success Assessment Tool (A-GIST) was proved to be valid and reliable, and it shows that it is necessary to optimize the items under the dimensions of maintenance and evaluation of diagnosis and treatment effect in the future.

Citation： YANG Nan, ZOU Kun, HE Siyi, ZENG Linan, LI Hailong, HUANG Liang, YI Qiusha, ZHANG Mingyue, HUANG Chao, WANG Qiang, ZHANG Lingli. Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST). Chinese Journal of Evidence-Based Medicine, 2024, 24(8): 904-909. doi: 10.7507/1672-2531.202308066 Copy

1.	Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust. Washington DC: National Academies Press, 2011.
2.	Pronovost PJ. Enhancing physicians' use of clinical guidelines. JAMA, 2013, 310(23): 2501-2502.
3.	中華醫學會雜志社指南與標準研究中心, 中國醫學科學院循證評價與指南研究創新單元, 世界衛生組織指南實施與知識轉化合作中心. 2021年醫學期刊發表中國指南和共識的科學性、透明性和適用性的評級. 中華醫學雜志, 2022, 102(30): 2319-2328.
4.	郭強強, 荀楊芹, 劉輝, 等. 臨床實踐指南的注冊. 協和醫學雜志, 2023, 14(1): 203-208.
5.	Liu M, Zhang C, Zha Q, et al. A national survey of Chinese medicine doctors and clinical practice guidelines in China. BMC Complement Altern Med, 2017, 17(1): 451.
6.	陳耀龍, 史乾靈, 趙俊強, 等. 從知到行: 跨越指南理論與實踐的鴻溝. 協和醫學雜志, 2020, 11(6): 746-753.
7.	Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak, 2005, 5: 23.
8.	Ierano C, Ayton D, Peel T, et al. Evaluating the implementability of antibiotic surgical prophylaxis guidelines. Infect Dis Health, 2020, 25(1): 11-21.
9.	Maue SK, Segal R, Kimberlin CL, et al. Predicting physician guideline compliance: an assessment of motivators and perceived barriers. Am J Manag Care, 2004, 10(6): 383-391.
10.	Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst, 2020, 18(1): 74.
11.	Stewart D, Al Hail M, Al-Shaibi S, et al. A scoping review of theories used to investigate clinician adherence to clinical practice guidelines. Int J Clin Pharm, 2023, 45(1): 52-63.
12.	Deng Q, Liu W. Utilization of clinical practice guideline on antimicrobial in China: an exploratory survey on multilevel determinants. BMC Health Serv Res, 2020, 20(1): 282.
13.	孫振球, 徐勇勇. 醫學統計學. 第4版. 北京: 人民衛生出版社, 2020.
14.	李燦, 辛玲. 調查問卷的信度與效度的評價方法研究. 中國衛生統計, 2008, 25(5): 541-544.
15.	史靜琤, 莫顯昆, 孫振球. 量表編制中內容效度指數的應用. 中南大學學報(醫學版), 2012, 37(2): 152-155.
16.	中華醫學會糖尿病學分會. 中國2型糖尿病防治指南(2020年版). 中華糖尿病雜志, 2021, 13(4): 315-409.
17.	Wiercioch W, Akl EA, Santesso N, et al. Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development. CMAJ, 2020, 192(40): E1138-E1145.
18.	Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
19.	黃超, 李海龍, 張川, 等. 指南臨床適用性評價實證研究: 以糖尿病指南臨床適用性評價為例. 中國循證醫學雜志, 2020, 20(5): 536-542.
20.	Tabachnick BG, Fidell LS. Using multivariate statistics (5th Edition). New York: Allyn and Bacon, 2007.
21.	張淼, 刁莎, 曾力楠, 等. 指南臨床適用性評價工具(2. 0版)信效度分析. 中國循證醫學雜志, 2023, 23(1): 80-84.
22.	李海龍, 張川, 黃超, 等. 指南臨床適用性評價工具信效度分析. 中國循證醫學雜志, 2020, 20(4): 378-382.
23.	付會斌, 林小鵬, 孔丹莉, 等. 世界衛生組織生存質量測定簡表在高校學科帶頭人生存質量評價中的信度與效度研究. 山西醫藥雜志, 2012, 41(9): 867-869.
24.	邱皓政, 林碧芳. 結構方程模型的原理與應用(第二版). 北京: 中國輕工業出版社, 2019.
25.	汪亞萍, 景文展, 吳俁, 等. SF-12量表評價我國接受ART的HIV/AIDS患者生存質量的信效度分析. 中國艾滋病性病, 2022, 28(4): 411-415.
26.	徐迪, 端瓊麗, 潘玲, 等. WHOQOL-BREF用于評價HIV感染者生活質量的信、效度研究. 中華疾病控制雜志, 2022, 26(5): 576-582.
27.	Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
28.	AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
29.	曾力楠, 易秋莎, 張川, 等. 制訂指南臨床適用性評價工具. 中國循證醫學雜志, 2020, 20(4): 466-475.

1. Institute of Medicine Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical practice guidelines we can trust. Washington DC: National Academies Press, 2011.
2. Pronovost PJ. Enhancing physicians' use of clinical guidelines. JAMA, 2013, 310(23): 2501-2502.
3. 中華醫學會雜志社指南與標準研究中心, 中國醫學科學院循證評價與指南研究創新單元, 世界衛生組織指南實施與知識轉化合作中心. 2021年醫學期刊發表中國指南和共識的科學性、透明性和適用性的評級. 中華醫學雜志, 2022, 102(30): 2319-2328.
4. 郭強強, 荀楊芹, 劉輝, 等. 臨床實踐指南的注冊. 協和醫學雜志, 2023, 14(1): 203-208.
5. Liu M, Zhang C, Zha Q, et al. A national survey of Chinese medicine doctors and clinical practice guidelines in China. BMC Complement Altern Med, 2017, 17(1): 451.
6. 陳耀龍, 史乾靈, 趙俊強, 等. 從知到行: 跨越指南理論與實踐的鴻溝. 協和醫學雜志, 2020, 11(6): 746-753.
7. Shiffman RN, Dixon J, Brandt C, et al. The GuideLine Implementability Appraisal (GLIA): development of an instrument to identify obstacles to guideline implementation. BMC Med Inform Decis Mak, 2005, 5: 23.
8. Ierano C, Ayton D, Peel T, et al. Evaluating the implementability of antibiotic surgical prophylaxis guidelines. Infect Dis Health, 2020, 25(1): 11-21.
9. Maue SK, Segal R, Kimberlin CL, et al. Predicting physician guideline compliance: an assessment of motivators and perceived barriers. Am J Manag Care, 2004, 10(6): 383-391.
10. Correa VC, Lugo-Agudelo LH, Aguirre-Acevedo DC, et al. Individual, health system, and contextual barriers and facilitators for the implementation of clinical practice guidelines: a systematic metareview. Health Res Policy Syst, 2020, 18(1): 74.
11. Stewart D, Al Hail M, Al-Shaibi S, et al. A scoping review of theories used to investigate clinician adherence to clinical practice guidelines. Int J Clin Pharm, 2023, 45(1): 52-63.
12. Deng Q, Liu W. Utilization of clinical practice guideline on antimicrobial in China: an exploratory survey on multilevel determinants. BMC Health Serv Res, 2020, 20(1): 282.
13. 孫振球, 徐勇勇. 醫學統計學. 第4版. 北京: 人民衛生出版社, 2020.
14. 李燦, 辛玲. 調查問卷的信度與效度的評價方法研究. 中國衛生統計, 2008, 25(5): 541-544.
15. 史靜琤, 莫顯昆, 孫振球. 量表編制中內容效度指數的應用. 中南大學學報(醫學版), 2012, 37(2): 152-155.
16. 中華醫學會糖尿病學分會. 中國2型糖尿病防治指南(2020年版). 中華糖尿病雜志, 2021, 13(4): 315-409.
17. Wiercioch W, Akl EA, Santesso N, et al. Assessing the process and outcome of the development of practice guidelines and recommendations: PANELVIEW instrument development. CMAJ, 2020, 192(40): E1138-E1145.
18. Moher D, Schulz KF, Simera I, et al. Guidance for developers of health research reporting guidelines. PLoS Med, 2010, 7(2): e1000217.
19. 黃超, 李海龍, 張川, 等. 指南臨床適用性評價實證研究: 以糖尿病指南臨床適用性評價為例. 中國循證醫學雜志, 2020, 20(5): 536-542.
20. Tabachnick BG, Fidell LS. Using multivariate statistics (5th Edition). New York: Allyn and Bacon, 2007.
21. 張淼, 刁莎, 曾力楠, 等. 指南臨床適用性評價工具(2. 0版)信效度分析. 中國循證醫學雜志, 2023, 23(1): 80-84.
22. 李海龍, 張川, 黃超, 等. 指南臨床適用性評價工具信效度分析. 中國循證醫學雜志, 2020, 20(4): 378-382.
23. 付會斌, 林小鵬, 孔丹莉, 等. 世界衛生組織生存質量測定簡表在高校學科帶頭人生存質量評價中的信度與效度研究. 山西醫藥雜志, 2012, 41(9): 867-869.
24. 邱皓政, 林碧芳. 結構方程模型的原理與應用(第二版). 北京: 中國輕工業出版社, 2019.
25. 汪亞萍, 景文展, 吳俁, 等. SF-12量表評價我國接受ART的HIV/AIDS患者生存質量的信效度分析. 中國艾滋病性病, 2022, 28(4): 411-415.
26. 徐迪, 端瓊麗, 潘玲, 等. WHOQOL-BREF用于評價HIV感染者生活質量的信、效度研究. 中華疾病控制雜志, 2022, 26(5): 576-582.
27. Brouwers MC, Kho ME, Browman GP, et al. Development of the AGREE II, part 1: performance, usefulness and areas for improvement. CMAJ, 2010, 182(10): 1045-1052.
28. AGREE Collaboration. Development and validation of an international appraisal instrument for assessing the quality of clinical practice guidelines: the AGREE project. Qual Saf Health Care, 2003, 12(1): 18-23.
29. 曾力楠, 易秋莎, 張川, 等. 制訂指南臨床適用性評價工具. 中國循證醫學雜志, 2020, 20(4): 466-475.

Chinese Journal of Evidence-Based Medicine

Reliability and validity analysis of Guideline Implementation Success Assessment Tool (A-GIST)

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content