The evidence system of traditional Chinese medicine based on the Grades of Recommendations Assessment, Development and Evaluation framework

Introduction

Evidence-based medicine (EBM) originated in the 1990s (1). It has become the current development trend and direction of clinical medicine. Decision-making in medicine must be based on the current best evidence, which is the key point of EBM, has been accepted by more and more people (2). “Evidence” is the key and core of EBM, while after nearly 20 years of development, the classical definition of evidence can not summarize the whole content of EBM (3,4). If we do not establish an evidence focused system, it will cause the misunderstanding and misuse of EBM (5,6), and will be a large drag on the development of EBM.

For thousands of years, clinical practice of traditional Chinese medicine (TCM), which is based on its unique theoretical system, has made an indelible contribution for health care. With the rapid development of TCM, there is a great need to assess the effectiveness and efficacy of TCM more scientifically and systematically (7), thus the integration of EBM and TCM has been the hot topic of discussion within the clinical research communities (8). For a comprehensive evidence-focused system, the first step is to establish the system of clinical evidence for TCM.

The aim of this paper is to discuss the methods of establishing the clinical evidence system of TCM with the clinical characteristics of its own.

Evidence is the fundamental of evidence-based practice (EBP), and the establishment of a comprehensive evidence system is a key requirement of effective development of EBM

In simple terms, EBM is defined as the current best evidence in health care decision-making in conjunction with expertise of the decision-makers and expectations and values of the patients, and its theoretical core is that decision-making in medicine must be based on current best evidence (1). EBM includes four steps: first, formulate the clinical questions; second, finding the current best evidence; third, critical appraisal of the evidence; fourth, applying the results to your patients. In clinical practice, EBM usually has two levels, first, finding and applying the evidence. Second, creating and applying the evidence which is the most challenging work of EBM. Therefore, “Evidence” is the core throughout EBP. The definition of evidence in EBM has gone through a series of evolutionary process (9). Currently, the understanding of evidence mainly refers to the effect and efficacy of interventions. However, for a more rounded definition of EBM, “Evidence” refers not only to the critical appraisal of evidence from clinical studies, but also the structure of the comprehensive evidence system that contains the evidence from clinical studies, patient values and preference, health economics and other factors. The foundation of EBP is how to establish such a comprehensive evidence system.

The premise and foundation of EBP in TCM is to establish the evidence system of TCM

As we discussed above, the core of EBM is evidence. The main approach of EBP is the development and implementation of the evidence-based clinical practice guidelines (CPG). Developing CPGs using scientific and standardized methods is also the current hot topic of discussion in TCM. Identifying, synthesizing and applying best available evidence are the key steps in guideline development process. Therefore, the establishment of the evidence system with the clinical characteristics of TCM is the key step in guideline development of TCM. At present, professionals of TCM have begun the research and development of CPGs, and there have been published some CPGs. However, the quality and applicability of the current guidelines do not meet the current requirement (10). As the deficiency in methodology, it is difficult to collect and create the special evidence with the clinical characteristics of TCM, therefore, establishing the evidence system with the clinical characteristics of TCM, which is based on the concept of EBM, is the core in developing the guidelines of TCM (11).

A new definition has been proposed by Grades of Recommendations Assessment^* Development and Evaluation (GRADE) approach^* which meets the potential requirement of EBP

There are two fundamental principles of EBM (12), one is “hierarchy of evidence”, which means evidence available in any clinical decision-making can be arranged in order of strength based on likelihood of freedom from error. The other is “Insufficiency of evidence alone”, that means evidence alone is never sufficient for decision-making. It has to be integrated with clinical expertise and patients’ expectations and values. Patients’ preferences and values are also very important in EBP. Preference is a basic concept in the theory of value of microeconomics, the concept of preference is subjective and relative, and it has an obvious individual difference, it also shows the characteristics of group. Therefore, evidence of patients’ preferences and values is an important component of the comprehensive evidence system.

The GRADE system has been proposed by GRADE working group in 2004 (13). GRADE offers a system for rating quality of evidence and grading strength of recommendations in guidelines. It also offers a transparent and structured process for developing evidence summaries for guidelines in health care and for carrying out the steps involved in developing recommendations. GRADE is “outcome centric”, that means rating is made for each outcome and quality may differ from one outcome to another within a single study and across the evidence body. In the GRADE approach, randomized controlled trials (RCTs) are regarded as high-quality evidence and observational studies as low-quality evidence. RCTs could be downgraded due to “risk of bias”, “inconsistency”, “indirectness”, “imprecision” and “publication bias” (14-17), while observational studies could be upgraded due to “large effect”, “dose response”, “all plausible confounding” (18). Ultimately, the quality of evidence for each outcome could be assessed as high to very low. Based on the GRADE approach, in decision-making, people should consider the quality of evidence, the balance between desirable and undesirable outcomes, the application of patients’ values and preference along with the resource use (19). The whole process is open and transparent, and reflects the concept of EBM. The GRADE approach provides a scientific system for rating the quality of evidence and also indicates that a comprehensive evidence system which include the current best evidence, the balance between desirable and undesirable outcomes, patients’ values and preference and the resource use, is the complete evidence system for EBM.

The GRADE approach is the right way for establishing the evidence system of TCM

The advantage of establishing the evidence system based on the GRADE approach

Based on traditional methodology, systematic reviews of well-designed RCTs are the current best evidence, while those including observational studies are regarded as low quality and not suitable for application within the context of EBM (20). With the development of clinical epidemiology and EBM, although RCTs have been the “Golden Standard” for medical research, the external validity of RCTs is very poor. The results always can not be applied to the patients in real world. In additional, in some special situation, RCTs can not be carried out (21,22). Therefore, it is incomplete for creating the evidence only from RCTs. The GRADE approach provides a new definition of quality irrespective of the design of studies, whether RCTs or observational studies. RCTs could be downgrade due to five factors, and observational studies could be upgrade due to three factors, that indicates RCTs could be low quality and observational studies could be high quality.

Moreover, in the GRADE approach, quality of evidence is only one factor of effective decision-making. The balance between desirable and undesirable outcomes, the application of patients’ values and preference and the resource use should be considered along with the quality of evidence in decision-making. Therefore, the GRADE approach is the current best methodology for establishing the evidence system.

The specific characteristic of the evidence system of TCM

As we discussed above, there are several disadvantages of RCTs in clinical research. As far as TCM is concerned, RCTs can not demonstrate the effect and efficacy of TCM completely. Meanwhile, observational studies are the main design for the clinical research of TCM (16). Comparing with RCTs, observational studies have better external validity and can be applied to the patients in real world more easily. Consequently, establishing the evidence system based on observational studies may accord with the current characteristics of TCM. In additional, Compare with western medicine, TCM usually has lower costs and fewer side effects. All of these completely meet the potential requirement of GRADE approach and EBM, thus, it is the right way for the establishment of TCM evidence system based on GRADE approach.

The initial attempts at establishing the TCM evidence system

The establishment of evidence system will provide methodological support for the guideline development of TCM. China Association of Acupuncture-Moxibustion has explored the method for establishing the evidence system of acupuncture therapy, and started the development of guidelines in 2011. In 2015, based on GRADE approach, China Association of Acupuncture-Moxibustion has published several guidelines of acupuncture therapy, that is the first attempts at establishing the evidence system of TCM (23).

The records from classics and experiences of famous TCM doctors are the important supplement of the TCM evidence system

Evidence is the information which is systematically assessed (8). Based on GRADE approach, records from classics and experiences of famous TCM doctors can not be the standard evidence of EBM. But these special information is the clinical characteristics and the significant components of TCM. With the development of EBM, to define evidence in spite of the clinical characteristics of TCM would cause misunderstanding and not conform with the potential concept of evidence. However, records from classics and experiences of famous TCM doctors could be a special kind of evidence, we can call them “pre-evidence”. If there are clinical studies focus on these pre-evidence, they can be translated into standard evidence, if not, they will be the important supplement of the evidence system of TCM, which will be the new ideal of translating the evidence for decision-makers.

Discussion

Finding, developing and applying evidence are the basic process of EBP, and clinical evidence is a comprehensive system. Along with the evidence of the treatment, decision-makers should also pay attention to the evidence of costs, patients’ values and preference. Currently, the quality of the clinical evidence of TCM is poor (24), the advantage of TCM could not be shown only rely on the evidence from the clinical research. GRADE approach could provide a better way to establish the comprehensive evidence system with the clinical characteristics of TCM. Based on this idea, we can integrate TCM with EBM more reasonably, and promote development of TCM.

In additional, how to translate the idea into practice is the main question should be focused on. At present in China, the quantity of the original study in patients’ values and preference is small, and the same as in TCM. Moreover, there is not a standard methodology for the study of patients’ values and preference. Although we propose the basic concept and frame of TCM evidence system, there are still lots of problems should be solved. Currently, there have been some research reports of patients’ values and preference in the world, we can take the experience of other countries for reference, integrate with the clinical characteristics of TCM and explore the methodology for the establishment of the evidence system. Based on this, we can finally translate the idea into real practice and promote the development of TCM and EBM.

Acknowledgements

The authors would like to thank Prof. Liang Du (West China Hospital) and Prof. Xian-Tao Zeng (Wuhan University) for their suggestions in manuscript writing.

Funding: The study is funded by Science and Technology Projects of State Administration of Traditional Chinese Medicine of Jiangsu Province (No. YB2015006); Projects of Gansu Province key laboratory of Evidence-Based and Transformation Medicine (No. EBMKT201502).

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Guyatt GH. Evidence-based medicine. ACP J Club 1991;114:A16. [PubMed]
2. Allen C, Clarke M. International Activity in Cochrane Review Groups With Particular Reference to China. Chin J Evid-Based Med 2006;6:541-9.
3. Sackett DL, Straus SE, Richardson WS, et al. Evidence Based Medicine: How to Practice and Teach EBM. New York: Churchill Livingstone, 2000.
4. Buetow S, Kenealy T. Evidence-based medicine: the need for a new definition. J Eval Clin Pract 2000;6:85-92. [Crossref] [PubMed]
5. Straus S, Haynes B, Glasziou P, et al. Misunderstandings, misperceptions, and mistakes. ACP J Club 2007;146:A8-9. [PubMed]
6. Stein ND. Misunderstandings, misperceptions, and mistakes. ACP J Club 2007;146:A11. [PubMed]
7. Wang YY, Liu BY, Xie YM. Establishing Appraisal Mechanism of Clinie Studies in Traditional Chinese Medicine with Proofing Medicine Methodology. Chinese Journal of Basic Medicine in Traditional Chinese Medicine 2003;9:17-23.
8. Li YP. The Strategy of Chinese Evidence-Based Medicine Center for Promoting the Modernization of Traditional Chinese Medicine. Chin J Evid-Based Med 2007;7:159-61.
9. Chen YL, Wang MS, Li X, et al. Definition of Evidence and Evidence-based Paradigm in Health Research. Chin J Evid-Based Med 2008;8:1034-8.
10. Chen KJ, Jiang YR. Current status and problems in developing clinical guidelines for Chinese medicine and integrative medicine. Zhong Xi Yi Jie He Xue Bao 2009;7:301-5. [Crossref] [PubMed]
11. Xie YM, Yu WY, Zhang Z. The method of creating the evidence for clinical guidelines of Traditional Chinese Medicine. Chinese Journal of Basic Medicine in Traditional Chinese Medicine 2013;19:27-30.
12. Guyatt GH. Five Directions for Evidence-based Medicine. Chin J Evid-Based Med 2006;6:159-61.
13. Atkins D, Best D, Briss PA, et al. Grading quality of evidence and strength of recommendations. BMJ 2004;328:1490. [Crossref] [PubMed]
14. Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence--study limitations (risk of bias). J Clin Epidemiol 2011;64:407-15. [Crossref] [PubMed]
15. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 7. Rating the quality of evidence--inconsistency. J Clin Epidemiol 2011;64:1294-302. [Crossref] [PubMed]
16. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 8. Rating the quality of evidence--indirectness. J Clin Epidemiol 2011;64:1303-10. [Crossref] [PubMed]
17. Guyatt GH, Oxman AD, Montori V, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. J Clin Epidemiol 2011;64:1277-82. [Crossref] [PubMed]
18. Guyatt GH, Oxman AD, Sultan S, et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol 2011;64:1311-6. [Crossref] [PubMed]
19. Guyatt GH, Oxman AD, Kunz R, et al. Going from evidence to recommendations. BMJ 2008;336:1049-51. [Crossref] [PubMed]
20. Chen YL, Li YP, Du L, et al. Evolution of Levels of Evidence and Strength of Recommendations in Medical Research. Chin J Evid-Based Med 2008;8:127-33.
21. Smith GC, Pell JP. Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of [randomized] controlled trials. J Int Assoc Physicians AIDS Care (Chic) 2004;3:108-9. [Crossref] [PubMed]
22. Glasziou P, Chalmers I, Rawlins M, et al. When are randomised trials unnecessary? Picking signal from noise. BMJ 2007;334:349-51. [Crossref] [PubMed]
23. CAAM. Evidence based clinical guidelines for acupuncture. Beijing: China Press of Traditional Chinese Medicine, 2015.
24. Min J, Mao B, Liu AR. Quality Assessment of Randomized Controlled Trials Related to Traditional Chinese Medicine Published in the Chinese Journal of Evidence-Based Medicine. Chin J Evid-Based Med 2010;10:362-6.