Why do we need systematic review and meta-analysis, and what are their potentials: The exponential proliferation of healthcare literature and the potential of research synthesis

In this blog, EBHC DPhil student, Ali Mulhem, discusses the importance of systematic review and meta-analysis to healthcare providers and clinicians in the context of EBM.

My first contact with Evidence-based medicine (EBM) was in 2006, my first year in med school. The scientific committee of the student union of my faculty at Aleppo University was organizing one-week seminars about EBM. Topics ranged from primary care, cardiology, neurology, infectious disease and surgery; however, one thing was in common: the methodology for answering clinical and healthcare questions was robust and fascinating for me. That was the first moment I heard about systematic review (SR) and meta-analysis (MA). 13 years later, I started my journey with EBM in Oxford, finishing the MSc program in EBHC and now the DPhil. In this blog, I will share why I think healthcare providers and clinicians need EBM, focusing on SR and MA.

Why do we need research synthesis?

Today, there is an unprecedented proliferation of scientific literature, especially the medical one. For example, if we search PubMed for the term "COVID-19" on 12 Feb 2023, we will get more than 322 000 hits, including more than 2260 RCTs. This is a huge amount of evidence for busy clinicians and healthcare professionals to keep tracking. In one study titled "how much effort is needed to keep up with the literature relevant to primary care?" Alper et al. estimated that physicians trained in epidemiology would take approximately 627 hours to keep up-to-date with the literature. In another study titled "Journal reading habits on internists", Saint et al. estimated that clinicians spend only 4,4 hours per week reading medical journal articles. Thus, the unprecedented proliferation of the healthcare literature is accompanied by a very short time to keep track.

Moreover, much of the scientific literature is controversial and replicated. Ioannidis, in a study published in JAMA 2005, "contradicted and initially stronger effects in highly cited clinical research," estimated that 32% of the research showed contradicted or stronger effect results, and 44% was replicated, with only 24% remained largely unchallenged. Not only controversial and replicated but also inconclusive is much of the healthcare literature. Djulbegovic and others estimated in an empirical study published in the Journal of clinical epidemiology that 30% of RCTs are inconclusive, resulting in the FRIN cliché (Further Research Is Needed). In my master's dissertation, I reviewed and meta-analyzed 64 studies that studied the effect of invasive surgery in patients with acute neurological conditions /stroke, trauma, and bleeding); 40 studies out of the 64 were inconclusive in their results.

Thus, because of: 1. the exponential proliferation of the literature, 2. The time shortage in healthcare, 3. The conflicting and inconclusive results of medical research, with 4. The consequent wastage of recourse through replication, we, healthcare professionals,  need to learn more about research synthesis, we need to learn more about systematic reviews and meta-analysis.

What are research synthesis and their potential?

The first example of a published SR might be the treatise by James Lind in 1753 on the treatment of scurvy (vitamin C deficiency). He described his paper as "a critical and chronological view of what has been published on the subject" and wrote: "It became requisite to exhibit a full and impartial view ….., it was necessary to remove a great deal of rubbish". In 1904, Karl Pearson published what we might consider the first meta-analysis about inoculation against enteric fever. Since then, the research synthesis in SR and MA has been developed rapidly. The foundation of the Cochrane Collaboration in 1993 was a turning point. SR and MA are considered the best methods to answer research questions and find evidence. Cochrane defines SR as an attempt to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a specific research question. The MA is the statistical attempt of this synthesis quantitively. Many examples in the medical literature show what SR and MA can bring to scientific knowledge and their potential. The MA by Crowley, Chalmers and Keirse in 1990 showed, after a decade of controversy, that corticosteroids given to women who are about to give birth prematurely can save the life of the newborn child. This conclusion is now standard practice in obstetrics and saves the lives of many children. On the other side, after two SR and MAs in 1998 and 2005 by Alderson and Roberts, the false practice of giving corticosteroids to patients with head trauma has been stopped after finding a harmful effect.

MA and SR can not only bring new evidence or change the false healthcare practice, but it also has the potential to save resources, time and lives. In a cumulative meta-analysis about the effect of B-blockers for patients with myocardial infarction, Freemantle and others showed that this beneficial effect could have been seen through a MA since 1981; however, researchers kept conducting RCTs till 1997 about this topic, just wasting resources and also lives of patients by preventing them from a beneficial effect. Another example shows this fact can be seen in the cumulative MA by Antmann and others published in JAMA 1992, where the beneficial effect of streptokinase for patients with myocardial infarction could have been seen since 1979, but the licensing in many countries was not issued till the late eighties (after more than 7 years).

Besides summarizing the evidence and saving recourses and lives, SR and MA can find new knowledge and generate new hypotheses. Through meta-regression,  Berkey and others in 1995 found an explanation for why the effect of tuberculosis vaccination varied in its efficacy between countries. After plotting the effect size of the vaccination against the distance from the equator, they found a significant regression coefficient with decreasing the effect of vaccination by increasing the degrees of latitude.

Learning systematic review and meta-analysis can help us, healthcare professionals, in 1. confirming available evidence, 2. refuting false practice, 3. saving resources, and lastly, 4. finding new knowledge and generating hypothesis.

My fascination with EBM, especially with systematic review and meta-analysis, is still growing. I can not only recommend every clinician, healthcare provider and professional to learn more about EBM and its research synthesis methods. The Oxford MSc and DPhil programs in EBHC greatly help in this regard.   

Further Reading

• Alper, Brian S et al. "How much effort is needed to keep up with the literature relevant for primary care?." Journal of the Medical Library Association : JMLA vol. 92,4 (2004): 429-37.
• Saint, S et al. "Journal reading habits of internists." Journal of general internal medicine vol. 15,12 (2000): 881-4. doi:10.1046/j.1525-1497.2000.00202.x.
• Ioannidis, John P A. "Contradicted and initially stronger effects in highly cited clinical research." JAMA vol. 294,2 (2005): 218-28. doi:10.1001/jama.294.2.218
• Djulbegovic, Benjamin et al. "Optimism bias leads to inconclusive results-an empirical study." Journal of clinical epidemiology vol. 64,6 (2011): 583-93. doi:10.1016/j.jclinepi.2010.09.007
• Mulhem, Ali, et al. "Is Decompressive Craniectomy Better than Standard Care Alone for Increased Intracranial Pressure: A Systematic Review and Meta-Analysis." Available at SSRN 3927059.
• Roberts, Devender et al. "Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth." The Cochrane database of systematic reviews vol. 3,3 CD004454. 21 Mar. 2017, doi:10.1002/14651858.CD004454.pub3.
• James, Lind. Treatise On the Scurvy: In Three Parts, Containing an Inquiry Into the Nature, Causes, and Cure, of That Disease; Together With a Critical and Chronological View of What Has Been Published On the Subject. Forgotten Books., 1901.
• Clarke, Mike, and Iain Chalmers. "Reflections on the history of systematic reviews." BMJ evidence-based medicine vol. 23,4 (2018): 121-122. doi:10.1136/bmjebm-2018-110968.
• "Report on Certain Enteric Fever Inoculation Statistics." British medical journal vol. 2,2288 (1904): 1243-6.
• OCEBM Levels of Evidence Working Group*. "The Oxford 2011 Levels of Evidence ".
• Chandler J, Cumpston M, Thomas J, Higgins JPT, Deeks JJ, Clarke MJ. Chapter I: Introduction. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.
• Crowley, P et al. "The effects of corticosteroid administration before preterm delivery: an overview of the evidence from controlled trials." British Journal of obstetrics and gynaecology vol. 97,1 (1990): 11-25. doi:10.1111/j.1471-0528.1990.tb01711.x
• Freemantle, N et al. "beta Blockade after myocardial infarction: systematic review and meta regression analysis." BMJ (Clinical research ed.) vol. 318,7200 (1999): 1730-7. doi:10.1136/bmj.318.7200.1730
• Antman, E M et al. "A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction." JAMA vol. 268,2 (1992): 240-8.
• Roberts I, Schierhout G, Alderson P     Absence of evidence for the effectiveness of five interventions routinely used in the intensive care management of severe head injury: a systematic review     Journal of Neurology, Neurosurgery & Psychiatry 1998;65:729-733.
• Alderson, P, and I Roberts. "Corticosteroids for acute traumatic brain injury." The Cochrane database of systematic reviews vol. 2005,1 CD000196. 25 Jan. 2005, doi:10.1002/14651858.CD000196.pub2