Is it time for OCEBM Levels of Evidence v.3?

“Nothing is more powerful than an idea whose time has come.” ~ Victor Hugo

The science of evidence-based medicine (EBM) and its practice have evolved over the years, with research methods developed to address various clinical questions. The COVID-19 pandemic has further highlighted the importance of using evidence to better inform clinical practice and policy-making decisions. At the same time, EBM must learn from the unprecedented changes during the pandemic. For example, the large number of published papers about predictive and simulation models has led to significant uncertainties, under, or, over-estimation of the predicted outcomes, and misinterpretation of findings. This is potentially attributed to the lack of adequate knowledge about the quality of evidence generated from the models, and the accompanying uncertainties, where “models are only as good as the data they are based on”.¹ The pandemic has also highlighted the importance of health economic evaluation as an integral research question: “does the intervention offer good value for the money spent?” and the indispensable necessity to evaluate the quality of the generated evidence. 

Therefore, there is an urgent need for an updated version of the OCEBM 2011 levels of Evidence, to achieve the following objectives:

1. Revise and update the existing Levels of Evidence after a decade to accommodate the rapidly growing literature on emerging study designs and priority areas.
2. Include new research question(s) and assign hierarchy/levels of evidence according to the study design.
3. Set a guide for clinicians, researchers, and policy-makers about the quality of evidence generated from the different research designs and highlight any existing uncertainties.

Existing gaps in the current OCEBM 2011 Levels of Evidence v.2

1. Absence of a research question addressing “Health Economic Evaluation” with the appropriate hierarchy/levels of evidence for the different study designs.
2. Not including modelling/simulation-based studies within the hierarchy/levels of evidence under the research questions of diagnosis, prognosis, and screening.
3. Hybrid research methods (mixed-methods research) are not accommodated in the levels of evidence.²

As a result of this, I propose the following modifications to address the current gaps in the existing OCEBM levels of evidence:

1. To include modelling/simulation-based studies within the hierarchy/levels of evidence in the research questions of diagnosis, prognosis, and screening.
2. To add a question about ‘health economic evaluation’ asking, “what are the health economic benefits of this intervention?” and to include the different study designs within the hierarchy/levels of evidence.
3. Highlight the level(s) of evidence generated from mixed methods research.

Suggested mechanism to address these gaps

1. Establish consensus task force groups in the three proposed areas; (1) modelling and simulation; (2) health economic evaluation; (3) mixed methods research, to review their inclusion in the levels of evidence framework. 
2. Engage EBM experts to evaluate and judge the quality of the evidence in each of the suggested areas, and link the evidence generated from different research study designs to the appropriate OCEBM levels of evidence.
3. Reach consensus among the various stakeholders about the suggested changes, and disseminate the newly updated version of the levels of evidence.

These gaps have long existed in the current 2011 Levels of Evidence and were only emphasized during the pandemic, reminding us of the inevitably dynamic nature of EBM that should adapt to and accommodate the changing world.

The time has come to address these gaps and release a comprehensively inclusive version of the Levels of Evidence, to be updated and developed by the OCEBM.

“If not now, then when?

If not us, then who?” ~ John Lewis

References:

1. Jefferson T, and Carl Heneghan C. Modelling the models. Accessed on July 3, 2022 at: https://www.cebm.net/covid-19/modelling-the-models/
2. Glasziou P, Vandenbroucke J. and Chalmers I. Assessing the quality of research. BMJ. 2004 Jan 3; 328(7430): 39–41.