Sixth year medical student, Ben, shares details of upcoming project, focusing on critically appraising tools used to influence clinical decision-making
30 May 2022
Students
Sixth year primary care medical student, Ben, will be joining the Centre for Evidence-Based Medicine for a three-week period as part of his special study module (SSM), to pursue his interests in meta-analysis and interventions.
My name is Ben, and I'm one of the final year medical students at Oxford. I have a broad range of interests within medicine and no idea (yet) where I would like to take my career. Given the opportunity to pursue almost anything I would like for two 3-week special study modules (SSM) after my final exams, there are few areas as universally necessary and applicable as evidence-based medicine (EBM). We're fortunate to have the Centre for EBM here in Oxford, and our course reflects the university's historical association with evidence-based practice by allowing us to be exposed to this essential element of modern medicine throughout our degree. Despite being known among friends for harping on about evidence, I had never been involved with any rigorous research within my time at medical school and was eager to get stuck in with some practical EBM to complement my theoretical interest.
WHAT DID I DO/WHAT WILL I BE DOING?
David Nunan, who is involved with our EBM teaching and is CEBM SSM lead, first met with me prior to figure out what shape our 3-week module might take. Immediately we identified that we share a keen interest in sport and exercise as well as the importance of communication of evidence. It just so happens that Dr Nunan is mentoring a master's student who was about to embark on an overview of systematic reviews (SRs) with network meta-analyses (NMAs) examining the evidence for exercise as therapy for non-specific chronic lower back pain (NSCLBP). I decided I would get the most out of being involved in a project close to my interests, so agreed to assist Chris Banks-Pillar with his master's project.
Network meta-analysis is a relatively modern method which uses data from direct and indirect comparisons of interventions to provide a ranking of the comparative efficacy of interventions (see figure). They are increasingly used to inform healthcare policy and clinical decisions. All tools used to influence clinical decision-making, particularly new ones, must be subjected to rigorous scrutiny to ensure that they are being properly applied and that they are producing robust, repeatable, and applicable results. This project aims to do exactly that.
Since the research question had already been established, our first step was to write the study protocol and register it on the Open Science Framework (OSF) website. This was new ground for me - and, as I've found with much of EBM, it's one of those things that seems super obvious once you've seen one and done one, but slightly confusing and perplexing when it's your first time. I realised having a detailed plan before getting started makes life much, much easier.
Our project can be divided into three steps:
Step one: We'll be checking the reporting and methodological quality of the SRs with NMAs making use of tools such as PRISMA-NMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses - Network Meta-Analyses), AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews), and ROBIS (Risk Of Bias In Systematic reviews).
Step two: Here we focus on the primary literature. Our strategy is to see if there is an intervention that does well across all SRs with NMAs, and to look into the underpinning studies to see whether these conclusions are valid. We hypothesise that some of the overlooked aspects might be publication bias, small-study bias, and confounding co-interventions. We will re-extract the data from the studies, re-assess the risk of bias analyses, and perform our own synthesis including only low risk-of-bias studies and exercise-only interventions.
Step three: A continuation of step 2; here we dive deeper into the primary literature to check the quality and applicability of the studies using tools such as TIDieR (Template for Intervention Description and Replication).
REFLECTIONS AND LEARNING POINTS
It's definitely felt like I've had a personalised SSM, which was a real pleasure to take part in. I have to thank David Nunan for taking the time to have a chat with me and identify my interest in sports and exercise medicine. This project has turned out to be perfect for me, as I will be required to take part in so many different aspects of the evidence process from reviewing and critiquing to extracting and synthesising data, all with the guidance and support from an expert in the field. Even though my 3-week SSM has come to a close, I have continued to be a part of this project and hope to follow it through to completion.
So far, I've learned that appraising literature properly is much more complex than I thought it would be. The step-up in critical thinking from "reading" (i.e., scanning for quotable phrases) primary literature for my master's thesis in Chemistry to studying medicine was huge - but I didn't expect that literature appraisal at the CEBM would be another league entirely. It gives you an appreciation of the importance of attention to detail when you're trying to establish what's best for patients. It requires a passion for the subject, as spending hours poring over the same handful of papers doesn't make the time fly as well as reading Bad Science on a sun lounger in Greece (although both activities ignite the same fire for spotting inconsistencies and thinking critically). But it's a crucial process that needs to be done properly if we are to know which research is a product of quality rather than quantity.
On that last point - I would encourage any of my colleagues to consider joining the CEBM for an SSM or project. You know as well as I do that there will be a requirement to 'publish or perish' in our pursuit of a specialist career, whether we like it or not. If it's not in your nature to produce work for the sake of it, feed two birds with one scone and help improve the standard of research in a field you're passionate about. Although the level of evidence appraisal to be undertaken here is far more detailed than I'd be able to do for any single clinical decision, it has given me a far greater appreciation for what needs to be addressed in research, and hopefully one day I can produce literature that would stand up to such scrutiny and therefore make a truly meaningful impact.
What to read next
Sixth year medical student shares evidence-based project as part of the special studies module (SSM)
16 May 2022
Sixth year primary care medical student, Charlotte, joined the Centre for Evidence-Based Medicine for a three-week period as part of her special study module, to improve her knowledge of evidence-based medicine. In this blog, Charlotte discusses her project, focusing on evaluating the reporting of medication adherence to pharmacological interventions in coronavirus (SARS-CoV-2) related randomized control trials (RCTs). Her project has now been published as a peer-reviewed article.