Medical student blog- Systematic Review Signals
2 April 2019
Students Systematic Reviews
“The combination of these two outcomes has made this a really worthwhile experience, and one we would highly recommend to our colleagues considering where to spend their time on the special studies module”
We are two 6th year medical students at Oxford. As part of our course, we have the opportunity to undertake short, self-selected, special study modules (SSM). We both chose to undertake our three-week placement in the Centre for Evidence-Based Medicine (CEBM). The importance of systematic reviews in underpinning clinical, and indeed research, decisions was a lasting impression from our short EBM module in our 4th year delivered by CEBM tutors. This was an ideal training opportunity to get more comfortable with some of the tools and methods that we will be using throughout our careers.
WHAT DID WE DO?
Neither of us felt comfortable explaining what processes were involved at each step of a systematic review or what constituted “good” methodology. The opportunity to learn some of these skills came in the form of a rapid review of the literature relating to risk communication. More specifically, how useful ‘age’ tools and metrics were at communicating a person’s risk of poor health outcomes and whether this had any effect on their health-related behaviours and/or outcomes. The aim was to determine if a systematic review of this question was needed:
Population – adults interacting with health care services
Intervention – communication of chronic disease risk through the use of either a body/organ age metric
Control – risk communication that did not involve the use of an age metric (e.g.absolute risk, usual care)
Outcome – change in health behaviours and/or health outcomes.
We used PubMed and TRIP databases to search for relevant articles using relevant search terms, including: “biological age” “lung age”, “heart age”, “fitness age”, “body age”, “effective age”, “brain age”, “vascular age”. Snowballing of relevant articles also revealed some additional relevant terms including.
We were only interested in level 1 and 2 evidence – systematic reviews and randomised controlled trials (RCTs) as these provide the best type of evidence to answer this intervention question. Once eligible studies were identified, full-texts were checked for inclusion or exclusion. We analysed the quality of the methods in systematic reviews using the AMSTAR 2 tool and in RCTs using the Cochrane risk of bias 2.0 tool.
WHAT DID WE FIND?
We identified 1 overview of systematic reviews (level 1 evidence), 2 systematic reviews (level 1 evidence) and 13 RCTs (level 2 evidence) that met our inclusion criteria.
Evidence from systematic reviews
The overview included one of the 2 systematic reviews included in our search. These 2 reviews more broadly assessed the effect of different risk communication strategies for smoking cessation and cardiovascular risk, respectively, without focusing specifically on ‘age’ based tools.
The smoking cessation study was a Cochrane review scored highly on the AMSTAR 2 tool (11 out of 16 = high-quality). The review included one study assessing a ‘lung-age’ tool (see ‘Evidence from RCTs’ below) which found that people randomised to being told the age of their lungs relative to someone older or younger had a higher confirmed smoking cessation rate and lower cigarette consumption rate.
The cardiovascular risk review scored poorly on AMSTAR 2 (5/16, quality). It included 15 RCTs comparing different risk communication strategies for cardiovascular risk. Due to the heterogeneity of the studies examined a meta-analysis was not possible.
Both reviews concluded that the current state of the literature was not conclusive to make any recommendations. These reviews were completed over 6 years ago and as such our search encountered many new trials which were not available at the time the reviews took place.
Evidence from RCTs
The 13 RCTs examined a number of different ‘age’ metric. The risk of bias associated with these studies is shown in figure 1; overall few studies had a low risk of bias (3/13), with key issues being poor or poorly described randomisation (9/13) and allocation concealment (11/13) methods.
Data for 18 outcomes are provided in Table 1; 7/18 outcomes demonstrated clinical and statistical benefit in favour of the intervention; 3/18 suggested clinical benefit for intervention but did not meet statistical significance; 8/18 showed no clinical difference and were not statistically significant, 2 of which favoured control (see Table 1).
Different methods were used to calculate the age metrics, formulas were not standardised and the measurement of patient behaviour varied significantly between the studies.
Our findings indicate that currently available systematic review evidence needs updating to capture the scope of different ‘age’ communication metrics and tools, their validation, intended use and assessments of their impact. The evidence from RCTs on the effect of using age metrics to communication chronic risk on subsequent patient behaviour and health outcomes is generally poor and under evaluated. There is a need for large, high-quality trials to decrease uncertainty in the available evidence.
WHAT DID WE LEARN?
We are pleased that we managed to complete a rapid review in the short period of time we had. Alongside teaching sessions we had with Dr David Nunan, we have learnt a huge amount about the processes of appraising evidence and sound research methodology. The combination of these two outcomes has made this a really worthwhile experience, and one we would highly recommend to our colleagues considering where to spend their time on SSM. An abstract of this work has been submitted for presentation at EBMLive and we also plan to submit for publication in a peer-reviewed journal.
The next stage for both of us is our elective. We’re heading off to a combination of Australia, New Zealand and Vanuatu. As of next year, we will be working as foundation year 1 doctors in Oxford (Bavidra) and Wales (Adam) which we are both looking forward to (with a little apprehension of course!).
Acknowledgements: We would like to thank Jon Brassey for performing searching and initial screening of studies.
Table 1: Summary of Findings from included RCTs
|‘Age’ metric||Study||Outcome||Summary of effect||Favours||Clinically significant||Statistically significant|
|Lung age||Parkes 2008||Smoking cessation at 12m||Intervention: 14%
|Kaminsky 2011||One or more quit attempts in 1m||Intervention: 32%
|Drummond 2014||Smoking cessation at 6m||Intervention: 0%
|Foulds 2015||Smoking quit rate at 28 days||Intervention: 51%
|Takagi 2017||Smoking quit rate at 12 weeks||Intervention: 60% Control: 42%||Intervention||Yes||Yes|
|Continued abstinence 12m||Intervention: 79% Control: 69%||Intervention||Yes||No|
|Heart age||Lowensteyn 1998||Likelihood of returning for risk assessment||Intervention: 1.23 (0.96 to 1.60) Control: 0.77 (0.58 to 1.03)||Intervention||Yes||Yes|
|Bonner 2005||Smoking quit rate at 2 weeks||Intervention: 24% Control: 15%||Intervention||Yes||No|
|Lopez-Gonzalez 2015||Systolic blood pressure||Intervention -4.37 (-4.75 to -3.99) Control: 1.02 (0.8 to 1.25)||Intervention||Yes||Yes|
|Total cholesterol||Intervention: -6.54 (-7.23 to -5.85) Control: 5.36 (4.76 to 5.95)||Intervention||Yes||Yes|
|Risk score at 12 months||Intervention: -0.37 (-0.44 to -0.31) Control: 0.24 (0.19 to 0.29)||Intervention||Yes||Yes|
|Health age||Godin 1987||Smoking abstinence at 6m||Intervention: 48% Control: 44%||Intervention||No||No|
|Paek 2014||Positive exercise behaviour at 3m||Intervention: 6% Control: 6%||Intervention||No||No|
|Grover 2007||Achieving lipid target at 12m: With CVD||Intervention: 50% Control: 48%||Intervention||No||No|
|Without CVD||Intervention: 57% Control: 54%||Intervention||Yes||Yes|
|Body age||Liukkonen 2017||Physical activity at 12m (MET/min/wk)||Intervention: -17 (-1021 to 987) Control: -89 (-1109 to 926)||Intervention||No||No|
|Charlson 2008||All-cause mortality at 24m||Intervention: 4.1% Control: 4.4%||Intervention||No||No|
|Myocardial infarction at 24m||Intervention: 4.2% Control: 4.4%||Intervention||No||No|