Five things to consider before you do a systematic review

1. FORMULATE A GOOD RESEARCH QUESTION

The key to conducting a good systematic review is asking a useful research question.

What do you know? What don’t you know? You need to be specific – or you won’t find the information you’re seeking.

The research question needs to be relevant (e.g. to clinical practice or to policy) – otherwise, it could be a waste of time.

2. DEFINE THE STRATEGY YOU INTEND TO USE IN ORDER TO ANSWER THE RESEARCH QUESTION

Use PICO to pin down your research question:

Population

You can include studies in all groups, or restrict to certain groups of people. That choice affects the external validity of your findings, so you want to exclude only when there is a good reason. Do you want to know about the effects of a flu jab in young men? Or elderly people with respiratory disease? Or everyone? Is there a good reason to limit studies to a particular period of time?

Intervention

You’ll need to be clear exactly what intervention (or exposure) you want to know about. Do you want to know about using paracetamol to treat headaches? How about if caffeine is taken at the same time? Will you include those studies or study groups?

Comparison

Do you want to compare those using paracetamol with those using ibuprofen, or those using a dummy (placebo)? Think about how the comparison provides information to answer the research question you have.

Outcomes

A brief scope of the literature will indicate the types of outcomes typically reported. Do these allow you to answer your research question? What about outcomes you hadn’t thought of?

3. ENSURE TO ASSESS THE QUALITY OF INCLUDED STUDIES USING STANDARDISED CRITERIA

You need to assess the quality of data entering your review so as to establish how well the available information answers the research question you have posed.

• You could have a great research question – but if the evidence you find to answer it isn’t great, then we know we need to generate better information.
• A review of the literature without assessing the literature’s quality may be misleading or useless.
• Using a standard tool such as the Cochrane risk of bias tool helps make this process accountable and comparable with other reviews.

Assessing quality isn’t a task in isolation. It affects your interpretation of the information within the studies and can have a big impact on the next steps once your review is published.

4. DECIDE BEFOREHAND HOW YOU INTEND TO ANALYSE YOUR RESULTS

Determine ahead as to whether you would be using fixed or random-effect model for your meta-analysis. Think about how you will account for variations in the PICS (Participants, Interventions, Comparators, or Studies). What would you do if the studies do not provide enough data in order for you to combine their study results. 

5. DECIDE HOW YOU WILL TEST WHETHER YOUR RESULTS ARE VALID

While it is useful to statistically combine clinical trial data, the results of such combinations could be misleading. Therefore, decide ahead how you will test the validity of the results which you will present. This may include usage of funnel plots, sensitivity analysis or trim and fill analyses.

Once you’ve thought through the research question and quality assessment, write a protocol stating your research question and PICO, and how you plan to assess the quality of the evidence.

Use the PRISMA checklist to guide you. http://www.prisma-statement.org/

• Be clear and think ahead to avoid problems later on.

• Register your protocol on PROSPERO: http://www.crd.york.ac.uk/prospero/

Now you can go ahead and enjoy your conducting your review! All the best as you do!

Applications are now open for the new MSc in Systematic Reviews –  a course for health professionals who want to gain an understanding of the importance of systematic reviews in health care as well as the practical skills to conduct them. For further information, please visit the University of Oxford Department of Continuing Education website.