CEBM response: 'Lack of an association or an inverse association between low-density-lipoprotein cholesterol and mortality in the elderly: a systematic review' - a post publication peer review

Overview

The paper by Ravnskov and colleagues reports findings from a “systematic review” to answer the question “Is low density lipoprotein cholesterol (LDL-C) a strong risk factor for mortality in elderly people”. The rationale given for answering this question was that recent findings of an association between total cholesterol and cardiovascular mortality may have been confounded by the presence of high levels of high density lipoprotein cholesterol (HDL-C), which is associated with lower risk of cardiovascular disease (CVD).

The authors sought to test the hypothesis that if LDL-C was the essential causative agent of CVD, then it should be a strong risk factor for mortality in the elderly.

To explore this hypothesis, the authors stated that they carried out a search of PubMed to identify cohort studies performed in people aged 60 years or older and in the general population setting where LDL-C was investigated as a risk factor for all-cause and/or CV mortality. They report that they identified 19 cohort studies including 30 cohorts with a total of 68,094 participants for inclusion.

The authors report findings of a statistically significant inverse association between all-cause mortality and LDL-C in 14 of 28 included cohorts and of no association in 14 cohorts. A similarly finding was reported for a statistically significant higher CV mortality in the lowest LDL-C quartile in 2 out of 9 cohorts with no association in the remaining 7.

The authors conclude that their findings question the validity of the current cholesterol hypothesis and provide rationale for a re-evaluation of current guidelines recommending pharmacological reduction in LDL-C in the elderly as a component of cardiovascular disease prevention.

Given the important implications of these statements it is imperative that they are based on the totality of available evidence identified using rigorous and appropriate methods to find, appraise, and summarise the evidence-base. Here we provide a post publication critical appraisal of the methodology and evidence used to support the findings from the Ravnskov et al paper.

What question (PICO) did the systematic review address?

The question the paper looked to answer was not definitively stated in the title, abstract or introduction. Furthermore, we found no evidence of any predefined statistical analysis or study protocol to help clarify this.

The authors do state “we examined the literature assessing LDL-C as a risk factor for mortality in elderly people” and “then we should find that LDL-C is a strong risk factor for mortality in elderly people”. It is therefore unclear what the main question of the paper is attempting to answer. However, one can infer the following:

P – Elderly people aged 60 years or over in the general population

I/E – High baseline blood plasma Low-Density Lipoprotein Cholesterol

C – Low baseline blood plasma Low-Density Lipoprotein Cholesterol

O – All-cause and CV mortality

Is it unlikely that important, relevant studies were missed?

As no protocol, or prospective registration of this review on PROSPERO could be found,  information on the methods used by Ravnskov and colleagues were taken from those reported in the published paper.

The authors searched only one electronic database (PubMed) using a small number of text words and limit studies to the English language. There was no attempt to search for unpublished studies. A flow chart of the search and screening process is provided. 282 full text articles were screened, 263 of which were excluded with generic reasons for exclusion given but reasons for exclusion per study were not given. A table/references for all excluded studies was also not provided. The authors make  some acknowledgement to the limitations of using only one database and thus the potential weaknesses of their search strategy Therefore their search strategy and reporting thereof presents a high risk of bias for missing important and relevant studies.

Were the criteria used to select articles for inclusion appropriate?

The criteria for inclusion of cohort studies involving people aged 60 years or older selected randomly from the general population is reasonable given the question the authors looked to answer. However, additional inclusion of studies where the authors found no significant differences between participants and the source population’s demographic characteristics is ambiguous and represents a high risk of selection bias.

There is evidence that the  criteria for inclusion or exclusion were not uniformly applied across all studies. For example, the authors stated that they excluded “studies without multivariate correction for the association between LDL-C and all-cause and/or CV mortality”. In table 2, the authors present the factors corrected for by each of the included studies in multivariate analyses. It is noticeable that for Bathum et al. there were no details of the factors corrected for. Without knowledge of the factors corrected for it cannot be ascertained if indeed such a correction was performed. Therefore Bathum et al. may be considered ineligible according to the exclusion criteria stated by the authors themselves. At a minimum there should be acknowledgement of this, particularly given that Bathum et al. was the largest study included in the review (~two thirds of the 68,085 total included participants).

A second example is presented by the exclusion of the study by Ptsay et al. (reference 6) for the reason that “it included the same individuals as in a previous study” (Fried et al. [reference 7] is then cited). The Fried et al. study was published 6 years previously and showed a 0.66 relative risk for high LDL-C and mortality (95% CI or p-value were not presented). Ptsay et al. was a re-analysis of the original data used in Fried et al. and showed a much smaller (non-significant) relative risk of 0.97, but importantly presents 95% CI that show no association of LDL-C with mortality (0.92 to 1.04).

The authors also make no reference to the study by Postmus et al. published in February 2015 despite including data on LDL-C and mortality in three cohorts of people aged 60 years or over. The Postmus et al. study reported that a genetic disposition to high LDL-C was associated with increased mortality and that a genetic disposition to low LDL-C was associated with familial longevity. The findings from Postmus et al. also suggest that the results from previous studies demonstrating an inverse association with LDL-C and mortality (such as those included in the present review) were “probably biased”.

This bias is likely reflected by two critical issues that Ravnskov and colleagues appear to have failed to address – notably the presence of confounding for the association of LDL-C and mortality due to the effect of lipid-lowering treatment and/or high HDL-C.  The authors correctly write that “it is worth considering that some of the participants with high LDL-C may have started statin treatment during the observations period. Such treatment may have increased the lifespan for the group with high LDL-C.” However, the authors have failed to judiciously appraise lipid-lowering treatment drop-in and its potential implications for confounding. At the inception of a prospective cohort study, participants will be classified into two groups: high LDL-C or low LDL-C. Both groups will then be followed over time to record the rate of mortality to make comparison between the groups. The issue is that those who have high LDL-C are much more likely to be prescribed statins during the period of observation than the low LDL-C group. This would lead to an overall protective effect in the group with high LDL-C, making it appear that LDL-C is correlated with reduced all-cause mortality, when, in fact, it may be the effect of statin therapy.

Furthermore, the authors fail to adequately address the potential magnitude that this confounding may have and instead make the claim that, “any beneficial effects of statins on mortality would have been minimal because most statin trials have had little effect on CVD and all-cause mortality.” This claim is not backed and it is unclear why the authors have failed to reference randomised controlled trial evidence to demonstrate this.

In their introduction the authors acknowledge a known association of higher total cholesterol (TC) with higher risk of mortality in the elderly but state that “there may be a confounding influence in these studies, however, because TC includes high-density lipoprotein cholesterol (HDL-C), and multiple studies have shown that a high level of HDL-C is associated with a lower risk of CVD”. Given this important confounding factor it is imperative that any association between individual lipid components are controlled for in statistical analysis. In the present review, however, only three out of the 19 included studies appear to have controlled for HDL-C level in their analyses. It is highly possible that the observed inverse association for LDL-C and mortality is entirely mediated by a high HDL-C in the included cohorts.

Were the included studies sufficiently valid for the type of questions asked?

In the methods section, and at odds with the IMRAD process, the authors provide a brief narrative summary of their quality assessment of the included studies according to the Newcastle-Ottawa scale. Indication of quality is given by such statements as: “The design of the study [singular] satisfies almost all points of the reliability and validity according to the Newcastle Ottawa Scale as regards selection, comparability and exposure”. The results of the quality assessment are not presented for each included study to allow sufficient judgement of their methodological quality and no description as to what constitutes a “satisfactory” rating is provided by the review authors.

Were the results similar from study to study (was heterogeneity present)?

The authors did not adequately address heterogeneity between studies, with the only indication of such an analysis given as “all studies represented elderly people only; ascertainness of exposure (eg, measurement of LDL-C) was present in all studies, and outcome was unknown at the start.”

What were the results and how are they described?

The authors do not provide any details of their statistical analysis methods for their review. The authors provide information on data extracted for the association between initial LDL-C and risk of all-cause and/or CV mortality based on relative effect estimates as provided by the hazard ratio. We were interested to note that two of the authors (RV and DM) have previously criticised the use of relative risk estimates by statin advocates and researchers as statistical deception aimed at inflating claims about their effectiveness at preventing stroke, heart attack and heart disease-related deaths. This criticism appears to have been overlooked in the current paper which presents and discusses only relative risk estimates for all outcomes.

Results are described narratively and also presented Table 1. It is clear that a meta-analysis was not performed, however the authors do not state in their methods that a meta-analysis would be attempted. Data are presented for the association between LDL-C and all-cause mortality in tertiles or quartiles of LDL-C either as single effect estimates (hazard ratio) or narratively. Statistical significance is indicated by footnotes. Data for the association between LDL-C and CV mortality are presented narratively only.

In table 1 the authors present hazard ratios from six of the 19 included studies that suggest a large relative effect that increases with increasing LDL-C levels, indicating that higher levels of LDL-C are associated with an 11% to 59% relative risk reduction in all-cause mortality. For eight of 19 studies, no association between LDL-C and all-cause mortality has been indicated by the review authors. For two of the remaining five studies, the authors narratively describe an “mirror-J-formed association” or “inverse association”; for one study the absolute LDL-C values (mmol/L) for survivors versus non-survivors are given and for the two remaining studies the authors indicate no information was available.

Results for CV mortality are presented narratively only and are less informative; the majority of studies (11/19) did not provide information for the outcome; 7/19 are stated as showing no association with LDL-C and the remaining one study is stated as showing an “almost u-formed” association or “mirror-J-formed association” with LDL-C.

In table 1 it is stated that the effect estimates presented are hazard ratios. We have identified that the results from at least one study (Fried et al.) for the association between all-cause mortality and LDL-C are provided only as relative risk. For clarity, here is what is presented in Fried et al.: LDL cholesterol level higher than 3.96 mmol/L (153 mg/dL) had a significantly lower risk (RR, 0.66), compared with lower values of LDL cholesterol. The value extracted by the authors for this outcome appears correct (0.66). However, the review authors have stated this effect as “significant” but we found no indication of its statistical significance (neither P values or 95% confidence intervals) presented in Fried et al. Moreover, the authors of the review have presented the data from Fried et al. in the “IV” quartile column of their table. However, Fried et al. express their results in quintiles, and it is unclear exactly which quintile groups were being compared.

These errors raise doubts as to the accuracy of the data extraction performed by the review authors.

All the included cohort studies provide data that would be judged as low quality according to GRADE criteria. The majority of effect estimates for the association of LDL-C with mortality are >0.5, 95% CI’s are not provided and there is plausible confounding present. Therefore the quality of evidence cannot be upgraded and are more likely to be downgraded to very low quality (due to bias of individual studies) meaning any estimates of effect are very uncertain.

Reporting and methodological quality according to the AMSTAR tool

We additionally evaluated the article using the Assessment of Multiple Systematic Reviews (AMSTAR), a measurement tool to assess the quality of reporting and elements of the methodological quality. Of the 11 criteria, the article fulfilled only two (table below).

Conflicts of interest

The AMSTAR tool assesses whether conflicts of interest were stated. There are a number of conflicts of interest that accompany this review, some undeclared.

Several of the authors (RH, HO, RS, MK and UR) declare that they have written books with criticism of the cholesterol hypothesis. However, one of the co-authors (AM) is also featured (page 2) in a report sold on amazon that criticises current dietary guidelines on fat and cholesterol. The same co-author is also listed as an advisor to the UK National Obesity Forum, a charitable organisation which states on its website that it has received support from a number of organisations including commercial weight loss organisations and the pharmaceutical industry. Both of these potential conflicts of interest were undeclared.

Conclusions

There are a number of studies that seek to explore associations between LDL-C and mortality in samples of elderly people from the general population, despite a known positive association with cardiovascular disease. Given its significance, there is some justification for a rigorous and systematic review of the available literature related to cholesterol and mortality in the elderly.

Ravnskov and colleagues attempt to provide such a review. However there are serious methodological flaws with their study, not least the lack of a published protocol, searching of only one database, nonuniform application of inclusion/exclusion criteria, a lack of critical appraisal of the methods used in the included studies, no indication of the quality or uncertainty of the included data and issues with the accuracy of data extraction. A lack of controlling for confounding due to the effect of lipid-lowering treatment and HDL-C levels presents major bias and more likely underpins the majority of the observed inverse associations.

Given that the authors failed to account for this significant confounding as well as the methodological weaknesses of both the review and its included studies, the results of this review have limited validity and should be interpreted with utmost caution.  At this time it would not be responsible, or evidence-based, for policy decisions to be made based on the results of this study.

Authors of statement

Dr David Nunan  – Departmental Lecturer in Evidence Based Medicine, david.nunan@phc.ox.ac.uk

Dylan Collins – DPhil Candidate

Niklas Bobrovitz – DPhil Candidate

Dr Kamal R. Mahtani – GP and Deputy Director

Centre for Evidence Based Medicine, Nuffield Department of Primary Care Health Sciences, University of Oxford.

Disclaimer

Dr David Nunan and Dr Kamal R. Mahtani are both members of the Royal College of General Practitioners (RCGP) steering committee to support the new Physical Activity and Lifestyle clinical priority.

Dr Nunan and Dr Mahtani have received funding for research from the NHS National Institute for Health Research School for Primary Care Research (NIHR SPCR) and the RCGP for independent research projects related to physical activity and dietary interventions. Mr Collins receives funding from the World Health Organisation (WHO), Medecins Sans Frontieres (MSF), and The Rhodes Trust for research related to cardiovascular risk. Mr Bobrovitz has received funding for research from the NIHR SPCR for research projects related to variations in medication use and unplanned hospital admissions.  The views expressed are those of the authors and not necessarily those of the NHS, the NIHR, the Department of Health, the RCGP, WHO, MSF, The Rhodes Trust or any other institution named in this post.

They declare no other relevant conflicts of interest.