Quality of systematic reviews of economic evaluations in health care

Context: Reviews performed almost a decade ago showed considerable gaps in the quality of reporting and methods applied to economic evaluations of health care interventions. Measures taken by the research community to address the issue included the promulgation of guidelines and the publicizing of good practice in economic evaluation. Methods: To assess the quality of methods of systematic reviews, economic evaluations in health care, and reporting methods, we conducted full-text searches of private and public databases for the period 1990 through March 2001 and corresponded with researchers active in the field. A total of 102 reports were identified, but only 39 were included. Quality of systematic reviews was assessed by a 6-item checklist. Results: Quality of review methods was reasonable, but more attention needs to be paid to search methods and standardization of evaluation instruments. The reviews found consistent evidence of serious methodological flaws in a significant number of economic evaluations. Lack of clear descriptions of methods, lack of explanation and justification for the framework and approach used, and low-quality estimates of effectiveness for the interventions evaluated were the most frequent flaws. Modest improvements in quality of conducting and reporting economic evaluations appear to have taken place in the last decade. Conclusions: Proper allocation of resources on the basis of economic evaluations remains uncertain. Editorial teams and regulatory bodies should perform quality assurance based on a single widely accepted and validated standard instrument.