Original ArticlesMethods for systematic reviews of economic evaluations for the guide to community preventive services1
Introduction
This paper describes the procedures and instruments used to collect, abstract, adjust, and summarize results from economic studies reporting on cost, cost-effectiveness, cost-benefit or cost-utility of selected interventions for the Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations (the Guide). Methods were developed so that studies using disparate analytical methods can be consistently compared. Although these methods were developed specifically for the Guide, they are sufficiently complete and general enough to be readily adapted to a range of systematic reviews of economic evaluations in health care and public health.
Evidence-based reviews of effectiveness (e.g., the Guide to Clinical Preventive Services1) usually have not reported data regarding costs and financial benefits of preventive interventions. Users of evidence-based effectiveness reviews often do not have enough information to identify among effective interventions, those providing the greatest amount of health or financial benefit per dollar of resource used. This might be because of the absence of economic evaluation studies or problems of comparability among available studies. Although little can be done on a short-term basis to increase the quantity of available studies, systematic reviews of economic studies can address problems of comparability. In addition, they can improve the usefulness of existing studies just as systematic reviews bring together and interpret a body of evidence of the effectiveness of interventions.
The lack of accepted, or even openly debated, methods and instruments for systematic reviews of economic studies is significant in light of both the need for such methods and the precedent set for such methods in the effectiveness literature. Inconsistencies in the methods employed in published, peer-reviewed economic evaluation studies make results not comparable and lead to skepticism2 regarding the validity of the results of such studies. The lack of standardization in the design, analysis and reporting of economic evaluations is a matter of concern because changes in design and analysis produce substantially different results.
Independent published reviews regarding cost-effectiveness of clinical and public health interventions have been conducted by several authors. Tengs et al.3 conducted a comprehensive review on the cost-effectiveness of 500 life-saving interventions in the United States, including motor vehicle safety devices, control of environmental hazards, cancer screening, heart disease screening and treatment, vaccination for infectious diseases and interventions to promote smoking cessation. Interventions ranged from those that are cost-saving to those that cost $10 billion per life-year saved, with a median cost of $42,000 per life-year saved (1993 USD). Ramsberg et al.4 conducted a review regarding the cost-effectiveness of 165 life-saving interventions in Sweden, including screening and treatment for hypertension, road safety, smoking cessation and fire protection. Interventions ranged from those that are cost-saving to ones costing $4.9 billion per life-year saved, with a median of $19,500 per life-year saved (1993 USD). Graham et al.5 reported on the cost-effectiveness of 40 interventions in the United States, including cancer, coronary heart disease and injury prevention interventions. Cost-effectiveness ratios ranged from those costing less than $1,000 per quality-adjusted life-year (QALY) to ones costing $1 million (1995 USD) per QALY. Most of the cost-effectiveness ratios reviewed were clustered in the range of $10,000 to $100,000 per QALY.
In the process of providing the public health community with lists of the cost-effectiveness of various interventions expressed in consistent units, the previously mentioned reviews (and other reviews concentrating on a more restricted set of health conditions6, 7, 8), have also provided the first steps toward the standardization of systematic reviews of published economic evaluations. In addition to published lists of cost-effectiveness ratios, these studies have reported general inclusion and adjustment criteria. However, they usually do not provide an explicit or full description of the procedure used to adjust from the results of the primary studies to the results they report, nor do they provide the instruments used in the abstraction and adjustment of data. In addition, they leave the comparability of the results from various studies in question because they (1) make a limited number of adjustments to published ratios; and (2) report point estimates of cost-effectiveness ratios without sensitivity analysis of the adjustments.
Therefore, to our knowledge, standardized instruments for systematically translating economic data into comparable economic information have not been widely debated and are not available in the public domain.
The Guide is being developed by the Task Force on Community Preventive Services (the Task Force) in collaboration with many public and private partners under the auspices of the U.S. Department of Health and Human Services.9 The Guide will include systematic reviews of a variety of issues, including effectiveness, applicability of effectiveness, harm, other positive and negative effects, barriers to intervention implementation, and economic evaluations of selected interventions for which evidence of effectiveness is strong or sufficient based on explicit criteria. Standardized methods and instruments for searching, including and abstracting studies of effectiveness were developed for the Guide10 to reduce inconsistencies within and among chapters. A similar process was undertaken to develop methods and instruments for the systematic review of economic evaluations with the purpose of reducing error and bias in the abstraction and adjustment of results and making them comparable across interventions.
Section snippets
Methods
The methods for reviewing economic evaluations involve (1) applying explicit criteria for deciding which evaluation studies were to be included in the reviews; (2) using a standard abstraction form to record individual study characteristics, abstract data, and adjust reported results; and (3) interpreting and summarizing economic information from related studies of each effective intervention assessed.
Summarizing results: example for the Chapter on Vaccine-Preventable Diseases
An excerpt of the economic summary table for one of the interventions included in the Reviews of Evidence for Interventions to Improve Vaccination Coverage in Children, Adolescents and Adults12 is provided in Appendix B. Additional examples are shown in that report. Cost-effectiveness was expressed in this particular case as the cost of the program per additional vaccination above baseline coverage or the cost per fully vaccinated child. Baseline vaccine coverage is the coverage that would have
Discussion
At present, the body of economic evidence available to compare costs and returns of interventions to improve health is substantially limited in both quantity and comparability. Methods to review, abstract and summarize economic evaluations need to be developed and debated if economic information is going to be useful to managers, policy makers, program planners and researchers.
The development of the procedures and instruments for economic evaluation in the Guide required balancing several
Acknowledgements
The authors are particularly grateful to Marthe R. Gold MD, MPH, C. Kay Smith-Akin, MEd, Robert Deuson, PhD, Regina Pana-Cryan, PhD, and Scott Grosse PhD for their contributions and advice in the development of the economic evaluation abstraction form.
References (18)
- et al.
Data collection instrument and procedure for systematic reviews in the Guide to Community Preventive Services
Am J Prev Med
(2000) - et al.
Reviews of evidence for interventions to improve vaccination coverage in children, adolescents and adults
Am J Prev Med
(2000) Guide to clinical preventive services
(1996)- et al.
The Journal’s policy on cost-effectiveness analyses
N Engl J Med
(1994) - et al.
Five hundred life-saving interventions and their cost-effectiveness
Risk Anal
(1995) - et al.
The cost-effectiveness of life-saving interventions in Sweden
Risk Anal
(1997) - et al.
Evaluating the cost-effectiveness of clinical and public health measures
Ann Rev Pub Health
(1998) - et al.
Systematic review on the cost-effectiveness of public health interventions for HIV prevention in industrialized countries
AIDS
(1998) - et al.
Economic evaluation of cholesterol-related interventions in general practicean appraisal of the evidence
J Epidemiol Community Health
(1998)
Cited by (137)
Community Guide Methods for Systematic Reviews of Economic Evidence
2023, American Journal of Preventive MedicineA systematic review of adult tetanus-diphtheria-acellular (Tdap) coverage among healthcare workers
2019, VaccineCitation Excerpt :Thus, the aim of this study was to analyse, by performing a systematic review of literature, all published papers which evaluated Tdap coverage among HCWs, variables associated with the vaccination and efforts implemented to raise vaccination rates. This study followed the recommendations for systematic reviews published by the Centre for Reviews and Disseminations (CRD), Institute of Medicine (IOM) and Cochrane Collaboration and Community Guide to Community Preventive Services [5–7]. The following six databases were searched in Dec 7th, 2016 and updated in Apr 26, 2018: MedLine, Embase, SCOPUS, LILACS, Web of Science and Cochrane.
Community Guide Cardiovascular Disease Economic Reviews: Tailoring Methods to Ensure Utility of Findings
2017, American Journal of Preventive MedicineCost of New Technologies in Prostate Cancer Treatment: Systematic Review of Costs and Cost Effectiveness of Robotic-assisted Laparoscopic Prostatectomy, Intensity-modulated Radiotherapy, and Proton Beam Therapy
2017, European UrologyCitation Excerpt :We performed three separate searches, one each for RARP, IMRT, and proton beam therapy. We systematically searched Medline, Embase, and Web of Science for manuscripts that compared treatment with one of the new technologies to treatment with the predecessor standard treatment and that reported data on cost, with cost defined as cost analysis, cost effectiveness, cost benefit, or cost utility [13]. We limited our search to manuscripts in the English language that were published between January 2001 and July 2016, and excluded editorials.
Evaluation of School-Based Dental Sealant Programs: An Updated Community Guide Systematic Economic Review
2017, American Journal of Preventive Medicine
- 1
The names and affiliations of the Task Force members are listed on page v of this supplement and at http://www.thecommunityguide.org