Original Articles
Methods for systematic reviews of economic evaluations for the guide to community preventive services1

https://doi.org/10.1016/S0749-3797(99)00120-8Get rights and content

Abstract

Objectives: This paper describes the methods used in the Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations (the Guide) for conducting systematic reviews of economic evaluations across community health-promotion and disease-prevention interventions. The lack of standardized methods to improve the comparability of results from economic evaluations has hampered the use of data on costs and financial benefits in evidence-based reviews of effectiveness. The methods and instruments developed for the Guide provide an explicit and systematic approach for abstracting economic evaluation data and increase the usefulness of economic information for policy making in health care and public health.

Methods: The following steps were taken for systematic reviews of economic evaluations: (1) systematic searches were conducted; (2) studies using economic analytic methods, such as cost analysis or cost-effectiveness, cost-benefit or cost-utility analysis, were selected according to explicit inclusion criteria; (3) economic data were abstracted and adjusted using a standardized abstraction form; and (4) adjusted summary measures were listed in summary tables.

Results: These methods were used in a review of 10 interventions designed to improve vaccination coverage in children, adolescents and adults. Ten average costs and 14 cost-effectiveness ratios were abstracted or calculated from data reported in 24 studies and expressed in 1997 USD. The types of costs included in the analysis and intervention definitions varied extensively. Gaps in data were found for many interventions.

Introduction

This paper describes the procedures and instruments used to collect, abstract, adjust, and summarize results from economic studies reporting on cost, cost-effectiveness, cost-benefit or cost-utility of selected interventions for the Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations (the Guide). Methods were developed so that studies using disparate analytical methods can be consistently compared. Although these methods were developed specifically for the Guide, they are sufficiently complete and general enough to be readily adapted to a range of systematic reviews of economic evaluations in health care and public health.

Evidence-based reviews of effectiveness (e.g., the Guide to Clinical Preventive Services1) usually have not reported data regarding costs and financial benefits of preventive interventions. Users of evidence-based effectiveness reviews often do not have enough information to identify among effective interventions, those providing the greatest amount of health or financial benefit per dollar of resource used. This might be because of the absence of economic evaluation studies or problems of comparability among available studies. Although little can be done on a short-term basis to increase the quantity of available studies, systematic reviews of economic studies can address problems of comparability. In addition, they can improve the usefulness of existing studies just as systematic reviews bring together and interpret a body of evidence of the effectiveness of interventions.

The lack of accepted, or even openly debated, methods and instruments for systematic reviews of economic studies is significant in light of both the need for such methods and the precedent set for such methods in the effectiveness literature. Inconsistencies in the methods employed in published, peer-reviewed economic evaluation studies make results not comparable and lead to skepticism2 regarding the validity of the results of such studies. The lack of standardization in the design, analysis and reporting of economic evaluations is a matter of concern because changes in design and analysis produce substantially different results.

Independent published reviews regarding cost-effectiveness of clinical and public health interventions have been conducted by several authors. Tengs et al.3 conducted a comprehensive review on the cost-effectiveness of 500 life-saving interventions in the United States, including motor vehicle safety devices, control of environmental hazards, cancer screening, heart disease screening and treatment, vaccination for infectious diseases and interventions to promote smoking cessation. Interventions ranged from those that are cost-saving to those that cost $10 billion per life-year saved, with a median cost of $42,000 per life-year saved (1993 USD). Ramsberg et al.4 conducted a review regarding the cost-effectiveness of 165 life-saving interventions in Sweden, including screening and treatment for hypertension, road safety, smoking cessation and fire protection. Interventions ranged from those that are cost-saving to ones costing $4.9 billion per life-year saved, with a median of $19,500 per life-year saved (1993 USD). Graham et al.5 reported on the cost-effectiveness of 40 interventions in the United States, including cancer, coronary heart disease and injury prevention interventions. Cost-effectiveness ratios ranged from those costing less than $1,000 per quality-adjusted life-year (QALY) to ones costing $1 million (1995 USD) per QALY. Most of the cost-effectiveness ratios reviewed were clustered in the range of $10,000 to $100,000 per QALY.

In the process of providing the public health community with lists of the cost-effectiveness of various interventions expressed in consistent units, the previously mentioned reviews (and other reviews concentrating on a more restricted set of health conditions6, 7, 8), have also provided the first steps toward the standardization of systematic reviews of published economic evaluations. In addition to published lists of cost-effectiveness ratios, these studies have reported general inclusion and adjustment criteria. However, they usually do not provide an explicit or full description of the procedure used to adjust from the results of the primary studies to the results they report, nor do they provide the instruments used in the abstraction and adjustment of data. In addition, they leave the comparability of the results from various studies in question because they (1) make a limited number of adjustments to published ratios; and (2) report point estimates of cost-effectiveness ratios without sensitivity analysis of the adjustments.

Therefore, to our knowledge, standardized instruments for systematically translating economic data into comparable economic information have not been widely debated and are not available in the public domain.

The Guide is being developed by the Task Force on Community Preventive Services (the Task Force) in collaboration with many public and private partners under the auspices of the U.S. Department of Health and Human Services.9 The Guide will include systematic reviews of a variety of issues, including effectiveness, applicability of effectiveness, harm, other positive and negative effects, barriers to intervention implementation, and economic evaluations of selected interventions for which evidence of effectiveness is strong or sufficient based on explicit criteria. Standardized methods and instruments for searching, including and abstracting studies of effectiveness were developed for the Guide10 to reduce inconsistencies within and among chapters. A similar process was undertaken to develop methods and instruments for the systematic review of economic evaluations with the purpose of reducing error and bias in the abstraction and adjustment of results and making them comparable across interventions.

Section snippets

Methods

The methods for reviewing economic evaluations involve (1) applying explicit criteria for deciding which evaluation studies were to be included in the reviews; (2) using a standard abstraction form to record individual study characteristics, abstract data, and adjust reported results; and (3) interpreting and summarizing economic information from related studies of each effective intervention assessed.

Summarizing results: example for the Chapter on Vaccine-Preventable Diseases

An excerpt of the economic summary table for one of the interventions included in the Reviews of Evidence for Interventions to Improve Vaccination Coverage in Children, Adolescents and Adults12 is provided in Appendix B. Additional examples are shown in that report. Cost-effectiveness was expressed in this particular case as the cost of the program per additional vaccination above baseline coverage or the cost per fully vaccinated child. Baseline vaccine coverage is the coverage that would have

Discussion

At present, the body of economic evidence available to compare costs and returns of interventions to improve health is substantially limited in both quantity and comparability. Methods to review, abstract and summarize economic evaluations need to be developed and debated if economic information is going to be useful to managers, policy makers, program planners and researchers.

The development of the procedures and instruments for economic evaluation in the Guide required balancing several

Acknowledgements

The authors are particularly grateful to Marthe R. Gold MD, MPH, C. Kay Smith-Akin, MEd, Robert Deuson, PhD, Regina Pana-Cryan, PhD, and Scott Grosse PhD for their contributions and advice in the development of the economic evaluation abstraction form.

References (18)

There are more references available in the full text version of this article.

Cited by (137)

  • Community Guide Methods for Systematic Reviews of Economic Evidence

    2023, American Journal of Preventive Medicine
  • A systematic review of adult tetanus-diphtheria-acellular (Tdap) coverage among healthcare workers

    2019, Vaccine
    Citation Excerpt :

    Thus, the aim of this study was to analyse, by performing a systematic review of literature, all published papers which evaluated Tdap coverage among HCWs, variables associated with the vaccination and efforts implemented to raise vaccination rates. This study followed the recommendations for systematic reviews published by the Centre for Reviews and Disseminations (CRD), Institute of Medicine (IOM) and Cochrane Collaboration and Community Guide to Community Preventive Services [5–7]. The following six databases were searched in Dec 7th, 2016 and updated in Apr 26, 2018: MedLine, Embase, SCOPUS, LILACS, Web of Science and Cochrane.

  • Cost of New Technologies in Prostate Cancer Treatment: Systematic Review of Costs and Cost Effectiveness of Robotic-assisted Laparoscopic Prostatectomy, Intensity-modulated Radiotherapy, and Proton Beam Therapy

    2017, European Urology
    Citation Excerpt :

    We performed three separate searches, one each for RARP, IMRT, and proton beam therapy. We systematically searched Medline, Embase, and Web of Science for manuscripts that compared treatment with one of the new technologies to treatment with the predecessor standard treatment and that reported data on cost, with cost defined as cost analysis, cost effectiveness, cost benefit, or cost utility [13]. We limited our search to manuscripts in the English language that were published between January 2001 and July 2016, and excluded editorials.

View all citing articles on Scopus
1

The names and affiliations of the Task Force members are listed on page v of this supplement and at http://www.thecommunityguide.org

View full text