Original Articles
Developing an evidence-based guide to community preventive services—methods12

https://doi.org/10.1016/S0749-3797(99)00119-1Get rights and content

Abstract

Abstract:

Systematic reviews and evidence-based recommendations are increasingly important for decision making in health and medicine. Over the past 20 years, information on the science of synthesizing research results has exploded. However, some approaches to systematic reviews of the effectiveness of clinical preventive services and medical care may be less appropriate for evaluating population-based interventions. Furthermore, methods for linking evidence to recommendations are less well developed than methods for synthesizing evidence.

The Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations (the Guide) will evaluate and make recommendations on population-based and public health interventions. This paper provides an overview of the Guide’s process to systematically review evidence and translate that evidence into recommendations.

The Guide reviews evidence on effectiveness, the applicability of effectiveness data, (i.e., the extent to which available effectiveness data is thought to apply to additional populations and settings), the intervention’s other effects (i.e., important side effects), economic impact, and barriers to implementation of interventions.

The steps for obtaining and evaluating evidence into recommendations involve: (1) forming multidisciplinary chapter development teams, (2) developing a conceptual approach to organizing, grouping, selecting and evaluating the interventions in each chapter; (3) selecting interventions to be evaluated; (4) searching for and retrieving evidence; (5) assessing the quality of and summarizing the body of evidence of effectiveness; (6) translating the body of evidence of effectiveness into recommendations; (7) considering information on evidence other than effectiveness; and (8) identifying and summarizing research gaps.

Systematic reviews of and evidence-based recommendations for population-health interventions are challenging and methods will continue to evolve. However, using an evidence-based approach to identify and recommend effective interventions directed at specific public health goals may reduce errors in how information is collected and interpreted, identify important gaps in current knowledge thus guiding further research, and enhance the Guide users’ ability to assess whether recommendations are valid and prudent from their own perspectives. Over time, all of these advantages could help to increase agreement regarding appropriate community health strategies and help to increase their implementation.

Introduction

The Guide to Community Preventive Services: Systematic Reviews and Evidence-Based Recommendations (the Guide) is an initiative of the U.S. Department of Health and Human Services and is being developed by a 15-member, independent, nonfederal Task Force on Community Preventive Services (the Task Force) in cooperation with many public and private sector partners.1 The Task Force is supported by staff of the Centers for Disease Control and Prevention (CDC) and others who are developing, disseminating, and implementing the Guide. The Guide will make specific recommendations on selected interventions defined as activities that prevent disease or injury or that promote health in a group of people. Preventive interventions for individuals are considered in the Guide’s sister publication, the Guide to Clinical Preventive Services,2 and are not included in the Guide to Community Preventive Services. Interventions recommended in the Guide will address 15 major topic areas (i.e., chapters) selected by the Task Force.3 Chapters are organized into the following three major sections: (1) changing risk behaviors (e.g., reducing tobacco product use or increasing levels of physical activity); (2) reducing specific diseases, injuries, or impairments (e.g., reducing the occurrence of vaccine-preventable diseases or cancer); and (3) addressing ecosystem and environmental challenges (e.g., reducing health disparities attributable to differences in socioeconomic status). Guide reviews4, 5 and recommendations6, 7 are expected to be released as they are completed over the next two years; they will also be collected and published in book form.

Systematic reviews and evidence-based recommendations are playing an increasingly important role in decision-making about health-related issues. Over the past 20 years, information on the science of synthesizing research results has exploded.8, 9 Much of the available work on research synthesis on health related topics has, however, focused on clinical preventive services (i.e., preventive practices applied to target conditions among asymptomatic individuals)2 and medical care.9, 10, 11 Furthermore, experience in linking evidence to practice recommendations exists2, 12 but is more limited. Therefore, the Task Force is working to refine available approaches to systematic reviews and evidence-based recommendations for population-based and public health interventions. This paper provides an overview of the process being used to review evidence and to translate that evidence into recommendations provided in the Guide. An example of a review and its resulting recommendations that were developed using these methods is shown elsewhere in this issue.5, 7

Section snippets

Methods for developing reviews and recommendations

The Task Force determined that recommendations in the Guide should be based on systematic reviews of evidence aimed at showing the relationship of the intervention to particular outcomes and an explicit process for translating the evidence into recommendations. In the Guide, the term evidence includes: (1) information that is appropriate for answering questions about an intervention’s effectiveness; (2) the applicability of effectiveness data (i.e., the extent to which available effectiveness

Chapter development teams

Because of the broad and multidisciplinary character of many public health problems, the Task Force employs chapter development teams representing diverse perspectives. Approximately 4–10 individuals with methodologic or subject matter expertise lead the development of a chapter. An additional 15–20 subject matter experts, including practitioners, advise on chapter development. The broad experience of the team members is vital to: (1) ensure the usefulness and comprehensiveness of the

Develop a conceptual approach to organizing, grouping, selecting and evaluating the interventions evaluated in each chapter

The breadth of each Guide chapter requires the chapter development team to identify key areas on which to focus. A logic framework is a diagram mapping out a chain of hypothesized causal relationships among determinants, intermediate, and health outcomes. The logic framework is used to identify links between social, environmental, and biological determinants and pertinent outcomes; strategic points for action; and interventions that might act on those points. Perhaps most important, logic

Select interventions to be evaluated

An intervention is characterized by what was done, how it was delivered, who was targeted, and where it was delivered. Interventions can be either single-component—using only one activity—or multicomponent—using more than one related activity. Because population-based interventions usually are heterogeneous, the chapter development team must make explicit judgments about the extent to which interventions will be considered in the same body of evidence. These judgments are based on

Systematically search for and retrieve evidence

Analytic frameworks provide some of the inclusion criteria for identifying evidence by specifying the intervention(s) considered and the pertinent outcome(s). Other inclusion criteria are also specified (e.g., countries and years in which the study was conducted and languages in which it was communicated). Searches are performed for literature published in books and journals meeting the inclusion criteria and include searches of multiple computerized databases, reviews of reference lists, and

Assess the quality of and summarize a body of evidence of effectiveness

After the individual studies making up the body of evidence of effectiveness for an intervention are identified, they are evaluated, their results are extracted, the overall body of evidence is summarized, and the strength of the body of evidence (i.e., the confidence that changes in outcomes are attributable to the interventions) is assessed.

Each study that meets the explicit inclusion criteria is read by two reviewers who use a standardized abstraction form15 to record information about: (1)

Effectiveness

In general, strength of evidence of effectiveness (Table 2) links directly to strength of recommendation (Table 3). Evidence that is inconsistent in direction or size of effect based on definable characteristics of the population, setting, or the intervention should lead to separate recommendations for different situations. For example, some interventions could be recommended for urban populations but not for rural populations. Other interventions could be recommended in health department

Summarizing research gaps

The systematic reviews in the Guide identify existing information on which to base public health recommendations. An important additional benefit of these reviews is the identification of areas where information is lacking or of poor quality.

Discussion

The Guide builds on considerable previous experience in systematic reviews8, 9, 10, 11 and somewhat less experience in linking evidence to recommendations.2, 19, 20, 21 The Guide shares with those processes a commitment to a systematic process and to explicitness.

Population-based interventions differ from individual interventions that have been the focus of many previous health-related systematic reviews because of the level of scale at which the interventions are implemented and at which

References (27)

  • Mulrow CD, Oxman AD, eds. Cochrane collaboration handbook. In: The Cochrane library [CDROM, updated September 1997]....
  • The Cochrane Collaboration. The Cochrane database of systematic reviews, 1999 vol 2 [online database]. Available at...
  • Cited by (497)

    • Community Guide Methods for Systematic Reviews of Economic Evidence

      2023, American Journal of Preventive Medicine
    View all citing articles on Scopus
    1

    Some of this material has been previously published in: Shefer A, Briss P, Rodewald L, et al. Improving immunization coverage rates: An evidence-based review of the literature. Epidemiologic Reviews 1999;20:96–142.

    2

    The names and affiliations of the Task Force members are listed on page v of this supplement and at http://www.thecommunityguide.org

    View full text