“We address our safety campaigns to individuals”, wrote Patricia Barry in a 1975 article in Preventive Medicine (page 48).¹ Her description came at a time when individual error, negligence, misuse or abuse of equipment were typically viewed by injury prevention specialists as forming the causes of most injuries. Barry pleaded for a change of focus in injury prevention from the individual to the community.

Today, nearly 30 years later, the battle is largely won; involving communities in injury reduction programs has become a widely accepted strategy. Paralleling the shift away from disease prevention to health promotion, the community oriented approach to injury prevention attempts to implement changes which will simultaneously affect many individuals.² The model emphasises community participation and multidisciplinary collaboration,³ recognising that those most able to solve local injury problems are those people who live in that particular community.⁴ The World Health Organisation (WHO) Safe Community model is a well known framework for community level injury prevention that uses multiple strategies and targets all age groups, environments, and situations.⁵

The community based approach to injury prevention has gained widespread acceptance, but there still is a paucity of solid evidence of effectiveness of multitarget, multistrategy programs implemented in the community context. While Rahman (page 147⁶) claims that the Safe Community model is “recognised as an effective and long-term beneficial approach”, Langley and Alsop (page 132⁷) suggest that “greater caution should be exercised in promoting these broad, multifaceted intervention programs”. Petridou et al (page 174⁸) maintain that the findings have ranged “from encouraging to disappointing” and Klassen et al (page 101⁹) find the results “mixed at best”. Furthermore, there is little, if any, evidence about the cost effectiveness of community health promotion programs involving multiple strategies.^10,11 Few of the multifaceted community based programs have been evaluated and, to date, there are only a handful of longitudinal studies that document long term sustainability.⁵ It has been suggested that many of the available evaluations lack the methodological rigour required to provide conclusive evidence that the effects were attributable to the program.¹²

The aim of this study is to examine existing evaluations of multistrategy, multitarget community based injury prevention programs with regard to reported effectiveness. Factors related to the structure and process of the program and the context within which the program exists are analysed in order to examine how they influence the program outcomes. The objective is to contribute to a better understanding of modern community based injury prevention.

METHODS

Inclusion criteria and definitions

Programs were included in this analysis if they met four criteria: the program had to be (1) community based, (2) multistrategy, (3) multitarget, and (4) an evaluation of the program had to be published in a peer reviewed scientific journal. The programs included thus had to be community-wide, targeting a group or groups of individuals in a community and involving community representatives in the goal setting, program design, and implementation. They also had to employ multiple strategies—for example, education, home inspections, and environmental improvements. Finally, the programs had to target multiple injury categories: injury types (for example, burns, falls, and poisoning), injury environments (home, traffic, work, sports, etc), and other classifications, such as causes (for example, injuries due to alcohol misuse or violence).

Community based programs that addressed a specific injury category, for example, only falls or burns, were excluded. Programs focusing on a specific population subgroup (for example, a certain age group or risk group) had to target multiple injury categories in the target group to “qualify” for inclusion in this study.

Data collection

The program evaluations included in this study were obtained through systematic literature searches of the speciality injury prevention journals and journals in the domains of health promotion and public health. Relevant textbooks on the subject were also studied. Additionally, the PubMed (biomedical research) and SafetyLit (injury prevention research) databases were searched and abstracts of articles were inspected for contents pertaining to injury prevention evaluations. The following search terms (and combinations thereof) were used: “community”; “community-based”; “community-wide”; “community-oriented”; “injury prevention”; “safety promotion”; “program”; “intervention”; “project”; “initiative”.

Another source of program information was the Safe Community website operated by Karolinska Institutet, Stockholm, Sweden, which provides detailed information, including listings of published papers, on all designated Safe Community members. Each of the 78 Safe Communities was examined with regards to published evaluations. Table 1 lists the 16 programs that satisfied the selection criteria.

Data analysis

The analysis was aimed at elucidating to what extent the program outcome, that is, effectiveness, was associated with elements of the structure and process of the program and the environment in which the program was implemented. Effectiveness was predominantly assessed as injury rate reductions (what should count as evidence of effectiveness is not an uncontested issue, but this is not addressed in this study).

The analysis of the gathered data was performed as a structured review of the literature on each program. The “results” section of the published evaluations was studied to assess the effectiveness of the programs, while the “discussion” section was examined to analyse methodological limitations and determine the trustworthiness of the results. The assessments were made by the author. Information about the context, structure, and process of the programs was based on descriptions found in the “methods” and “results” sections of the evaluations. The “discussion” section provided the main information for the analysis of how the context, structure, and process may have influenced the results.

Table 2 summarises key findings regarding the effectiveness of the programs under study. Table 3 contains descriptions of contextual, structural, and process characteristics of the programs.

RESULTS

Outcome

The effectiveness of the 16 injury prevention programs varied considerably, from programs achieving little or no measurable effects, to programs that were associated with dramatic injury rate reductions. The programs can be classified into four broad categories based on assessments of the effectiveness of the program (injury incidence decrease) and the scientific rigour of the program evaluation.

Category A programs achieved substantial and statistically significant decreases in injury incidence, reporting injury rate reductions of between 15% and 50% for many of the targeted injury categories. The evaluations of these programs employed sufficiently rigorous quasiexperimental designs, incorporating adequate comparisons that supported the trustworthiness of the findings. Category B programs demonstrated more mixed results, while the programs of category C evidenced only minor or no degree of effectiveness. Category D is comprised of the programs that were evaluated without comparison areas. Some of the programs belonging to category D did demonstrate very positive results, but the weak evaluation design suggests that caution is warranted when interpreting the findings.

Context

Excluding the culturally defined community of CICPIHS (which targeted Native Americans across the United States), all programs defined community in geographical terms. Ranging from the small island of Vaeröy, Norway (1100 population) to Waitakere, New Zealand (156 000), the communities can be classified into three categories: (1) small rural towns; (2) “medium sized” towns and cities (population 22 000 to 55 000) with modest population density; and (3) urbanised regions with high population density. It is noteworthy that the four most successful programs were implemented in “medium sized” Scandinavian communities: Harstad, Lidköping, Motala, and Falun. However, aside from this observation, it was not possible to discern an association between intervention area population size and program effectiveness.

The social and cultural homogeneity of intervention communities was identified in four of the studies as important influences on program success. In some cases, when programs did not factor in community cohesion levels into program delivery, the programs suffered. The Safe Living study pointed to a lack of community cohesiveness, which made it difficult to achieve the desired outcomes. It was suggested that community based injury prevention programs “may work better in cohesive, homogeneous, stable, and isolated communities” (page 22¹³). The Illawara evaluation emphasised that the intervention area of Shellharbour is a geographical but not a cultural unity. There is a high level (about 20%) of migrant families whose first language is not English and a corresponding range of social and cultural values. This resulted in a poor sense of local community, which was discussed as an important reason for limited results.

The Waitakere program in New Zealand adapted to the ethnic heterogeneity of the community by separating the program into three components: a coordinator was appointed for each of the three population groups (Maori, Pacific, and general population). The evaluators felt the role of the project coordinators was pivotal to the success of the program and they concluded that the flexibility required to establish cultural procedures and practices for all cultures represented in the community was recognised. The Waitakere coordinators had limited previous knowledge of injury prevention, but, more importantly, they all had community development experience and knowledge of appropriate cultural processes. A similar approach was taken in the Safe Block program, in which individuals with minimal formal education but cultural affinity with the intervention area, were trained to coordinate the program. The Safe Block evaluators emphasised the critical importance of understanding community residents’ perceptions and values.

Another important contextual factor addressed in three studies was the socioeconomical status of the intervention area. However, the findings on the importance of this factor in program success were mixed. Illawara was described a “predominantly low socioeconomic community” and the evaluators concluded that social and economic issues may have had higher priority than injury prevention, which could have contributed to the limited program success (page 371¹⁴). The Safe Kids program was implemented in a disadvantaged community, with 39.5% of the population below poverty level, while the Safe Block program targeted a poor population (a median family income was given but it was not related to a national average or any other meaningful comparison). Neither the Safe Kids nor the Safe Block evaluators identified the socioeconomical status as a factor that affected the outcome negatively. On the contrary, Safe Block emphasised that the program demonstrated that it was feasible to carry out the program in “extremely poor, inner-city neighbourhoods” (page 680¹⁵).

Structure

The structure of the programs was analysed to determine to what extent the duration of individual programs were related to outcomes. Most successful programs were generally longer lasting than the other programs. Indeed, the four programs (Motala, Falun, Lidköping, and Harstad) that were studied over the longest time periods all belonged to category A of highly successful programs. On the other hand, all the category B programs, which also demonstrated considerable success, lasted only between one and three years.

Insufficient intervention effort and duration were discussed as important factors explaining a lack of significant success in five of the program evaluations: five city, Safe Kids, SCIPP, Naxos, and Safe Living. The SCIPP project was maintained for only 22 months because of limited resources. The evaluators believed that the benefits of changed safety knowledge and practices would manifest themselves over a longer time frame, and concluded that length of exposure “may be important to the success” of community based injury prevention programs (page 1525¹⁶). Similarly, the Naxos project lasted 20 months, which was thought to have been insufficient to reverse lifelong influences and risk-prone lifestyles.

The majority of the studies emphasised the importance of injury surveillance systems for the overall outcome of the programs. Injury surveillance systems were used in most programs to gather local injury data in order to determine the extent and nature of the injury problem within the community and to establish target injury categories. Many evaluations stressed the importance of local networking among community groups for the long term success of the programs. In fact, some evaluators viewed the synergy created by the local collaboration as the most important overall result of the program.

Process

The majority of the programs combined elements of passive interventions—that is, measures requiring little individual action on the part of those being protected, and active interventions, which require more participation from the individual. Active interventions were primarily educational, informational, and incorporating training technique, while the most frequently used passive intervention was improvements to the environment. The Harstad, Motala, Lidköping, Falköping, and SCIPP programs emphasised passive intervention measures, while Falun, Vaeröy, and the five city project relied mostly on active intervention (educational) efforts. The remaining eight programs used roughly equal “amounts” of passive and active measures. Three of the four category A programs (that is, the programs deemed most effective), Lidköping, Motala, and Harstad, relied primarily on passive measures. However, the fourth category A program, Falun, emphasised active intervention, underscoring the difficulty of finding a consistent pattern as regards which approach was the most successful.

Four of the program evaluations (Safe Block, CICPIHS, SCIPP, and Falun) attempted to assess “dose-response” relationships by linking process data to outcome, investigating which activities had the most effect in changing injury patterns. However, the evaluators were not able to associate the changes in effects by the specific type of intervention strategy.

DISCUSSION

The results of this study suggest that there is a complex relationship between the outcome and the context, structure, and process of community-wide injury prevention programs. Certain structural and process elements may work in one context, but not in another environment or at another point in time. Indeed, it has been shown that the application of an effective strategy, but at an inappropriate stage of community readiness, can delay or disable an entire project.¹⁷ It has even been suggested that community based programs implemented under less than ideal conditions have produced fairly negative results.¹⁸

The results highlight the importance of the contextual conditions of the programs, confirming that the socioeconomical status and social/cultural homogeneity of the intervention community are critically important factors that influence the effectiveness of the programs. While the evaluations of some of the most successful programs did not examine the level of community homogeneity of the intervention area and how this related to program success, it is obvious that the programs in Lidköping, Falun, Motala, and Harstad were implemented in affluent and reasonably cohesive communities. When similar programs were developed in Australia (Latrobe Valley, Safe Living, Illawara) and New Zealand (Waitakere), it was difficult to replicate the success of the Scandinavian programs because community cohesion levels were very different.

The analysis demonstrated that intervention programs cannot be seen as standardised “kits” to be distributed with minimal reference to the environment in which they are implemented; there is no blueprint for injury prevention programs. Other systematic reviews have revealed that injury prevention programs are generally more effective when they are tailored to address unique community characteristics, such as ethnicity or socioeconomic status.⁹ Welander et al (page 99¹⁹) conclude that the WHO Safe Community model “needs to be adapted and modified to the cultural and socioeconomic conditions and existing health set-ups of individual countries”, emphasising that “the structure used to promote safety will vary from community to community and country to country”. Still, there is no question that the role of contextual factors in the success of community injury prevention programs is often ignored in discussions of community based injury prevention. The amount of research concerned with interaction between contextual and other factors of injury prevention programs is negligible.¹⁸

Some of the programs examined in this review indicate that community level injury epidemiology data may not always capture the nuances of the injury panorama of a community with large variances in socioeconomic status or several culturally disparate subpopulations. Hence, community-wide interventions attempting to simultaneously affect as many individuals as possible may not be as successful in heterogeneous societies because community level injury data will have limited relevance among many segments of the population. There is reason to believe that “community” more frequently should be defined in cultural terms, rather than geographical, in order to improve community participation and community ownership of the effort.

Rather predictably, the study supports the conclusion that a certain degree of duration and intensity can be seen as necessary, but not always sufficient, conditions to ensure a successful program. This is consistent with studies that have shown that the implementation of effective community initiatives requires much time, effort, organisation, and resources.^20,21 Regrettably, no evaluation reported the financial resources devoted to the program. The process evaluations yielded inconsistent evidence of which type of intervention was most effective. It is generally accepted that programs which combine different intervention strategies rather than relying on a single strategy are most likely to be successful.^22,23 Passive environmental approaches have been identified as the most effective approach,²⁴ and it has been suggested that health education (active intervention) should not be used in isolation, instead being part of a balanced approach.²⁵ Three of the four most successful programs of this study emphasised passive interventions, but the fourth program in this “top” category relied almost exclusively on active interventions, indicating that the choice of strategies is one of several interdependent factors influencing the outcome.

Ultimately, this study demonstrates the current difficulties of finding scientific evidence to support the community based approach to injury prevention programming. Because of the inherent complexity of multistrategy and multitarget programs, and the interdependency between contextual, structural, and process aspects, it is still difficult to provide solid evidence describing the factors that are most important to achieve program effectiveness. Some of the problems of identifying key success factors are due to meagre descriptions of community characteristics and conditions, insufficient assessment of structural program components, and failure to establish process-outcome relationships. To advance the understanding of community-wide injury prevention programs, the evaluations that accompany them must develop in complexity.

CONCLUSIONS

Thirty years after the implementation of the first community based injury prevention programs, local involvement has become an important component of such programming. While the rationale for community based injury prevention programs is logically sound, there is, unfortunately, still a lack of literature describing the effectiveness of this approach to reduce community injury rates.

The results provided evidence for a complex relationship between the context, structure, and process of the programs, which makes it difficult to isolate individual elements or finding success factors that apply to all programs. The evaluations were found to be lacking important information pertaining to contextual conditions and structural elements of the programs. Given the importance of evaluations of injury prevention programs to the evolving practice, there is a definite need for more sophisticated evaluations.