Article Text
PDF
Abstract
The efficiency and effectiveness of child safety interventions are determined by the quality of the implementation process. This multinational European study aimed to identify facilitators and barriers for the three phases of implementation: adoption, implementation and monitoring (AIM process). Twenty-seven participants from across the WHO European Region were invited to provide case studies of child safety interventions from their country. Cases were selected by the authors to ensure broad coverage of injury issues, age groups and governance level of implementation (eg, national, regional or local). Each participant presented their case and provided a written account according to a standardised template. Presentations and question and answer sessions were recorded. The presentation slides, written accounts and the notes taken during the workshops were analysed using thematic content analysis to elicit facilitators and barriers. Twenty-six cases (from 26 different countries) were presented and analysed. Facilitators and barriers were identified within eight general themes, applicable across the AIM process: management and collaboration; resources; leadership; nature of the intervention; political, social and cultural environment; visibility; nature of the injury problem and analysis and interpretation. The importance of the quality of the implementation process for intervention effectiveness, coupled with limited resources for child safety makes it more difficult to achieve successful actions. The findings of this study, divided by phase of the AIM process, provide practitioners with practical suggestions, where proactive planning might help increase the likelihood of effective implementation.
- Interventions
Statistics from Altmetric.com
Introduction
There is a strong evidence base of effective child safety interventions that has been established over the last few decades.1–4 Many of these interventions have been implemented, and in the WHO European Region between 2000 and 2011, the number of deaths among children (0–14 years) due to injury has decreased by 44%.5 However, not all children in Europe enjoy the same level of protection. Child injury rates vary between and within countries and the gap in Europe, between high-income countries and low/middle-income countries, has widened.5
Widespread implementation of evidence-based child safety interventions, at all levels of governance, is one way to approach the problem.6 However, there are some important considerations during implementation. The implementation process itself is a determinant of intervention effectiveness: programmes that have been carefully implemented and are unimpeded by serious implementation problems are associated with better outcomes.7 In addition, the sustainability of interventions plays a role. Insufficient intervention duration can affect whether an intervention is effective.8
Despite the importance of implementation, scientific research in injury prevention is largely focused on outcome as opposed to process providing practitioners with little guidance as to how to make an intervention work.9–13
Several reviews have investigated the implementation process in different health contexts, such as diffusion of innovation within organisations and implementation practices in mental health and nursing.14–16 Regrettably, child safety interventions were not included in these large reviews.
There have, however, been a few studies addressing implementation issues specific to injury prevention. Brussoni et al9 explored a methodology to bring together scientific evidence and practitioner experience using the case of smoke alarm installation. The sustainability of community-based injury prevention interventions and the role of factors such as structure, process and context in the effectiveness of such interventions have been studied by Nilsen et al (2004, 2005).8 ,17 In addition, the feasibility of policy transfer for unintentional injury has been investigated.18 A recent study by Rothman et al19 explored the facilitators and enablers to enact child and youth injury prevention legislation in Canada. Finally, conceptual work by Bugeja et al20 addresses the research to practice gap in injury prevention by proposing a public policy approach to injury prevention, described from the practitioner's perspective.
Findings of these studies are broad, including the importance of windows of opportunity,20 resources9 ,18 ,19 and the challenges of multisectoral working.9
This qualitative study aims to build on this evidence base with a focus on child safety in a multinational context. The aim was to identify facilitators and barriers to adoption, implementation and monitoring of child safety interventions.
Methods
The study emerged within a large-scale European Union (EU) project: Tools to Address Childhood Trauma and Children's Safety (TACTICS).21 The implementation process was broken down into three broad phases: adoption, implementation and monitoring of good practice child safety interventions, referred to collectively as the AIM process. These phases constitute a simplified and condensed version of the stages of implementation as described by Fixsen et al,22 with additional emphasis on monitoring.
Definitions
By adoption, the authors refer to an explicit decision to take up an intervention. Implementation signifies action taken to put into operation an intervention including, as appropriate, enforcement activities. Monitoring denotes the collection and analysis of data for the specific purpose of examining how well an intervention is being implemented and its impact.
Data collection
Participants were invited to prepare a case study (presentation and a written account) of a good practice child safety intervention that had been implemented in their country.
To ensure broad coverage of the child safety field, one of the authors (JMM) developed a matrix, which was reviewed by the TACTICS scientific committee. The scope of the TACTICS project influenced the choice of injury categories due to its focus on the injury domains road, water and home safety and intentional injury prevention. To populate the matrix, participants were asked to submit good practice interventions from their countries (good practice as defined in the European Child Safety Alliance (ECSA) Child Safety Good Practice Guide).2 Cases were selected by the authors of this study to maximise coverage of issues and age groups, as well as to represent the governance level of implementation (eg, national, regional or local).
The participants prepared their presentation using a template and guidelines developed by the authors (see online supplementary appendix 1), which specifically elicited facilitators and barriers for each stage of the AIM process.
supplementary appendix
The presentations were made during two workshops that took place in Rome, Italy, in October 2011 and Copenhagen, Denmark, in May 2012. Each presentation was approximately 15 min duration. A data extraction form (see online supplementary appendix 2) was used to record details of the presentations. A question and answer session, attended by all the participants and four of the authors (BS, PS-B, JMM and JV), followed the presentations. The aim of the question and answer sessions was to clarify any unclear details and to allow free discussion to take place. Both the presentations and the question and answer sessions were audio recorded. Following the two workshops, participants wrote up their case studies using another template and guidelines (see online supplementary appendix 3) allowing them to elaborate on details of the cases.
supplementary appendix
supplementary appendix
Participants
Participants in the study were representatives from member organisations of the ECSA. The participants were either partners on the TACTICS project or individuals chosen by the project partner. Each participant represented a different country.
Ethics
Ethical approval was not sought because the scope of the study is not considered human subjects research according to the Dutch Medical Research Involving Human Subjects Act.23 Correspondingly, the ethics committee of Maastricht University does not review proposals that fall outside this definition. Nevertheless, all participants signed a project agreement as part of an EU-funded project that covered issues such as use of data and publication. Participants were informed ahead of time that presentations would be recorded.
Data analysis
Data analysis was done in three stages. In stage 1, one of the authors (BS) used thematic content analysis24 to analyse and code the data for statements of facilitators and barriers for each phase of the AIM process: adoption, implementation and monitoring. Phase 1 was concluded when all the data had been analysed and no new statements were found (data saturation). The result of phase1 was a list of facilitator and barrier statements grouped to the phase of the AIM process to which they applied. Data analysis was conducted by hand and with the use of Microsoft Excel.
In the second stage of analysis, four of the authors (BS, PS-B, KF and JMM) independently reviewed and grouped the statements into logical themes. The themes suggested by each author were then collated and harmonised, with the agreement of all the authors, into a final list of themes. The participation of the group helped ensure quality and increase objectivity.25
In the final phase of the analysis, four of the authors (BS, PS-B, KF and JMM) were asked to resort the statements, this time among the list of agreed themes. The author leading the analysis (BS) collated the results and where there were differences, the final content of each theme was agreed among all of the authors by consensus.
Results
Twenty-six cases from 26 countries in the WHO European Region were included in the study (table 1). Cases were included from six of the seven original categories of the matrix. The planned case for child maltreatment prevention was not included, as the participant was unable to present and attend the workshop.
The cases and countries included in the study
Data analysis was performed using three sources of data: the presentation slides, the written accounts and the notes taken during the workshop. In addition, we used the audio recordings to clarify and verify points; however, they were not transcribed.
The number of facilitators or barriers identified within the case studies decreased over the three phases of the AIM process. None of the case studies identified both facilitators and barriers for all three of the phases of the AIM process. The highest number of statements occurred for barriers to adoption, which had 24 statements and the lowest was 10 statements for facilitators to monitoring.
Categorisation of the statements and harmonisation of the results produced eight general themes applicable across the AIM process: management and collaboration, resources, leadership, nature of the intervention, political, social and cultural environment, visibility, nature of the injury problem and analysis and interpretation. A short description of each theme, where in the AIM process it appears and whether it was a facilitator or barrier, is displayed in table 2.
Identified themes within the adoption, implementation and monitoring (AIM) process
Adoption phase
The adoption phase (table 3) was generally characterised by facilitators and barriers to establish a collaborative partnership and building momentum for the AIM process. Strong leadership and commitment among project partners to the intervention was a facilitator. Participants described how taking a win–win approach to collaboration helped to maintain commitment and strengthen partnerships. The availability of resources (financial, human—including appropriate skills, time and data) was centrally important. Local data were used to assess the state of affairs and demonstrate the need for action, while comparative data highlighted inequalities or a low performance compared with neighbouring countries.
Facilitators and barriers identified at the adoption phase
Aspects of the intervention itself facilitated or hindered adoption. High-quality, inexpensive interventions, with good evidence of efficacy, previously trialled in other countries were easier to adopt. Interventions that constituted an extension of existing programmes and those with integrated preintervention research (eg, a needs assessment) also facilitated adoption. Interventions that were completely new were more difficult to adopt.
Political and public recognition of an issue facilitated adoption. Participants described how strong media coverage surrounding even a single injury event could benefit their campaign. Equally a lack of public demand, lack of government prioritisation and local government apathy were barriers to adoption. The nature of injury as a public health issue was a challenge at the adoption stage (eg, the need for multisectoral collaboration led to confusion among sectors concerning responsibility to act).
Implementation phase
Findings for the implementation phase (table 4) focused on maintenance of the collaborative partnership and progression through the AIM process. Facilitators included factors promoting partnership and leadership stability (such as organised, respected and enthusiastic partners). Routine project evaluation revealed problems and helped to solve them. A lack of evaluation was a barrier, particularly in the context of prolonging an existing intervention and learning from or demonstrating previous experience.
Facilitators and barriers identified at the implementation phase
Availability of sufficient resources, to match the intervention (and ideally its potential evolution), was essential. Difficulties regarding funding were said to impact human resource availability due to the time investment needed to secure funds. Some human resource issues were tangible (eg, lack of skills) and some were presented as more subjective (eg, staff fear of an increased workload); staff training and capacity building were cited as ways to address these issues.
Changes in the political, social and cultural environment affected the implementation phase and managing these changes required a flexible and innovative approach. High visibility of the injury issue and wide publicity of the intervention (eg, media interest and a dedicated website) was a facilitator. In addition, the sense that the problem being addressed was widely recognised drove momentum among organisers and decision makers.
Monitoring phase
Factors affecting the monitoring phase (table 5) were more centred on the feasibility of monitoring and some seemed to consider it an optional phase. Leadership facilitated monitoring if, for example, an external organisation, leader or champion required an evaluation as part of their participation. Likewise partnerships with institutions such as national research institutes or universities helped.
Facilitators and barriers identified at the implementation phase
The availability (or lack) of appropriate data was particularly relevant for monitoring. Practitioners aiming to establish a correlation between an intervention and a reduction in injury over time struggled to provide strong support using robust measures such as mortality rates. Moreover, it was said to be challenging to establish both baseline and follow-up measures for most injuries, because few countries have good data on non-fatal injuries, and minor injuries are not well captured by routine data collection methods.
Monitoring was, however, facilitated by predefined milestones, set project costs (including budgeting for monitoring) and integrated strategic indicators. Indicators could be continually monitored while detailed reports of milestones and project costs contributed to efforts to monitor progress. Interventions with a needs assessment (carried out during the adoption phase) also facilitated monitoring by providing a baseline of the situation before the intervention was implemented.
Discussion
This multinational study explored facilitators and barriers to the implementation process of child safety interventions. Participants presented their experiences of the AIM process and data analysis revealed eight themes: management and collaboration, resources, leadership, nature of the intervention, political, social and cultural environment, visibility, nature of the injury problem and analysis and interpretation.
Many of the themes identified were simply facilitators if present and barriers if absent. For example, resources are an advantage when present and a barrier when not. However, the discussions during the question and answer sessions that followed the presentations indicated that some of the facilitators and barriers were not independent. For instance, a well-integrated leader as part of a collaboration involving organisations with a good track record and reputation was reported to increase the likelihood of an intervention receiving funding. This was also true for barriers such as a lack of data; in one case, the presence of a key individual enabled them to initiate data collection. In this sense, there is interconnectedness between the themes we have identified and the facilitators and barriers contained within them. This idea is supported by findings from Nilsen et al8 where they discuss the interconnectedness of factors and the dangers of focusing too heavily on single factors while ignoring others.
Likewise, there seemed to be interconnections across the whole AIM process. The findings suggested that effort invested in the adoption phase appeared to pay off in later phases of implementation and monitoring. For example, building commitment to an intervention by using a win–win approach to collaboration and building a strong team early in the process appeared to contribute to other facilitators in the implementation phase, such as enthusiasm among partners, and a common understanding of the long-term nature of the process. This idea is supported by experiences in sports injury prevention26 and mental health practices.22 ,27
The AIM process also appeared to be somewhat cyclical. Participants described how demonstrated efficiency in previous interventions helped them to secure funding and support for intervention extensions and new interventions. However, many of the participants of this study did not report on the monitoring of their interventions. This was because either the intervention had not yet reached the monitoring phase or because monitoring had not taken place. This apparent lack of intervention monitoring is concerning as progress in the field of injury prevention will not be achieved without effective evaluation.28
Many parallels exist between our findings and the findings of implementation studies in injury prevention and other fields. The Quality Implementation Framework from Meyers et al16 is based on a synthesis of 25 frameworks and refers to many of the facilitators and barriers identified over the AIM process in this study. The role of, and interaction between, formal and informal leadership is explored in detail by Bryson et al29 and Armistead et al.30 In addition, Huxham31 provides a detailed overview of the management issues involved in joint working across organisations, reflecting findings such as the benefit of clear aims and roles, the need to understand the long-term nature of the process and difficulties for the collaborative partnership if a key individual is lost.
Nilsen et al8 elaborate on the challenge to achieve effective leadership, without relying too heavily on a single individual. A possible solution to this might be found in the approach taken by Donaldson et al26 to use intervention mapping as a way to create an implementation structure potentially more resilient to change.
From the injury prevention literature, our findings on the importance of policy windows and the benefit of national leadership are supported by several studies.9 ,19 ,20 Barriers identified within the theme management and collaboration (eg, challenges for multisector partnerships) and within the theme resources (challenge of short-term and inflexible funding arrangements) are also supported.9
Participant experiences contained in the theme visibility drew our attention to particularities for injury prevention among children also described by Rothman et al.19 The importance of visibility (ie, political and public recognition) of the issue is an important aspect of implementation, particularly in multisectoral collaborations.32 Participants of this study reported that emotive single injury events among children could increase public awareness of the issue. High-profile cases of an injured child could be seen as an opportunity (although a sad one) for injury prevention practitioners to draw attention to the issue, launch an intervention or highlight the preventable nature of injury and demand action. Social media may be a useful tool in this regard.33 In this sense, the political, social and cultural environment plays a significant role in visibility. As described by Hanson et al,10 ‘science can make a difference provided that research evidence is injected into public discourse in a way that is meaningful to policy makers, politicians and the general public’.
Limitations
There are some limitations to this study. First, although participants were encouraged to collaborate with others involved in the intervention on which their case study was based, this was not always possible. Some cases were presented from one person's perspective, while others were delivered by someone that had not been personally involved in the intervention. In the latter case, the presentation had been produced using interviews with relevant stakeholders. These issues may affect the validity of some of the facilitators and barriers identified.
Second, the level of detail in the presentations and written case studies varied. None of the case studies identified facilitators and barriers for all three of the stages of the AIM process and the number of facilitators and barriers decreased over the three phases. As a result, cases that provided a high level of detail may be overrepresented in the results and the adoption and implementation phases are likely to be better explored than the monitoring phase. The lack of detail regarding the monitoring phase may be due to a lack of intervention monitoring in the injury field or response fatigue among participants as the monitoring section was the last reporting section.
The presentations and written case studies were completed in English, which while the working language in the field was the second language for most participants. This was a challenge for some and is reflected in reduced detail in the written summary of the case studies. However, the question and answer sessions did allow clarification when questions arose. Overall, the consistency in facilitators and barriers identified across the interventions, which represented both different areas of child injury and the views, and experiences of practitioners working in child injury in 26 different countries suggest a reasonable level of validity.
Conclusion
This study identified facilitators and barriers to the AIM process of child safety good practice interventions. Major facilitators were effective management and collaboration, sufficient resources, a high-quality intervention and receptive political, social and cultural environment. Dominant barriers were lack of resources, lack of political support (leadership) and problems surrounding building and sustaining multisectoral collaborations (management and collaboration). In addition, facilitators in the area of visibility such as making use of a high media focus on a child injury event were highlighted.
To our knowledge, this is the first multinational study of the implementation process for child safety good practice interventions. The findings, divided by phase of the AIM process, demonstrate the importance of each phase and provide practitioners with suggested areas where proactive planning might help increase likelihood of effective implementation.
We believe that the field would benefit from further qualitative research based on the themes identified in this study. For example, research exploring the interconnectedness between the facilitators and barriers and the themes and phases of the AIM process. In addition, looking at specific mechanisms to overcome some of the barriers and identifying strategies to capitalise on facilitators would be a welcome contribution to the field.
What is already known on the subject?
Wide implementation of evidence-based child safety interventions is required to protect children from the risks of injury. However, the quality of the implementation process is a determinant of intervention effectiveness; higher levels of implementation are associated with better outcomes.
What this study adds?
This study compiled experiences of the implementation process from across Europe. The facilitators and barriers and the corresponding themes identified could help child safety practitioners avoid or manage obstacles and build in factors that will improve the quality of intervention implementation.
Why birds collide with vehicles
To better understand birds’ responses to approaching vehicles, researchers at the USDA Wildlife Services’ National Wildlife Research Center placed captive brown-headed cowbirds in a simulation chamber. The birds watched videos of vehicles approaching at various speeds. Researchers discovered that the birds could successfully avoid slower moving vehicles, but could not escape if the speed was over 75 mph. Comment: As is true for humans!
Fourth United Nations Global Road Safety week
A main theme of the upcoming Global Road Safety week is to push for a general reduction in the speed of cars. Comment: This single measure could have the greatest chance of reducing road deaths and injuries.
Questions about Black Hawk crash
The crash of a Black Hawk helicopter in 2015 caused US$7 million in damage to the aircraft and the injuries to four crew members cost the Army US$2.3 million. The cause of the crash has not been revealed but questions have arisen about maintenance and the rescue itself. The aircraft had 6392 flight hours and was not considered airworthy. The communications plan during rescue was confused because of a shortage of frequencies and a flight surgeon was available but was not part of the rescue crew.
Gasoline sniffing and suicide
Canada’s treatment of Aboriginals has been deplorable for decades and injury rates, including suicides, among these communities are far higher than elsewhere. In the 1970s, there was an epidemic of gas sniffing among young Aboriginals resulting in increases in blood lead levels. There is now speculation that the resulting brain damage may be related to the high suicide rates. A paper in Psychiatry Research suggests the mechanism may involve genetic changes resulting in depression.
Acknowledgments
Some of the findings have been previously reported in the technical report of the TACTICS project: http://www.childsafetyeurope.org/tactics/national-case-studies.html.
References
Footnotes
Contributors BS, PS-B, JMM, JV and HB were all involved in the design of the study. Data collection and analysis was conducted by BS, PS-B, JMM, JV and KF. All authors were involved in revision of the manuscript.
Funding European Commission under the EU Health Programme 2008–2013 project number 20101212.
Competing interests None.
Provenance and peer review Not commissioned; externally peer reviewed.