Background

There is considerable high-quality evidence that exercise training programmes can prevent lower limb injuries (LLI) in sport.1–4 However, relatively few studies have investigated the implementation or effectiveness of such injury prevention programmes (IPPs) in real-world settings and contexts.5 6 Frameworks such as the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework provide a solid basis for designing and evaluating the implementation of IPPs.7 8 However, the specific RE-AIM components for the implementation of IPPs in team ball sports are rarely reported.6

Numerous cross-sectional studies have reported the level of awareness, adoption and implementation of LLI prevention programmes among sports participants and coaches.9–13 In general, these have found moderate to high awareness, limited programme use and very limited programme fidelity. Several cross-sectional follow-up studies assessing coach awareness or uptake of, or player compliance with, LLI prevention programmes following an intervention (eg, coach training workshop, resource dissemination, programme delivery by an external agent and so on) have also reported high post-intervention awareness, moderate programme use and high programme modification.14–18

Four published studies have evaluated the impact of different strategies on the implementation of sports injury prevention interventions by community sports coaches and participants. In a study of the impact of a squash eye protection promotion strategy, players exposed to the strategy were 2.4 times more likely to wear appropriate eyewear than players exposed to usual implementation practice.19 An examination of team and player adherence to the FIFA11+ LLI prevention programme showed that soccer teams with coaches exposed to an 11+ workshop performed a significantly higher number of 11+ exercises compared with teams with coaches who accessed the 11+ online.20 Rugby union coaches exposed to a theory-informed, evidence-informed and context-informed diffusion plan to promote the uptake of a neck and spinal IPP had greater increases in programme knowledge, frequency of training players in the programme and perceived quality of programme delivery than control coaches.21 Finally, a study comparing an interactive mobile application to written materials to implement an ankle IPP showed equal compliance among athletes exposed to either implementation method.22

The National Guidance for Australian Football Partnerships for Safety study is one of the first large-scale sports injury prevention implementation studies to use a controlled, ecological design to assess the effectiveness of an exercise training IPP at the population level. Its associated IPP, known as FootyFirst, is an exercise training programme developed to target LLI in community Australian football (community AF).23 An evidence-informed and context-specific implementation plan was developed to raise awareness of FootyFirst and to support its uptake and delivery in community AF clubs in three specific geographic regions.24

This paper evaluates the impact of the multistrategy implementation plan on FootyFirst-related implementation activities. Implementation activities are compared across community AF regions and clubs that received FootyFirst with implementation support (FootyFirst+S) and FootyFirst without implementation support (FootyFirst+NS). It was hypothesised that there would be greater FootyFirst implementation activity among clubs in the FootyFirst+S regions than in clubs in the FootyFirst+NS region.

Methods

A controlled ecological trial, underpinned by the principles of pragmatic health research,25 was conducted with community AF clubs. Full details of the study design, including additional information about the choice and locations of the three geographical regions, are available elsewhere.26 The evaluation design was underpinned by the RE-AIM Sports Setting Matrix.8 This paper reports the impact of the implementation plan on the FootyFirst implementation activity—reach, perceived effectiveness, adoption, implementation and maintenance—among the targeted clubs.

Results

The total RE-AIM scores were more than 2.1 times higher for clubs in the two FootyFirst+S regions than for clubs in the FootyFirst+NS region, after the first year of implementation of FootyFirst in each region (table 3). In addition, there was no significant difference between the total RE-AIM scores between the clubs in the two FootyFirst+S regions after the first year of implementation. This suggests that the addition of the implementation support to FootyFirst helped facilitate the desired implementation activity in the targeted clubs.

There was no evidence of implementation of FootyFirst among clubs in the FootyFirst+NS region, and these clubs had lower dimension-specific scores for nearly all of the other RE-AIM dimensions, compared with clubs in the FootyFirst+S regions (table 3). There were non-zero dimension-specific scores for all RE-AIM dimensions for clubs in the FootyFirst+S regions.

When the RE-AIM dimension and total RE-AIM scores were compared after 2 years of FootyFirst implementation in region 1 (FootyFirst+S) and region 2 (FootyFirst+NS), clubs in region 1 scored significantly higher on every RE-AIM dimension than did clubs in region 2: reach: 1.89 versus 1.44; perceived effectiveness: 0.75 versus 0.18; adoption: 1.20 versus 0.22; implementation: 0.23 versus 0.00; maintenance: 0.66 versus 0.14 (figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Comparison of mean RE-AIM dimension scores (range 0–2) and total RE-AIM scores (range 0–10) for clubs in region 1 (FootyFirst+S) and region 2 (FootyFirst+NS), after 2 years of FootyFirst implementation. FootyFirst+NS,  FootyFirst without implementation support; FootyFirst+S, FootyFirst with implementation support; RE-AIM, Reach Effectiveness Adoption Implementation Maintenance.

Discussion

There has been considerable attention given to preventing LLIs in sport over recent years, and several exercise training interventions have been developed and evaluated. However, the effectiveness of these interventions has been limited by the extent to which the target groups (eg, players or coaches) have implemented them.18 32 The majority of the evidence of the efficacy exercise training interventions comes from RCT trials, many of which have rarely considered implementation factors. When studies have considered implementation issues, this has typically been as a minor component, and most have reported limited aspects of intervention implementation.33 The evaluation reported in this paper addresses this significant gap in the literature by demonstrating how to influence IPP implementation processes and outcomes.

There are many challenges involved in conducting implementation research in real-world settings, and a pragmatic approach is warranted to make best use of multiple and varied data sources. It is important to recognise that injury reductions will only occur when implementation activities, such as those that increase awareness leading to actual performance of the intervention, are achieved successful. The vast majority of previous studies related to exercise training IPPs in sport have not considered whether intervention target groups actually implemented the interventions as intended.33 This is problematic because this information is needed to link injury reductions to the implemented intervention. Moreover, how an individual responds to implementation activities is significantly influenced by a range of factors such as how the intervention is implemented, who implements it and the broader ecological system in which it is implemented. Only one other study—an evaluation of an exercise training programme in junior soccer players exposed to coach-led programme delivery with or without additional physiotherapist involvement20—has specifically compared programme implementation strategies. However, that evaluation was part of a highly controlled RCT and the measure of implementation activity was limited to adherence (defined as the proportion of all possible sessions where the 11+ was delivered, the number of team 11+ sessions/week and the mean number of team 11+ exercises/session).

The RE-AIM Sports Setting Matrix was developed to guide the design and evaluation of sports IPPs, especially those delivered through community sport.8 It achieves this through its extension of the RE-AIM framework34 developed to incorporate the different ecological levels of the sports delivery system. The evaluation presented in this paper used the RE-AIM Sports Setting Matrix to evaluate multiple aspects of the implementation of an exercise training intervention at the team, club and regional league levels, in relation to different levels of implementation support. Overall, the mean total RE-AIM score was significantly higher for clubs in the FootyFirst+S regions than for clubs in FootyFirst+NS region, indicating that the implementation support had a major impact on FootyFirst implementation activity. Moreover, on every RE-AIM dimension, the clubs in the FootyFirst+S regions scored significantly higher than the clubs in FootyFirst+NS region.

The reach dimension score measured the awareness of FootyFirst among the intended target groups. This was the highest scoring RE-AIM dimension, approaching maximum scores for clubs in the FootyFirst+S regions. The reach dimension score was still high among clubs in the FootyFirst+NS region (although lower than for clubs in the FootyFirst+S regions) and the highest for any RE-AIM dimension in the FootyFirst+NS region clubs. This is not surprising as all clubs in all regions were provided with the FootyFirst resources, so awareness should have been high.

In this evaluation, the original RE-AIM effectiveness dimension34 was modified to a measure of perceived effectiveness because, according to the Diffusion of Innovations theory,35 the rate of translation of an innovation, in contrast to the public health impact, depends more on the end-users’ subjective perception of its effectiveness than it does on objective evidence of an innovation’s efficacy. There was a significantly higher score for the perceived effectiveness dimension for clubs in the FootyFirst+S regions than for clubs in the FootyFirst+NS region. This probably reflects the additional educational activities provided to the former, as part of the implementation plan. The fact that FootyFirst was perceived to be more effective when accompanied by the implementation plan is also demonstrated by lower injury counts in regions where clubs were exposed to FootyFirst+SF (Finch et al unpublished data, 2017).

The RE-AIM adoption dimension is best understood as a measure of the extent to which the intended target groups used FootyFirst. Among clubs in all regions, the adoption dimension scored second highest, and the mean scores were significantly higher for clubs in the FootyFirst+S regions than those in the FootyFirst+NS region. This suggests that clubs had difficulty transitioning from awareness of, to use of, FootyFirst without implementation support.

The lowest scoring RE-AIM dimension was implementation, though the score for this dimension was still significantly higher for clubs in the FootyFirst+S regions than for clubs in the FootyFirst+NS region. The implementation dimension is a measure of the extent to which the intended target groups used FootyFirst properly. We have previously shown that the fidelity of FootyFirst implementation was low in a sample of clubs because it was not delivered as intended.36 The low implementation scores among clubs in the FootyFirst+S regions may be due to the high standard set to achieve the implementation dimension aim, with all players required to complete the correct number of all exercises, at least twice a week, and advance through appropriate progressions. It is also possible that the implementation support provided was insufficient to ensure improvements in the implementation activities measured for this dimension.

The last RE-AIM dimension, maintenance, was a measure of coach and club intentions to continue using FootyFirst. The mean maintenance score was significantly higher in clubs in the FootyFirst+S regions than for clubs in the FootyFirst+NS region. It is possible that the implementation support provided, and higher adoption and implementation scores among clubs in the FootyFirst+S regions, demonstrated to coaches and clubs in those regions that FootyFirst was feasible and beneficial, and this was reflected in opinions about its ongoing value.

This evaluation demonstrates that the additional implementation support provided to clubs in the FootyFirst+S regions facilitated higher levels of implementation activity among those clubs compared with clubs in the FootyFirst+NS region. However, the perceived effectiveness, implementation and maintenance mean scores for clubs in FootyFirst+S regions were all between 0 and 1 (ie, located between ‘Partially Achieved’ and ‘Not Achieved’), and the 95% CI for the total RE-AIM mean scores for clubs in FootyFirst+S regions included the midpoint value of 5.00. This may reflect the difficulty in gathering data related to these specific RE-AIM dimensions. Alternatively, the implementation support provided may not have been sufficient to facilitate high levels of programme implementation and maintenance among the targeted clubs. It is beyond the scope of this paper to report on the facilitators and barriers to FootyFirst implementation. Data related to these issues were gathered in the postseason surveys and interviews and will be reported in a subsequent publication. It is anticipated that barriers to implementing FootyFirst that emerge from this study will be similar to the barriers to the adoption and use of LLI prevention exercise programmes in other team ball sports including the: nature of the IPP, competency and confidence of coaches, attitudes and beliefs of coaches and players, support from external sources and the time and resources available to implement IPPs at training.33 37 There are some limitations to the approach adopted in this study. First, there was no single objective data source available to measure the achievement of the aims of the RE-AIM dimensions. Therefore, a pragmatic evaluation approach was adopted that relied on a convergent design, merging quantitative and qualitative data from a range of sources for an overall interpretation. However, the reliability of the coding approach developed for this study was assessed (see online supplementary file 3), and there were very high levels of agreement across independent assessors suggesting robustness in the findings.

Second, this evaluation relied on the availability and completeness of data from the multiple data sources. Evidence supporting the level of achievement of the aims of the five RE-AIM dimensions was available from the full gamut of sources—such as surveys, interviews, direct observations, notes from the research team and participation in implementation activities—for clubs in the FootyFirst+S regions. In contrast, the evidence for clubs in the FootyFirst+NS region came predominantly from active participation in FootyFirst-related communication (such as emails and telephone calls), being provided with FootyFirst resources (mainly by post), research assistant notes and recall of engagement with clubs and completion of FootyFirst surveys. In the FootyFirst+NS region, only one club participated in a postseason interview, and none participated in a direct observation or provided weekly implementation data. Therefore, the evidence to demonstrate the achievement of the aims of each of the RE-AIM dimensions for clubs in the FootyFirst+NS region was drawn from limited sources and may be less accurate than that for clubs in the FootyFirst+S regions. Finally, clubs in the FootyFirst+NS region did not have the opportunity to participate in FootyFirst-related passive communication activities (Twitter and autotext messaging) or attend FootyFirst-related events because these activities were directly related to the implementation strategies used in the FootyFirst+S regions.

In summary, this study evaluated the implementation of an exercise training programme in terms of all RE-AIM dimensions and the desired implementation activities underpinning them. By comparing these dimensions across two regions exposed to the same IPP content but with different implementation support, it demonstrates that successful implementation of an IPP requires more than just a good programme. The fact that both total and all dimension-specific RE-AIM scores were significantly higher for clubs in the FootyFirst+S regions, than for clubs in the FootyFirst+NS region, supports our unpublished findings that injury reductions were also greater in the former, as assessed in a controlled ecological evaluation. Together, this gives us some confidence in concluding that exercise training programmes will only be effective when the investment in designing and delivering evidence-based and context-specific implementation plans matches the investment currently made in developing the evidence-based programme content.