Article Text

Download PDFPDF

Evaluation of the ThinkFirst Canada, Smart Hockey, brain and spinal cord injury prevention video
  1. D J Cook1,
  2. M D Cusimano1,
  3. C H Tator2,
  4. M L Chipman3
  1. 1Injury Prevention Research Office, St Michael’s Hospital, Division of Neurosurgery, Toronto and ThinkFirst Foundation of Canada, Toronto Western Hospital, Toronto, Ontario
  2. 2ThinkFirst Foundation of Canada, Toronto Western Hospital, 399 Bathurst Street, Toronto, Ontario, M5T 2S8
  3. 3Department of Public Health Sciences, University of Toronto
  1. Correspondence to:
 Dr Michael Cusimano
 St Michael’s Hospital, Injury Prevention Research Office, Division of Neurosurgery, 38 Shuter Street, Suite 2-018, Toronto, Ontario M5B 1A6, Canada; injurypreventionsmh.toronto.on.ca
  1. C Macarthur, Senior Scientist4
  1. 4Population Health Sciences, Hospital for Sick Children Research Institute, Toronto, Canada; colin.macarthursickkids.ca

    Abstract

    Objective: The ThinkFirst Canada Smart Hockey program is an educational injury prevention video that teaches the mechanisms, consequences, and prevention of brain and spinal cord injury in ice hockey. This study evaluates knowledge transfer and behavioural outcomes in 11–12 year old hockey players who viewed the video.

    Design: Randomized controlled design.

    Setting: Greater Toronto Minor Hockey League, Toronto Ontario.

    Subjects: Minor, competitive 11–12 year old male ice hockey players and hockey team coaches.

    Interventions: The Smart Hockey video was shown to experimental teams at mid-season. An interview was conducted with coaches to understand reasons to accept or refuse the injury prevention video.

    Main outcome measures: A test of concussion knowledge was administered before, immediately after, and three months after exposure to the video. The incidence of aggressive penalties was measured before and after viewing the video.

    Results: The number of causes and mechanisms of concussion named by players increased from 1.13 to 2.47 and from 0.67 to 1.22 respectively. This effect was maintained at three months. There was no significant change in control teams. There was no significant change in total penalties after video exposure; however, specific body checking related penalties were significantly reduced in the experimental group.

    Conclusion: This study showed some improvements in knowledge and behaviours after a single viewing of a video; however, these findings require confirmation with a larger sample to understand the sociobehavioural aspects of sport that determine the effectiveness and acceptance of injury prevention interventions.

    • brain injury
    • spinal cord injury
    • child health education
    • hockey
    • sport injury

    Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Participation in ice hockey has significant inherent risk for injury. Several studies have shown that head and neck injuries are the second most common injuries in hockey, after musculoskeletal injuries.1–3 Of special concern is the issue of concussion. Goodman et al calculated an incidence of 5.95 and 4.63 concussions per 1000 player hours in games during the 1998–99 and 1999–2000 seasons, respectively, in 15–20 year old males.4 Concussions are most commonly caused by body contact, both legal and illegal, followed by players falling onto the ice and striking the goal post or the boards.4,5 Concussion has significant short and long term consequences and can occasionally be fatal.6–8 Players with concussion can have neuropsychological deficits that may be permanent.9 Other long term symptoms may form a constellation called post-concussive syndrome.10 With over 560 000 youth playing ice hockey in countries like Canada (2001 Hockey Canada statistics, unpublished), and with increasing numbers in other northern hemisphere countries, the potential for long term deficits after concussion is significant among sport related public health issues. Thus, there is a need to develop programs to prevent head injury among those participating in hockey. Body contact, violence, and lack of respect among hockey players are associated with injury.11 Therefore, education and minimization of anger and violence in hockey are required to prevent injuries.

    In Canada, several injury prevention initiatives have been implemented in hockey with varying results. Mandatory helmet and face mask use is an example of passive prevention that has been in effect in Canadian Minor Hockey since the late 1970s. This rule has decreased the incidence of skull fractures and has resulted in a dramatic reduction in facial injuries, particularly catastrophic eye injuries.12–14 Additionally, recent research has demonstrated a decrease in the severity of concussion with the use of full face masks.15,16 Tator and Edmonds identified checking or pushing from behind as a major cause of spinal cord injuries in hockey in 1984,17 which led to the implementation of a major penalty for checking from behind by Hockey Canada beginning in the 1989–90 season. Subsequently, a study by Watson et al demonstrated a significant decrease of 6.16 to 4.49 head and neck injuries per 1000 university level competitive games after the introduction of this penalty.18 However, the incidence of head and neck injury remains high, and thus, additional interventions are required to decrease injuries further.3,19

    Education has been recommended as an injury prevention intervention in hockey.5,20,21 Several educational safety programs have been created for hockey including, Smart Hockey with Mike Bossy (ThinkFirst Foundation of Canada, 1988), Heads Up: Head Injury and Trauma In Sport (Sport Medicine Council of Alberta, 1999), Heads Up, Don’t Duck (Massachusetts Medical Society Department of Public Health and Education, 2001), and Heads Up Hockey (USA Hockey). None of these programs has undergone systematic evaluation, and conclusions regarding the effect on injury cannot definitively be made. Recently, ThinkFirst of Canada developed a concussion and spinal cord injury prevention video targeted at hockey players 10 years of age and older. The purpose of this study was to evaluate the knowledge transfer and decay and effect on penalties in 11 year old male hockey players achieved through a single viewing of this video.

    METHODS

    Institutional human subjects approval was obtained through the University of Toronto General Ethics Review Board and informed consent was obtained from all participants and parents.

    Video development

    A team of neurosurgeons, sports medicine specialists, team therapists, coaches, trainers, psychologists, physicians, league officials, representatives of hockey organizations, and hockey professionals revised the Smart Hockey with Mike Bossy video to add a major component on concussions. This updated version is called Smart Hockey. The new video was produced as a one hour national network television program, which was then distributed publicly as individual copies through hockey organizations. A written component was also developed for distribution with the video, but this was not evaluated in the present study because we wanted to observe the effects of the video in the context of children watching a television broadcast, which would not include the written information. Moreover, at the time we evaluated the video, the written component was not in its final edited form (for more information and complete versions of the video and brochure please contact ThinkFirst Canada through contact information given for CHT at end of paper).

    The video included medical information, training lessons, and personal statements delivered by recognized professional players, team physicians, coaches, referees, and trainers. Information was presented using several strategies, including reading and listing factual information in the video, having key concepts repeated in personal statements from recognizable stakeholders, and demonstration of key points in video clips from games and in a simulated coaching session with a prominent professional hockey player.

    Sampling

    The 34 Atom “A” competitive level teams in the Greater Toronto Hockey League (GTHL) (2001–02 season, age 11 at the beginning of the season) were numbered and randomly assigned to experimental and control treatment groups using a random number table. This group of players was selected as they were beginning their first season of body contact hockey and were being trained for full contact play, making the video of immediate relevance to them. A letter outlining the project requirements was sent to all coaches in the league. Up to three subsequent phone calls were made to coaches who did not respond to the initial letter. Coaches who refused to show the video to their teams were asked for an interview and independent screening of the video and were requested to describe their reasons for refusal, attitudes towards educational materials in hockey, and general feedback on the quality of the video. Coaches of teams who agreed to participate were also interviewed and asked to view the video and provide similar feedback. Control team coaches viewed and rated the video at the end of the regular season. Team placement in league standings was compared at the end of the season by averaging accumulated season points (2 for a game win, 1 for a game tie, and 0 for a loss) in each group to identify differences in the level of play among the teams.

    Outcome measures

    (1) Player questions

    The knowledge transfer study consisted of two questions, “How does a player get a concussion?” and “How does a player feel when he has a concussion?” Players were asked to list their answers which represented the mechanisms or symptoms of concussion, respectively (table 1). The sum of correct responses for each question and number of players scoring zero versus more than one correct answer were used to measure knowledge change in the symptoms and mechanisms categories. The experimental group initially received these questions at a mid-season practice, then at the same practice, viewed the Smart Hockey video, and were retested with the same set of questions five minutes after the video. Control groups were initially asked the questions at a mid-season practice, without viewing the video. Both groups answered the same questions three months after the initial test without additional viewing of the video.

    Table 1

    Results of knowledge transfer survey questions

    For these two questions, four comparisons, two across groups and two within groups were made: (i) experimental compared to control at baseline before any exposure, (ii) experimental compared to control three months after the baseline, (iii) baseline compared to immediately after the video within the experimental group, and (iv) baseline scores compared to three month scores within the experimental group. Bonferroni correction for four comparisons requires p = 0.01274 for each individual test to yield p = 0.05.

    (2) Penalty analysis

    The total number of penalties received during the entire season for both groups were collected at the end of the season. Penalties were recorded by type for each team over the 34 game regular season (Hockey Canada rules are used in the GTHL, rule definitions are available online at http://www.canadianhockey.ca). The mean number of penalties per 1000 player hours was compared in control and experimental groups before and after the initial survey with a t test to assess change in total penalties and pre-/post- change in individual penalty by type.

    (3) Qualitative analysis

    Coach interviews followed a structured methodology, with questions focused on the reasons for participating in or refusing the study, their prior experience with educational materials for injury prevention, their goals for teams and players in the current hockey season, their experiences with concussed players, and their views on aggressive behaviour in hockey. The follow up written survey repeated these questions and asked for further details on their perception of the video after viewing it. Content analysis of all coach surveys and interviews was completed using the grounded theory approach. Transcribed interview and written survey data were reviewed twice by a single blinded observer and common statements in the responses were identified. A third review of the data was completed to tabulate results.

    RESULTS

    The control group consisted of two teams including a total of 30 boys (mean (SD) age 11 (0.27) years), the experimental group consisted of three teams with a total of 45 boys (mean (SD) age 11 (0.25) years). One control and one experimental team responded to the initial letter and one control and two experimental teams were recruited through phone conversation. Seven additional coaches refused the request to participate, but agreed to an interview and independent viewing of the video with evaluation. Twenty two of 34 coaches did not participate in the study, with seven refusing interview and the remainder not responding to a letter or phone messages.

    There was no significant difference in total regular season points between control and experimental groups. Four of the teams reached play-off play: one control team and one experimental team were eliminated in the quarter final, while two met in the semifinal.

    Results for the test questions about concussion mechanism and symptoms are presented in tables 2 and 3. The results are summarized as total number of correct answers per player in each group, and then broken down by number of specific answers per group. The groups were equivalent in baseline knowledge levels. The experimental group showed significant improvements in knowledge after exposure to the video (as measured by the average of total correct answers per player, p<0.05, and by the number of individuals getting 0 answers pre-exposure compared with post-exposure), and these improvements were maintained at three months. In contrast, the control group did not show any significant change in knowledge.

    Table 2

    Answers to question “What does a person feel like when he gets a concussion?”

    Table 3

    Answers to question “How does a person get a concussion?”

    There were no significant changes from pre-video to post-video levels in the total number of penalties per game between groups. However, in the control group, interference penalties significantly decreased from mean (SEM) 50.0 (5.0) to 28.4 (5.7) per 1000 player hours and holding penalties significantly increased from 12.5 (2.5) to 22.7 (0.0) penalties per 1000 player hours. In the experimental group, a significant pre-video/post-video change (p<0.05) was observed in cross checking and checking from behind from mean (SEM) 23.7 (1.3) to 13.0 (3.4) and 38.4 (3.7) to 7.6 (0.69) penalties per 1000 player hours, respectively. Penalties recorded for each group, listed by type, are summarized in table 4 (see Hockey Canada Rule Book Glossary at http://www.canadianhockey.ca/e/develop/officials/glossary.html for detailed definitions of penalties).

    Table 4

    Summary of all penalties in control and experimental groups before and after intervention. Penalties are recorded as mean (SEM) number of penalties per group per 1000 player hours

    The qualitative analysis of interview and survey data with coaches is summarized with the most frequent responses in tables 5 and 6. Additionally, it was observed that four of five coaches agreeing to participate as compared with one of seven coaches refusing to participate had discussed the decision with other team administrators. Suggestions for improving the video included making the video shorter in length, reviewing the safe-play points at the conclusion of the video, including more professional players with injuries, and add appearances by players perceived as aggressive in the National Hockey League. No coach wanted to eliminate body contact from league play. Education, equipment design, and increased penalties for illegal plays were identified by coaches in both groups as potential injury prevention initiatives. Coaches in both groups indicated that the most influential impact on player behaviour in Atom “A” hockey was the head coach. There was consensus that changing coaching behaviours would lead to the greatest impact on player behaviour.

    Table 5

    Summary of qualitative analysis of coach responses in interviews and surveys: common concerns with showing safety videos

    Table 6

    Summary of qualitative analysis of coach responses in interviews and surveys: common reasons for supporting showing safety videos

    DISCUSSION

    There are several important findings in this study. Firstly, exposure to a well designed educational video featuring role models that many children in North America aspire to can significantly improve knowledge of concussions, and this knowledge does not decay over three months. It is of special interest to note that this information was presented using lists, displayed and read aloud in the video by current well known professional players describing their experience with concussion. The video was designed and studied in the context of previous injury prevention videos. For example, ThinkFirst has previously presented video based curriculum as a part of the ThinkFirst for Kids and ThinkFirst for Teens programs (for more information see http://www.thinkfirst.ca). This video is a part of a best practice injury prevention strategy as appraised by Volpe et al.24

    The lack of knowledge decay seen in our study at three months was not expected. One possibility for this observation was continued emphasis of video information by the coaches of the teams in the treatment group. However, all experimental team coaches denied overt efforts to reinforce information presented in the video at the time they were interviewed before the second survey. Another explanation might be parental reinforcement of video messages; however, parents were not specifically interviewed in this study. Observation of coaching and parenting practices after video presentation may elucidate changes that result in long term maintenance of knowledge.

    Although there was no change in total penalties, two types of high risk penalties (cross checking, defined as an illegal contact between two players in which the penalized player purposefully strikes the victim with the hockey stick held with both hands in a horizontal position and checking from behind, defined as any intentional body contact from behind) showed a declining trend in the group of young players who viewed the video. Checking from behind was one of the main injurious behaviours emphasized in the video.

    These results are encouraging since checking from behind has been shown to be the major cause of catastrophic spinal injury in hockey.17 In the video, these penalties are identified as high risk manoeuvres. Of special interest is the lack of change in hooking penalties (defined as an intentional use of the stick in a hooking fashion to pull and impede another player) in the experimental group, as this is also a potentially high risk penalty that was not addressed in the video. Whether these results will hold true with larger samples and over different age groups and levels of play are subjects of future work. Assessment of knowledge and behavioural changes after repeated viewings of the video will also provide valuable information on reinforcement of injury prevention messages. It is possible that the intervention would have been even more effective if coaches and players had been instructed to read and review together the written injury prevention booklet that accompanies the Smart Hockey program. Subsequent distribution of individual copies of the video has included the written information.

    Key points

    • Ice hockey is a popular recreational sport that carries significant risk for brain injury.

    • The ThinkFirst Smart Hockey educational video designed for hockey players increased awareness of hockey related brain and spinal cord injury.

    • Aggression, measured in penalty minutes, was not altered in young hockey players who watched the video.

    • Hockey coaches' negative attitudes and behaviours were a significant barrier to implementing injury prevention programming to youth and their parents.

    • Coaches who participated in the prevention strategy often identified a personal history of concussion or knowledge of a player having sustained a concussion.

    Another interesting but not surprising finding was that coaches who refused team participation perceive body contact as a critical factor in winning games. It is unfortunate that some coaches appear to place players at increased risk in order to try to win games. This “win at all costs” attitude should be discouraged and coaches should be mandated by the league to stress injury prevention in their practice routine.3–5 Balanced coaching strategies in the context of the many skills required to play hockey will hopefully reduce this risk. Negative attitudes of some coaches represent potential barriers to educational interventions such as the video. Coaches who agreed to show the video often identified a personal history of concussion or a significant, recent team history of concussion. These experiences gave coaches a special knowledge of the importance of concussive brain injury and made them likely to accept this. This may have biased our study since some of the effects seen may have been from continual coach reinforcement rather than the video.

    This study is limited in a number of other respects with regards to its conclusions. The sample size was limited in size and scope. Teams were randomized before consent rather than after consent. Players were surveyed at the mid-points and end points in the season. Competitive play and aggression may change during the season with an increase during elimination play for teams attempting to make play-off rounds at season end.

    There was potential in the design for a Hawthorne effect. That is, player behaviour may have changed simply by being surveyed or exposed to researchers. Hence, a third group to control for the Hawthorne effect by following penalty minutes and not administering surveys could strengthen this study. There may be bias between referee penalty calls and rule enforcement that limited the consistency of penalty data between games. Finally, we did not attempt to measure and had no control over exposure to factors which may have influenced our outcomes, such as exposures to other strategies such as body checking clinics.

    The present study suggests that the ThinkFirst of Canada Smart Hockey video has potential to improve knowledge about concussion among young competitive male hockey players. However, the basic principles of injury prevention suggest that additional steps are required, including development/modification and enforcement of rules, a reorientation of attitudes in the game, player selection criteria, coach selection criteria, referee selection and training, parent education, and administrator education. Future studies with a larger number of players and consideration of the above limitations are planned.

    Acknowledgments

    The authors would like to gratefully acknowledge the support of the Ontario Neurotrauma Foundation. The results and conclusions are those of the authors.

    REFERENCES

    COMMENTARY

    CRITIQUE AND RECOMMENDATION (EVALUATION OF EVALUATIONS)

    Cook et al describe the development and evaluation of an educational video to prevent concussion and spinal cord injury among 11 year old competitive ice hockey players. Five Atom “A” teams in Toronto were randomly assigned to experimental (n = 3) or control (n = 2) groups. The intervention consisted of a one hour educational video, which reinforced key concepts via stakeholder statements, video clips from games, and simulated coaching sessions. Teams in the experimental arm were shown the video once mid-season. Key outcomes included knowledge of the aetiology and symptoms of concussion as well as analysis of penalty infractions. Outcomes were measured in both groups at baseline (mid-season), immediately after the video in the experimental group, and at three months in both experimental and control groups.

    CRITIQUE

    Three methodologic issues—sampling bias, choice of analytic method, and “clinical significance”—limit the findings of the study. First, only 5/34 eligible teams (15%) participated in the study. Of the 29 coaches who refused (14 explicitly, 15 implicitly) only seven (24%) were interviewed. Furthermore, teams were randomized before consent. Therefore, if the few teams that agreed to participate did so based on their knowledge of group allocation (intervention or control), such sampling bias would distort the treatment effect estimate. In addition, because the participation rate was so low, the treatment effect identified is unlikely generalizable to the population of all eligible hockey teams. Second, the authors used a cluster randomized design to allocate hockey teams to intervention or control groups, but performed statistical analyses at the level of the individual subject. The effect of cluster randomization is to increase the size of standard errors (variability within and between clusters must be taken into account) and increase p values, compared with a study of the same size using simple randomization.1 Analyses at the level of the individual subject rather than at the level of the experimental unit (in this case the hockey team) may lead to spurious positive findings. Last, while the results were statistically significant, the “clinical significance” of the findings was less impressive. For example, the increase in knowledge in the intervention group translated into an average of 1.3 more symptoms of concussion identified (from 1.1 to 2.4 out of eight possible symptoms) and an average of 0.55 more causes of concussion identified (from 0.7 to 1.2 out of three possible causes).

    RECOMMENDATIONS

    Conducting a community based cluster randomized trial of an injury prevention initiative is not easy, and the authors should be applauded for taking a systematic approach to the evaluation of the Smart Hockey educational video. Issues arising from their evaluation, however, should be used to optimize future studies. For example, the quantitative and qualitative data suggest that the video presentation may benefit from educational theory input as well as input from end users (hockey coaches). Consideration of strategies to increase participation in injury prevention trials by competitive sports teams is also required. The outcomes data could also be used to estimate the relationship between the variance components (within and between clusters). This information is needed for sample size calculations that take into account the cluster randomized design. Last, as in all evaluations, the link between knowledge, behaviour, and injury rates needs to be defined. Ideally, all injury prevention evaluations ought to have injury rates as the primary outcome.

    REFERENCES