Schools have an important role to play in child injury prevention. School based educational programs can increase cycle helmet wearing,¹ improve road crossing² and cycling behaviour³ and the prevention of poisoning.⁴ School-based injury prevention programs aimed at preventing a range of injuries including the “Lifeskills” program,⁵ the injury minimisation program for schools (IMPS) in the UK⁶ and the “Risk Watch” program in the USA and Canada^7,8,9,10 have all demonstrated improvements in knowledge, attitudes or self-reported behaviours and several have demonstrated improvements in observed life support⁶ or safety skills.⁵

The UK Fire and Rescue Service is developing its role in fire prevention, including working with schools to promote fire safety within the curriculum.¹¹ As part of this the Risk Watch Injury Prevention program has been piloted in some UK schools.¹² Risk Watch is an educational injury prevention program, developed in the USA and adapted for use in the UK by Nottinghamshire Fire and Rescue Service. It aims to help children and families create safer homes and communities by teaching the knowledge and skills they need to make positive choices about their personal safety and well-being.⁹ It employs a variety of educational approaches including practicing making safer choices, resisting peer pressure to take risks and influencing family members and others to take action to reduce risks. Opportunities exist for a variety of other organisations and agencies to reinforce classroom lessons with additional information and a real world perspective.

Although some positive findings have emerged from Risk Watch evaluations in the USA and Canada, these may not be generaliseable in to the UK. As yet no randomised controlled trials have been published evaluating Risk Watch in the UK. We therefore report findings from such a trial amongst primary school children in Nottingham, UK.

METHODS

Participants

All 88 state-funded primary (ages 5–11) or junior (ages 8–11) schools in Nottingham City were invited to teach the Risk Watch program and those agreeing (n = 58) were invited to take part in the trial. As we were unable to fulfil our sample size requirements from Nottingham City schools, the first two Nottinghamshire County schools who agreed to teach Risk Watch were also invited to participate in the trial. In total 60 schools were invited to participate in the trial. Children in one class per participating school, of years 3 (age 7–8), 4 (age 8–9) or 5 (age 9–10) whose parents did not withhold consent and who completed a baseline questionnaire, were eligible to participate.

Intervention

Teachers in participating schools delivered the Risk Watch intervention. They were trained to deliver the intervention by Fire Service personnel or by other teachers who had been trained by Fire Service personnel. The Fire and Rescue Service provided free teaching resources including Risk Watch folders and “Risky Boxes”. Folders were targeted at specific age groups, with one folder for years 3 and 4 and one for year 5 and included background information, lesson plans and activities for pupils (table 1). Each folder covered eight topic areas, four of which were chosen for the evaluation; bike and pedestrian safety, falls, poisoning, fire and burns. These topics were chosen because they result in the most frequent and severe injuries in the UK for primary school children.^13–15 Risky Boxes included resources and materials for use during lessons (see http://www.notts-fire.gov.uk/Internet/NottsFR/nottinghamshirefireandrescue.nsf/HomeFrame?OpenFrameSet for more details). Each participating school agreed to teach at least one Risk Watch topic of their choice from the four chosen for the evaluation. The control group schools agreed to delay delivering the Risk Watch program until follow up assessments had been made.

Objective

To evaluate the effectiveness of the Risk Watch program in increasing safety knowledge and skills, and self-reported safety behaviour.

Outcomes

The primary outcome measure was safety knowledge. Secondary outcome measures included self-reported safety behaviour, observed safety skills and teachers’ views of, and satisfaction with teaching the Risk Watch program.

Children’s knowledge and self-reported behaviour were measured using age-appropriate pencil and paper questionnaires completed in the classroom at baseline and follow-up. These questionnaires also recorded age, gender and whether the family had access to a car, the latter being an indicator included in the Townsend deprivation score¹⁶ which is commonly used in health research in the UK. We considered this a suitable indicator of socio-economic status as children aged seven should be able to provide such information. Knowledge questions were illustrated pictorially and required ticking boxes or circling hazards. Researchers read questions aloud to the class to enable children with poorer literacy skills to complete questionnaires. Separate questionnaires were used for children in years 3 and 4 and year 5 as the objectives of Risk Watch varied by age.

The questionnaire was initially piloted on 52 children from one school. After making amendments, it was re piloted on a further 48 children from two schools. In total it was piloted on 42 children aged 7–8 years (48% boys) and 58 aged 9–10 years (45% boys). Two of the schools had a lower percentage and one had a similar percentage of children receiving free school meals to the national average.

Safety skills were assessed at follow-up in a random sample of eight children from each school by observation of role-play in three age-appropriate injury scenarios. Children in years 3 and 4 were required to demonstrate the “Stop, drop and roll” procedure, crossing a road safely and what to do in the event of a poisoning. Children in year 5 were required to demonstrate what to do in a domestic fire, crossing a road safely, and putting on a cycle helmet correctly. Each skill comprised 4–5 elements with marks awarded for each element correctly demonstrated (maximum 5 marks per scenario). One child was assessed at a time, jointly by two observers who also took part in the role play. The skills assessment tool was piloted on eight children from two pilot schools.

At the end of the study a postal questionnaire was sent to all teachers of intervention group classes. The questionnaire asked about which Risk Watch topics were taught, teachers’ views of, and satisfaction with teaching Risk Watch, usefulness of the materials and training, use of activities involving pupils’ parents as part of Risk Watch and involvement of local organisations with an injury prevention role.

Sample size

Based on data from the first pilot, the mean percentages of questions answered correctly were 40% (SD 20%) for years 3 and 4 and 59% (SD 20%) for year 5. A sample size of 226 (113 in each group) would allow an absolute difference of 7.5% of questions answered correctly to be detected between treatment groups (80% power and a 2-sided 5% significance level). The design effect was estimated as 2.2 based on a mean class size of 25 and an ICC of 0.05. The required sample size therefore totalled 498, with 249 children (equivalent to 10 schools) per group.

Assignment

Schools were stratified by tertiles of the percentage of children receiving free school meals, and randomly allocated within strata to treatment group. One member of the research team (DK) generated the computerised allocation schedule and a researcher independent of the trial, allocated schools to treatment group.

Masking

It was not possible to blind participants or teachers to treatment group allocation. Researchers doing outcome assessments were blinded to treatment group allocation, but maintaining blinding was not always possible as teachers frequently mentioned teaching Risk Watch at follow up assessments. Data entry was undertaken blind to allocation.

Ethical approval

An ethical review of the trial was provided by Lincolnshire local research ethics committee.

Statistical analyses

Data were entered into an Access database. All 25 questionnaires completed in one school were double entered by a second researcher and discrepancies identified. The error rate was 0.4% of data field entries. Scores were created which represented the percentage of correct responses for safety knowledge for each topic area and the percentage of correctly demonstrated safety skills, for each topic area and for all safety skills.

Balance of characteristics between treatment groups was assessed informally. Random effects models were used to adjust for clustering by school. Linear regression was used to estimate differences between the means for knowledge scores. Logistic regression was used to estimate odds ratios for binary knowledge or safety practices questions and for skewed scores which were categorised as all questions or skills correct versus fewer. All models were adjusted for stratum, school level baseline values, child age, gender and car ownership. Linear regression was also used to estimate differences between the means for safety skills with models adjusted for stratum. As some knowledge questions were asked only of children in some year groups resulting in a small number of schools providing data, these were summarised at school level and means were compared using t tests weighted by the number of participants from each school.¹⁷ Pre-specified sub-group analyses examining whether the treatment effect varied by age, gender or car ownership were undertaken by adding interaction terms to models.

Participants were analysed by treatment group regardless of whether they received the intervention, so those who were absent during Risk Watch teaching were included in the analysis. As schools were free to choose which topics they taught, they were only included in the analysis for the topics they had chosen to teach. Models were checked by examining plots of residual values. Data were analysed using Stata version 8.

RESULTS

Participant flow and follow up

The flow of schools and participants through the trial is shown in fig 1. Twenty two schools were recruited, two of which withdrew after randomisation but prior to completing baseline questionnaires. In total 20 schools and 459 children participated in the trial. Baseline assessments were undertaken between 15/11/04 and 28/04/05 and follow up assessments between 14/04/05 and 28/06/05. The median time between assessments was 126 days (IQR 106, 140) in the intervention group and 120 days (IQR 79, 131) in the control group.

Analysis

Ninety percent of participating schools enrolled children aged 3–11 years, all were mixed sex schools and the median percentage of children receiving free school meals was 29.5% (IQR 21.5%, 41.0%.) Characteristics of participants and self-reported safety behaviours are shown in table 2 and safety knowledge in table 3. There was some evidence that intervention group children were younger and were more likely to come from families without access to a car than control group children, otherwise treatment groups were well balanced at baseline.

At follow up intervention group children correctly answered a significantly higher percentage of fire and burn prevention questions than control group children. No significant difference was found for knowledge of poisoning, falls prevention or bike and pedestrian safety (table 4). Analyses of specific knowledge questions indicated that intervention group children were significantly more likely to know the actions to take if clothes catch fire and the correct way to wear a cycle helmet than control group children (difference between school means 35.3% (95% CI 22.7% to 47.9%) and 6.3% (95% CI 1.4% to 11.1%) respectively). They were also significantly more likely to know the actions to take in a house fire and on finding tablets than control group children (OR 2.80 (95% CI 1.08 to 7.22) and OR 3.50 (95% CI 1.18 to 10.38) respectively)

Risk Watch was more effective in increasing fire and burn prevention knowledge amongst younger than older children (difference between means age 7 = 19.5% (95% CI 7.3% to 31.7%); age 8 = 9.3% (95% CI 1.9% to 16.7%); age 9 = 5.4% (95% CI −1.1% to 12.0%); age 10 = −3.8% (95% CI −13.3% to 5.6%); p = 0.04). It was also more effective in increasing bike and pedestrian safety knowledge amongst boys than girls (difference between means boys  = 5.4% (95% CI −0.2% to 11.0%); girls  = −3.6% (95% CI −9.3% to 2.0%), p = 0.003). There were no significant differences for other scores by age, sex or car ownership.

There was little evidence of an impact on self-reported safety behaviours. Intervention group children were only significantly more likely to report never playing with matches (OR 1.84 (95% CI 1.06 to 3.20), p = 0.03). In terms of safety skills (table 5), intervention group children were significantly more likely to correctly demonstrate all procedures to follow if clothes catch fire and they demonstrated a significantly higher percentage of correct actions overall than control group children.

The eleven classes in which Risk Watch was taught had a total of 12 teachers, 11 (92%) of whom completed the follow up questionnaire. The majority (8, 73%) of teachers used the background information in the Risk Watch folders and told the children about the safety rules (10, 91%). Most teachers modified the lesson plans (8, 80%). Fewer than half the teachers used materials from the Risky boxes (4, 36%), used outside organisations to support the teaching (5, 46%), gave children activities to undertake with their parents at home (3, 27%) or organised activities outside school premises (1, 9%).

DISCUSSION

Principal findings

The first year of delivery of the Risk Watch program in primary schools was effective in increasing some aspects of children’s knowledge of fire and burn prevention, poisoning prevention and of bike and pedestrian safety. The program was more effective in increasing fire and burn prevention knowledge in younger than in older children and in increasing bike and pedestrian safety knowledge in boys than in girls. It was also effective in increasing children’s safety skills in terms of fire and burn prevention. There was little evidence to suggest the first year of the program had an impact on self-reported safety behaviours.

Strengths and weaknesses of the trial

The strengths of our trial include recruiting the required number of schools and achieving high follow up rates in both treatment groups. Despite small numbers included in the skills assessment we were able to demonstrate that at least for fire and burn prevention, increases in safety knowledge did translate into improved safety skills.

Our trial evaluated the first year of teaching the Risk Watch program, an incremental program aiming each year to build upon knowledge and skills gained through the program in previous years. The effect demonstrated after the first year may therefore underestimate the program’s full impact. Furthermore, the intraclass correlation coefficients for our primary outcome measures were larger and the mean class size was smaller than those used in our sample size calculations; so some negative findings may reflect insufficient power. In addition two important elements of the program, namely involvement of parents and of other organisations or agencies, may require a longer period of time to achieve. This may partly explain the lack of impact on children’s safety behaviours, many of which would require parental involvement to change.

The majority of schools were in disadvantaged areas, so it may not be possible to apply our results to schools in more affluent areas. Furthermore our findings may also not be applicable to low or middle income countries.

Comparisons with previous research

There have been several evaluations in the US and in Canada of the Risk Watch program. A non-randomised controlled study compared changes in safety knowledge in children from schools using Risk Watch and comparison schools.^7,8 Increases from baseline in knowledge scores across all topic areas for Grades 3–4 and 5–6 were between 3–8% points for Risk Watch schools and between 0–5% points for comparison schools, but the statistical significance of these gains was not reported. A cluster randomised controlled trial in 12 Canadian elementary schools assessed the impact of one year of the Risk Watch curriculum.¹⁰ Preliminary findings indicate that Risk Watch was not effective in reducing medically attended injuries or near misses or in changing observed safety behaviours. Significant increases in knowledge were demonstrated, but these equated to only one additional correct response amongst intervention group children compared to control group children.

Our findings are somewhat more positive than these, possibly reflecting our use of differing assessment tools. Unlike previous evaluations, we chose not to use the questionnaires provided by the Risk Watch program as we considered most children would answer most questions correctly at baseline, so limiting the potential for demonstrating increases in knowledge. We did not measure the impact of Risk Watch on injury occurrence as we considered it unrealistic to expect the first year of an incremental safety education program to demonstrate injury reductions and this would also require a substantially larger sample size.

There has been one other published evaluation of a school based program aimed at preventing a range of injuries using a controlled, but non randomised design.⁶ The IMPS program teaches 10–11 year olds about risk and the consequences of taking risks. It has been found to be effective in increasing knowledge of calling 999, first aid for burns and choking, basic life support skills and identification of a range of dangers and hazardous actions.⁶ These are consistent with our findings, that a safety education program delivered to primary school children is effective in improving at least some aspects of safety knowledge and skills.

Implications for injury prevention practice and further research

We have demonstrated that the first year of an educational program delivered by teachers in primary schools is effective in increasing some aspects of children’s safety knowledge and skills. Primary schools should therefore consider delivering the Risk Watch program as part of their safety education curriculum. Evaluations with a longer implementation and follow up period and on a larger scale are required to examine duration of effect, the effect of building upon previous years of delivering the Risk Watch program and impact on child injury rates. Further research is required to examine potential explanations for the greater effect amongst particular groups of children.

It is important to remember that safety education can only ever form part of a comprehensive injury prevention strategy. Evidence from systematic reviews demonstrates that engineering and legislative approaches are more effective in reducing injury morbidity and mortality than educational approaches, and that a combination of approaches may be the most beneficial.^18,19 In view of this, policy makers should ensure that injury prevention strategies do not rely solely on educational approaches.