Methods

STUDY COMMUNITY

The study was conducted in the Borough of East York, in collaboration with the East York Health Unit, SAFE KIDS Canada, and the Ontario Cycling Association. East York is a borough of Metropolitan Toronto, with a population of 100 000 and a school age population of around 10 000. There are 22 elementary schools in East York. Six schools (selected because of their previous experience in research projects conducted by the East York Health Unit) participated in the study. Statistics Canada census data were used to define schools as high, middle, or low income based on the average family income in the census tract within which the school was located.^{22, 23} Using arbitrary cut off points, census tracts with average family incomes of $60 000, $40 000, and $32 000 (Canadian dollars) were considered high, middle, and low income, respectively. Only grade 4 children in the selected schools participated in the study.

INTERVENTION

The Kids CAN-BIKE Festival is a cycling safety skills course developed by the Canadian Cycling Association for children 8 to 13 years old.²⁴ This playground based course is taught by trained and certified instructors, with a student:instructor ratio of around 6:1. The festival includes six stations—two equipment stations (helmet/clothing check and bicycle check) along with four bicycle handling stations (straight line riding, shoulder checking, signaling, and stopping and starting). Each station takes around 15 minutes to complete, therefore, the festival runs for around 90 minutes. These four bicycle handling skills are emphasised because studies have suggested that weaving on the road, swerving into traffic without looking, not signaling, and cycling through stop signs are risk factors for serious injury.^{7, 8} At each station, the children are also taught about pertinent traffic regulations and appropriate safe cycling behaviors. For example, at the straight line riding station, the children are taught that a bicycle has the same right to the road as a car and that weaving on the road is a risky behavior. The objective of this study was to evaluate the effectiveness of the Kids CAN-BIKE Festival in improving bicycle handling skills, knowledge, and attitudes.

It should be noted that the Kids CAN-BIKE Festival is considered an introductory level course in bicycle safety instruction. On completion of the festival, the children are encouraged to enroll in the Canadian Cycling Association CycleRight program. This 10 hour course provides more detailed instruction on the fundamentals of safe bicycle operation through discussion, practice on the playground, and on-road training.

STUDY DESIGN

A randomized controlled trial design was used. The unit of randomization was the school, with random allocation of the intervention (the festival) to one school from each income area. Baseline assessments of safe cycling behavior, knowledge, and attitudes of grade 4 children in the intervention and control schools were conducted in June 1995. Children in the intervention schools participated in the Kids CAN-BIKE Festival in the afternoon after the baseline assessments. Follow up evaluation of both groups of children (intervention and control) took place in September 1995, after the school summer holidays. (To ensure equal opportunity for all children in the study, the control schools received the intervention after the follow up assessments in September.) All evaluations were conducted in the playgrounds of the participating schools.

OUTCOME MEASURES

The primary outcome measure was three components of safe cycling behavior: (a) straight line riding, (b) coming to a complete stop, and (c) shoulder checking before a left turn. These three behaviors (scored yes or no) were operationalised a priori. Bicycle handling skills were assessed on a 50 × 20 meter section of the playground. Each child was instructed to ride their bicycle along a 30 meter chalk line, stop at the stop sign, and turn left around a pylon. (The children were asked to imagine that they were riding along a main road.) Cycling behaviors were assessed by an independent observer who had no interaction with the children. Straight line riding required the child to remain within 10 cm of either side of the chalk line for the entire 30 meters (with an allowance of 2–3 meters grace on “take-off”). A complete stop required the child to slow down to a halt at the line and put at least one foot on the ground. Shoulder checking required the child to take his/her eyes off the road ahead and turn his/her head to look over the left shoulder.

The reliability of these outcome measures was determined by having two independent observers score 20 children of similar age during a pilot study. The level of inter-rater agreement was estimated using the κ coefficient.²⁵

Secondary outcomes included safe cycling knowledge and attitudes. Data on these outcomes were collected using a 13 item, self report questionnaire, pretested for readability and comprehensibility on the children in the pilot study. Demographic information included age, sex, bicycle ownership, helmet ownership, and helmet use. The remaining items addressed the children's attitudes towards, and knowledge of, traffic regulations: Do you think a bicycle is more like a toy or a car? Do you prefer to ride your bicycle on the sidewalk or on the road? When cycling on the road do you move to the curb between parked cars? Do you think that a car has more right to the road than a bicycle? Is it important for cyclists to signal before making a left turn? Is it OK for cyclists to only slow down at stop signs when there is no traffic? Do you need to wear a helmet if you are cycling on a bike path? Is it OK to ride a bicycle that is too big? Do you always wear a helmet when you are riding your bicycle?

ANALYSIS

Because of the cluster randomization design, a two sample t test was used to test for differences in behavior, knowledge, and attitudes between the intervention and control groups.²⁶ This test involves calculation of the “average” proportion or event rate for each group (by summing the cluster specific rates and dividing by the number of clusters). These “average” proportions for the two groups (experimental and control) are then compared under the t distribution with 2(m−1) degrees of freedom, where m is the number of clusters. This approach to hypothesis testing for differences in event rates or proportions in the setting of community intervention trials with randomization by cluster has been shown to be both appropriate and robust.²⁷

The time between baseline assessment and follow up evaluation was around three months. Therefore, to determine whether safe cycling behavior, knowledge, or attitudes changed significantly over time, relative risks for each outcome variable (follow up v baseline) were calculated. (A maximum likelihood test was used to determine if the school specific relative risks were homogeneous.) If so, a summary relative risk was calculated using the Mantel-Haenszel method, along with estimation of the 95% confidence interval around the point estimate.²⁸

SAMPLE SIZE

A priori, the prevalence of safe cycling behavior in the control group was estimated to be 40%. We wished to have 80% power, at an alpha level of 5%, to detect a twofold difference in the prevalence of safe cycling behavior between the intervention and control groups. For a trial with randomization by individual, a sample size of around 23 per group would be required to detect this effect size.²⁹ However, because randomization by cluster reduces efficiency, the sample size estimate was increased by a factor of 2.5 (to around 60 per group) based on Cornfield's formula.^{30, 31}

ETHICS

Written informed consent for participation in the trial was obtained from the parents. Ethical approval for the study was received from the Hospital for Sick Children Research Ethics Board.

Results

A total of 141 grade 4 children participated in the trial: 73 in the intervention group and 68 in the control group. (No parent refused permission for their child to participate.) Two schools contributed two grade 4 classes to the study, while the remaining schools each provided one. The average number of children per class was 18 (range 16–21 children), and all children were either 9 or 10 years old. The pilot study showed that the scoring system for measuring safe cycling behavior was reliable, with high interobserver agreement (κ coefficients ranging from 0.9–1.0). Because of a scheduling error, one school received the intervention in September rather than June. However, the intention-to-treat analysis (based on original group allocation) gave results that were essentially identical to those obtained from the efficacy analysis (based on actual group allocation). The following results pertain to the intention-to-treat analysis.

Of the 141 children with baseline measurements, 117 (83%) were available for follow up in September. The rates of loss to follow up were similar across the six schools (p=0.459). A baseline comparison of responders (n = 117) with non-responders (n = 24) showed no differences on age (50% v 64% aged 9 years, respectively; p=0.226), group allocation (52% v 50% allocated to the intervention group, respectively; p=0.849), bicycle ownership (99% v 96%, respectively; p=0.312), helmet ownership (97% v 100%, respectively; p=1.000), or helmet use (88% v 87%, respectively; p=0.813). Responders and non-responders were also similar on baseline safe cycling behaviors: straight line riding (60% v 50%, respectively; p=0.374), coming to a complete stop (88% v 83%, respectively; p=0.511), and shoulder checking before a left turn (1% v 0%, respectively; p=1.000). Non-responders, however, were more likely to be female compared with responders (70% v 43%, respectively; p=0.018).

Baseline demographic characteristics of the intervention and control groups are provided in table 1. There were no significant differences between the two groups on age, sex, bicycle ownership, helmet ownership, or helmet use. The frequency of safe cycling behaviors by the two groups (straight line riding, coming to a complete stop, and shoulder checking before a left turn) are described in table 2. Both at baseline and at follow up there were no significant differences between the two groups. In summary, more than half of all children were able to maintain straight line riding at baseline, with a significant improvement in this behavior (up to 90%) at follow up. Most children came to a complete stop at baseline (87%) and at follow up (83%), while only 1% of children shoulder checked before turning left, both at baseline and at follow up. As shown in table 3, both at baseline and at follow up, there were no significant differences however, between the intervention and control groups in safe cycling knowledge or attitudes.

Over time, that is, from baseline to follow up, the frequency of straight line riding in both groups of children significantly increased. This improvement over time was similar and consistent across schools—in other words, the school specific relative risks were homogeneous. As shown in table 4, children were 1.5 times more likely to maintain straight line riding at follow up compared with their performance at baseline. There was no evidence of a similar change over time for the other two safe cycling behaviors. With respect to changes in knowledge and attitudes over time, at follow up, children were less likely to ride their bicycles on the sidewalk and less likely to consider that a car had more right to the road than a bicycle.

Discussion

The results of this randomized controlled trial suggest that the Kids CAN-BIKE Festival—a stand alone, playground based bicycle skills training program—is not effective in improving safe cycling behavior, knowledge, or attitudes among grade 4 children. Study strengths included random allocation of the intervention to schools and relatively few children (17%) lost to follow up. (Although girls were more likely to be non-responders than boys, the 16 female non-responders were distributed equally between the intervention and control groups.) Information bias was minimized by using a standardized protocol to measure safe cycling behavior, with all behaviors operationalised a priori. Outcome measurement was shown to be reliable during pilot testing. Confounding bias was considered unlikely, given that random allocation resulted in the intervention and control groups being similar on all measured characteristics.

There are several reasons that might explain why the Kids CAN-BIKE Festival was not shown to be effective. First, the ability to detect a “clinically important” difference (statistical power) is an important issue in any null study. Post hoc power analyses showed that the sample size in this study was adequate to provide greater than 80% power to detect a 40% difference between the two groups in straight line riding and shoulder checking. However, because of the high baseline prevalence of coming to a complete stop (90% in both groups), this study had limited power to detect a difference between groups on this behavior. Another issue is whether the a priori effect size was optimistic, that is, the intervention effect may have been smaller or more subtle. The results, however, do not show any trend of a consistent (but not statistically significant) improvement in outcomes in the intervention group compared with the control group.

A related issue is whether the intervention was sufficiently potent to effect change. Both the quality and quantity of an intervention contribute to its potency.²⁹ The quality of the administration of the Kids CAN-BIKE Festival was good, given that instruction was provided by enthusiastic, certified volunteers, with a low student:instructor ratio at each school. The quantity of the intervention, however, may have been too little. In other words, two hours of playground based instruction may not be sufficient to teach young children the fundamentals of safe cycling behavior. A further issue was the three month delay before follow up evaluation. Because of this delay, it was not possible to determine whether the timing of improvement in straight line riding differed between the two groups, for example, whether the intervention group improved immediately, while the control group required practice and experience over the summer to achieve the same change.

Other possible reasons for the lack of an intervention effect include poor compliance, information sharing between the two groups, inadvertent teaching of the control group at baseline, and non-blind evaluation of safe cycling behavior. Although one school failed to receive the intervention as scheduled, the results of the efficacy analysis (based on actual group allocation) were similar to the intention-to-treat analysis. The efficacy analysis showed no differences between the intervention and control groups at follow up on straight line riding (89% v 89%, respectively; p=0.976), coming to a complete stop (85% v 82%, respectively; p=0.831), or shoulder checking before a left turn (0% v 1%, respectively; p=1.000). The possibility of information sharing between the two groups was considered unlikely, given that the schools involved were separated by distances of several miles. In addition, most young children ride their bicycles close to home.³² To minimize inadvertent teaching of the control group at baseline and measurement error at follow up, standardized methods for the delivery and evaluation of the intervention were developed a priori. For example, every child was given the same instruction on how to complete the course, with an independent observer used to assess bicycle handling skills. Because of funding limitations, all observations were made by a research nurse who was not blind to group allocation. The prior experience of this nurse was in public health and health promotion. Therefore, it could be argued that any bias in measurement (because of unblinding) would have been towards finding an effect, rather than towards the null. Measurement of safe cycling behavior was also shown to be reliable during pilot testing.

Maturation of bicycle handling skills through practice and experience over the summer holidays is the most plausible explanation for the improvement in straight line riding in both groups of children at follow up. The same reason also likely explains the increased level of confidence in both groups (as measured by their attitudes) towards on-road cycling, that is, less likely to ride on the sidewalk and less likely to consider the car as having more right to the road. Neither experience nor the intervention, however, were sufficient to improve shoulder checking before left turns. In other words, the children did not appear to be capable of mastering this bicycle handling skill. The setting for this study was not “real life” in that training and evaluation were playground based. Therefore, the children were asked to imagine that they were riding their bicycle on a main road, approaching a stop sign, and preparing to turn left after the sign. Although it could be argued that the artificial setting was the reason why so few children shoulder checked before turning left, against this argument is the fact that most children came to a complete stop at the “stop sign”. Anecdotally, many of the children expressed anxiety about shoulder checking, largely because of a fear of taking their eyes off the road ahead. Last, because the children were not distracted by traffic or pedestrians and because they were eager to please their adult instructors, the safe cycling behaviors exhibited in this study may, in fact, represent optimal behaviors.

To our knowledge, there is only one published study that has formally evaluated a bicycle skills training program for young children.³³ In this study, 34 children in one grade 5 class were randomly assigned to one of two interventions—classroom and playground instruction on safe cycling behavior, or playground instruction only. A grade 5 class (n = 15) in another primary school served as the control group. Outcome measures included slowing down at an intersection, shoulder checking, signaling, and compliance with right-of-way traffic rules.

At follow up testing at eight weeks, safe cycling behavior was improved in both intervention groups compared with the control group. However, there were no differences between the three groups in the prevalence of compliance with right-of-way traffic rules.

This study had several limitations. First, because children in the same class received the different interventions, there was the potential for information sharing between the two groups. Second, although the study used a cluster design, the unit of analysis was the individual. Therefore, the results are likely to be liberal rather than conservative. Last, the summary scores for safe cycling behavior were not intuitive. The authors compared the “average proportion of behavior elements correctly carried out” between the three groups. However, the 35 behavior elements were not described in detail. Therefore, it was difficult to interpret the outcome data.

For children to ride safely in traffic requires that they are knowledgeable about traffic rules, can read and interpret signs, and have the necessary cognitive and motor skills.³² Studies have shown that school age children lack relevant traffic safety knowledge.^{18, 34} In addition, a survey of students in grades 4 through 8 showed no correlation between knowledge of traffic rules and the prevalence of bicycle related injury.³⁵ The evaluative study by van Schagen and Brookhuis showed that education did not lead to rule based behavior. In other words, despite increased safety knowledge, the cycling behavior of the children was unpredictable, defensive, and dependent on the behavior of other traffic.³³ Descriptive case series of bicycle injuries in young children suggest that failure to straight line ride, come to a complete stop, and shoulder check before a left turn may be risky behaviors.^{7, 8} The results of our trial, however, suggest that brief, one time, educational interventions are not effective in improving these behaviors.

Implications for prevention

In general, prevention strategies based on educational programs must be coupled with attention to health beliefs, barriers to the intervention, and the ability of the individual to perform the task.³⁶ Interventions that repeat the message in different forms and contexts are also more likely to succeed. Therefore, community based education programs that allow for repetition of bicycle safety messages, several opportunities for practice, and parental involvement, may represent a more effective approach to improving bicycle safety in children. It is also possible that young children (under 10 years) may not be able to master the basic cognitive and motor skills necessary for the complex task of riding a bicycle on the road.

In summary, bicycle skills training programs are common in schools and communities, but there are few published data on their effectiveness.²¹ The evidence to date suggests that brief educational interventions are not effective in improving safe cycling behavior in young children. As a final comment, our study evaluated intermediate outcomes—behavior, knowledge, and attitudes.³⁷ Population based studies are needed to determine whether bicycle skills training programs reduce the frequency and severity of bicycle related injuries in children.