Methods

For the purpose of this study, the surveillance population of interest was injured children presenting to a hospital emergency department. At the time of the study, the Ottawa-Carleton region of Ontario, Canada was served by a single paediatric tertiary facility (the Children's Hospital of Eastern Ontario) and five general hospitals with emergency department facilities: Montfort Hospital, Civic Hospital, General Hospital, Queensway-Carleton Hospital, and Riverside Hospital. (The Children's Hospital of Eastern Ontario was the only participant in CHIRPP.) Using intercensal estimates, the 1992 Ottawa-Carleton population was 682 000 persons, of whom 25% were children (0–18 years).

Data on injured children presenting to emergency departments in the region were collected over 30 days. Because of limited resources, simple random sampling, using a random numbers table, was used to select 10 days each month for three consecutive months (June through August, 1992). Injured children were identified through review of the emergency department registration logs and the medical charts at each hospital. (Registration logs are computerized files that contain clinical and demographic information on each patient presenting to the emergency department. This information is extracted from the emergency medical record by clerical staff.)

The numerator for CHIRPP sensitivity was the number of injured children attending the Children's Hospital of Eastern Ontario (and thus eligible for capture by the surveillance system), whereas, the denominator was the total number of emergency department attended childhood injuries in the region. Injured children transferred from a general hospital to the Children's Hospital of Eastern Ontario were included in the numerator.

The representativeness of CHIRPP data was assessed by comparing injured children presenting to general hospitals (and thus missed by the system) with injured children presenting to the Children's Hospital of Eastern Ontario. Demographic, social, and clinical information were collected on both groups of injuries (missed and captured), using the medical chart as the primary data source. Predictor variables included the child's age and sex, day and time of presentation, delay before presentation, nature of injury, injury intent, and disposition. The univariate relationship between each of these independent variables and being missed by CHIRPP (yes/no) was examined, with stepwise logistic regression analysis used to assess the relative importance of the predictor variables.⁹

Sample size calculation was based on an α error of 0.05, power of 0.80, a 10% prevalence of the predictor variable among captured injuries, and the ability to detect a 1.5-fold increased risk of being missed by CHIRPP. Based on an anticipated missed:captured ratio of 1:2, 648 missed injuries and 1296 captured injuries were required.¹⁰ Given a frequency of around 50 injuries per day presenting to the emergency department of the Children's Hospital of Eastern Ontario, missed and captured injuries were accrued over 30 randomly selected days during the three summer months.

Results

Over the 30 days, 2386 children in the Ottawa-Carleton region presented to an emergency department because of injury. Of these, 1552 (65%) presented to the Children's Hospital of Eastern Ontario (including 18 transfers). The remaining 834 injured children sought treatment at a general hospital. Of the five general hospitals in the region, however, injured children presented to only three: the Queensway-Carleton Hospital (631 children), the Civic Hospital (107 children), and the Montfort Hospital (96 children). This pattern of utilization of general hospitals is likely explained by the close proximity of the General Hospital and the Riverside Hospital to the Children's Hospital of Eastern Ontario; both general hospitals are within several hundred yards of the paediatric facility. CHIRPP sensitivity was, therefore, 65% (1552/2386); 95% confidence interval (CI) 63% to 67%.

The representativeness study compared the 834 missed injuries (that is, children presenting to general hospitals) with the 1552 injuries captured by CHIRPP (that is, children presenting to the Children's Hospital of Eastern Ontario). Univariate analyses showed no differences between missed and captured injuries on sex, day, time of presentation, or injury intent (see table 1). The median delay before presentation to the emergency department was one hour for both missed and captured injuries.

Nature of injury was coded using the 26 categories on the CHIRPP form. Analysis by this variable showed that “minor” injuries, for example, lacerations, abrasions, and sprains, were more likely to be missed by CHIRPP (see table 1). In contrast, more “severe” injuries, for example, fractures, dislocations, and concussions were less likely to be missed. (For concussions, the association was not statistically significant, largely because this injury type was relatively rare.) Analysis by disposition confirmed this observation. In other words, injuries resulting in admission to hospital were more likely to be captured by CHIRPP (that is, attend the Children's Hospital of Eastern Ontario), compared with injuries treated and released from the emergency department. In summary, minor injuries were more likely to be treated at a general hospital, whereas, severe injuries were more likely to be managed at the tertiary children's hospital. Because of the correlation between disposition and nature of injury, only disposition was retained in the logistic regression analysis.

The median age of missed injuries was 12 years, compared with a median age of eight years for captured injuries. Further graphical analysis by risk deciles showed a clear binary relationship between age and capture. Because of this relationship, age was modeled as a binary variable (>14 years compared with ≤14 years). As shown in table 1, older children were more likely to be missed by CHIRPP, compared with younger children. Of note, examination of the distribution of missed and captured injuries by nature of injury and by age showed consistent results. In other words, the pattern whereby specific minor injuries (for example, lacerations, abrasions, and sprains) were more likely to be missed by CHIRPP, while severe injuries (for example, fractures, dislocations, and concussions) were less likely to be missed was consistent for the two age groups (>14 years and ≤14 years).

The main effects logistic regression model included only disposition and age. The adjusted odds ratio of being missed by CHIRPP for admitted injuries (compared with those treated and released) was 0.43 (95% CI 0.23 to 0.80), whereas, the adjusted odds ratio of being missed by CHIRPP for children older than 14 years (compared with children 14 years or younger) was 3.52 (95% CI 2.87 to 4.32). There was no interaction between disposition and age.

Discussion

This study showed that 35% of injured children in the Ottawa-Carleton region attended a general hospital and, therefore, were missed by CHIRPP. Comparison of missed with captured injuries also showed systematic errors in capture by the surveillance system; older children were more likely to be missed, whereas, severely injured children (that is, those admitted to hospital) were less likely to be missed.

Sampling bias was considered unlikely, given that simple random sampling was used to select the days for review. Information bias was minimized by using a standardized data collection form and the same data source (the medical chart) for both groups of injuries. Logistic regression analysis was used to adjust for the effects of confounding variables.

Calculation of CHIRPP sensitivity was based on the assumption of complete capture of childhood injuries presenting to the Children's Hospital of Eastern Ontario. Previous research, however, has shown that capture at specific CHIRPP centers varies from 30% to 90%.¹¹ Therefore, based on a capture rate of 30% at the Children's Hospital of Eastern Ontario, CHIRPP sensitivity would decline to 20% (466/2386); 95% CI 18% to 22%. A capture rate of 90% would give a CHIRPP sensitivity of 59% (1397/2386); 95% CI 57% to 61%.

Inappropriate inclusion of non-injuries in the CHIRPP database has been shown to be negligible, with system specificity estimated at 100%.¹¹ Further, in our study, 129 children (5%) lived outside the Ottawa-Carleton region, with the majority attending the Children's Hospital of Eastern Ontario. Exclusion of these children from the analyses, however, did not materially alter the results.

The study was conducted over three summer months. Therefore, the influence of season on CHIRPP sensitivity and representativeness could not be assessed. Further, given that the research was conducted in a single region, cautious generalization of the findings is warranted. At the time of the study, however, the 10 CHIRPP centers across Canada were considered relatively homogeneous; all were urban, teaching, tertiary care paediatric centers affiliated with a local medical school. Further, it could be argued that the systematic errors in CHIRPP capture (that is, related to age and injury severity) speak to a pattern of health care utilization by families that is generalizable to most urban centers and not unique to the Ottawa-Carleton region.

Children older than 14 years were more likely to attend a general hospital and, therefore, be missed by CHIRPP. Possible explanations may be that the Children's Hospital of Eastern Ontario is perceived by the population as providing care to young children (0–14 years), or that adolescents themselves prefer a general hospital. Children admitted to hospital because of an injury, however, were more likely to attend the Children's Hospital of Eastern Ontario, that is, be captured by the surveillance system. It is likely that both parents and physicians choose that severely injured children be managed in the regional tertiary paediatric facility. Another explanation, however, is that admission policies for injured children may differ between general hospitals and children's hospitals.

Evaluation of the quality of injury surveillance data is necessary if prevention activities based on the data are to be appropriate and effective.⁷ CHIRPP collects national data on childhood injuries and, therefore, represents an important resource for research and policy development. Sensitivity is low, however, with almost one third of all emergency department attended injuries missed by the surveillance system. Therefore, any estimation of the national burden of childhood injury presenting to emergency departments, based on CHIRPP data alone, would require adjustment upwards.

Furthermore, there are systematic errors in CHIRPP capture. For example, adolescents are systematically missed by the surveillance system. This error limits the use of CHIRPP data to determine priorities, identify populations at risk, and evaluate control programs for injuries that predominantly affect this age group. In such situations, supplementary data from general hospitals may be necessary. In summary, although CHIRPP data may be useful for the identification of emerging problems and for hypotheses generation, given the systematic errors in capture, these data should be used cautiously in studies of etiology.