There is an urgent need for evaluative work surrounding the prevention of pediatric injuries on farms. Coordinated efforts to prevent childhood farm injuries have increased in the past five to 10 years and have led to the implementation of farm safety interventions across the United States and Canada.¹ However, there have been few rigorous evaluations of these interventions^1–3 and the problem of pediatric farm injuries persists.^4–9

Many farm work related injuries occur because children are assigned work normally performed by adults, and much of this work is beyond their developmental abilities.^10,11 The need to consider child growth and development in the assignment of safe farm work to children led to the development of the North American Guidelines for Children’s Agricultural Tasks (NAGCAT).¹² First released in 1999, NAGCAT provide assessment tools that cover 62 different agricultural jobs common for children aged 7–16 years (see www.nagcat.org). Since their introduction, these guidelines have been utilized increasingly as an injury prevention resource.

When NAGCAT were developed, there was little empirical evidence that could be used to inform work guidelines for children in agriculture.¹² Therefore, a consensus development process was followed.¹³ To date, the results of the NAGCAT consensus process have not been challenged or tested empirically.

The overall goal of this research was to evaluate the NAGCAT resource and its potential for preventing injuries by systematically applying NAGCAT to case descriptions of fatal, hospitalized, and restricted activity injuries experienced by children. The evaluation focuses upon NAGCAT and the prevention of injuries to children engaged in farm work. Our objectives were to: (1) describe the characteristics of injured children and leading mechanisms of injury; (2) identify whether there were NAGCAT guidelines that could be applied to these cases; (3) describe specific farm jobs covered by NAGCAT that were commonly associated with these injuries; (4) identify which components of NAGCAT were most commonly violated; and (5) estimate the proportion of injuries that may have been prevented had NAGCAT been available and applied (efficacy).

METHODS

Data sources

In order to conduct this novel evaluation, three case series were assembled representing fatal, hospitalized, and restricted activity pediatric farm injuries. A pediatric farm injury was defined broadly as any injury to a child younger than 18 years of age that occurred during activities related to the operation of a farm and/or occurred at a farm work site (excluding injuries occurring in the farm house/residence). This definition recognizes that farms are typically composed of a residence adjacent to a worksite where farm children are consequently exposed to a range of occupational hazards. We were deliberate in examining the full spectrum of worksite related pediatric farm injuries and a collective examination of these three case series provided this comprehensive overview of important patterns of childhood injury.

Fatality data

There is no national database of childhood farm fatalities in the United States,¹⁴ so pediatric fatalities were initially identified for Canada using a process identical to that employed by the Canadian Agricultural Injury Surveillance Program’s national fatality⁷ and hospitalization⁶ registries. The written investigation reports of all fatalities to farm children in Canada from 1990–2001 were accessed on-site at provincial coroner’s offices to characterize each injury event by the person involved, external cause, nature of injury, location of injury, time of injury occurrence, and circumstances of injury. This Canadian fatality case series was supplemented with 17 case reports from the National Institute for Occupational Safety and Health (NIOSH) state Fatal Assessment and Control Evaluation program.¹⁵ These occupational fatality investigation reports from 1992–2000 were obtained from NIOSH and reviewed centrally.

Hospitalization data

Hospitalized pediatric injuries were identified via research agreements with five regional pediatric hospitals (three in Ontario, two in Alberta) and one general hospital (Alberta). Hospitalized farm injuries were defined as injury related hospital admissions of children where the International Classification of Diseases¹⁶ external cause of injury (E code) was coded on the hospital separation file as (1) E919.0, injuries caused by agricultural machinery, and/or (2) a location of injury E code indicating that the injury occurred on a farm. Following ethics board approval from each hospital, computerized and written medical records from 1989–2002 were reviewed on-site and detailed descriptions of the injury circumstances were collected.

Restricted activity injury data

Restricted activity injuries were identified from a national survey conducted in the United States by the National Agricultural Statistics Service. The survey was conducted in 1999 with 50 000 randomly selected farm households.¹⁷ Farm injuries were defined as those occurring on the farm to youth that resulted in at least four hours of restricted activity or required professional medical treatment. This definition allowed for inclusion of hospitalizations, but fatalities were specifically excluded.¹⁷ During the survey, detailed information was compiled about each injury that occurred in the calendar year of 1998, including details about each injury victim, the circumstances of injury, and the consequences of that injury. The data file with all personal identifiers removed was obtained from NIOSH upon approval from the National Agricultural Statistics Service.

Instrument

A standardized data abstraction instrument was developed. Existing research instruments¹⁸ and administrative questionnaires^17,19 were consulted during its construction. Other questions were developed specifically for this study based upon the NAGCAT resource.¹²

The data abstraction instrument was divided into three main categories: (1) case demographics, (2) the injury event, and (3) identification of the most relevant NAGCAT guideline. Case demographics included the age and gender of the child and whether the child lived on the farm or was hired for work purposes at the time of the injury. The injury event section addressed the activity that the child was involved in, circumstances surrounding that injury event, the mechanism of injury, the location of injury occurrence, the nature of injury, whether farm work was being performed, and the involvement of the child in that farm work.

The most relevant NAGCAT guideline section was developed using NAGCAT as a framework.¹² All 62 NAGCAT guidelines were listed and the guideline most applicable (if any) to the work activity that the child was engaged in at the time of injury was identified. Once identified, additional questions addressed the main categories of NAGCAT including: adult responsibilities, main hazards, safety reminders, training, and level of adult supervision. As it was not possible to directly assess the developmental capabilities of the injured child, a child development specialist, Dr Richard Clark, identified minimum age requirements for each NAGCAT guideline based upon the child development questions and the norms for child development as inferred from standards published by the Centers for Disease Control and Prevention (2000)²⁰ and contemporary child development texts.^21,22

A final summary question asked the rater to estimate: “if the NAGCAT guideline was applied, would this injury have been prevented?” Response categories were (1) yes, 100% certain, (2) no, 100% certain, and (3) maybe (along with an estimate of likelihood provided by the rater). This was used as a proxy for efficacy.

In cases where NAGCAT were not applicable to the farm work event, a narrative text field was provided to describe the reasons why NAGCAT could not be applied. Supplemental questions asked for suggestions for additional guidelines or modifications to existing guidelines.

The data abstraction form was validated through iterative testing using consecutive series of cases from each data source. A study glossary was developed to clarify individual questions and definitions to ensure consistency of responses, as well as to identify consensus decision points. Study investigators independently reviewed samples of 10–12 cases, the results were entered in separate data tables, and all discrepancies were identified and resolved. Iterative testing continued with expansion of the study glossary at each step until a high level of agreement was reached, and then the final data collection commenced. Satisfactory agreement could not be reached for the restricted activity case series, so two investigators (BM, WP) abstracted all cases jointly.

Data collection

The coinvestigators (BM, RJB, WP) collected the data at the 10 provincial coroner’s offices, hospitals in Alberta (3) and Ontario (3), and centrally from the NIOSH case investigations and restricted activity injury files. Cases that were difficult to classify (80/934; 8.6%) were identified during the data collection process and brought to the full research team for discussion. These were resolved by consensus, and precedents for each difficult case were recorded. Data quality checks were established to ensure that the data were collected and entered accurately.

Data analysis

Descriptive statistics were used to characterize both the full case series and the subset of cases where the child was engaged in farm work, by demographics and the nature of the work involved in the injury. All analyses were stratified by injury outcome (fatal, hospitalized, and restricted activity injury) and focused on the working child subset. NAGCAT guidelines associated with individual cases were classified and their relative frequencies were described. Violations to each of the main categories of NAGCAT (adult responsibilities, supervision, child development, training) were summarized. The potential efficacy of NAGCAT to prevent each injury was estimated. Finally, for cases where children were engaged in farm work but no NAGCAT guideline was applicable, the farm jobs were classified and described by frequency.

RESULTS

Full sample

Table 1 describes the 934 injured children in the full case series. The majority of injured children were male (76.4%), lived on the farm (73.2%), and were children of the farm owner (69.4%). All ages from 1 to 17 years were represented, but the largest percentages were children 1–6 years of age (33.5%). In 30.3% (283/934) of the cases, the child was engaged in farm work. The remainder of the paper will focus on this subset of cases.

Children engaged in farm work

Similar to the full sample, children engaged in farm work were largely male (81.3%), lived on the farm (73.5%), and were children of the farm owner (67.8%) (table 2). The largest percentages of working children, however, were in the older age groups (fig 1).

Mechanisms of injury

Leading mechanisms of injury were machinery entanglement (19.8%), being struck by an animal (10.6%), runovers (10.3%), and rollovers (9.9%), but varied by injury outcome (table 3). Rollovers (26.5%), runovers (22.0%), and machinery entanglements (13.2%) dominated the fatality cases. A large percentage of hospitalized injury cases involved machinery entanglements (41.5%). The leading mechanisms of restricted activity injury were being struck by an animal (16.5%) and cut/pierced by an object (15.6%).

Applicability of NAGCAT

There was an applicable NAGCAT guideline in 64.9% of cases (table 4). Eight children engaged in farm work were excluded because they were younger than the ages covered by NAGCAT, and 27 cases were excluded because there was insufficient information available to identify the farm job the child was engaged in at the time of injury. Overall, the leading categories of work described according to NAGCAT (table 5) were general activities (27.3%), animal care (26.7%), and tractor fundamentals (16.1%) (see www.nagcat.org for a full description). Leading individual guidelines were working with large animals (14.3%), driving a farm tractor (no implement attached) (9.3%), and farmwork with an all-terrain vehicle (8.7%), and these varied by the injury outcome as shown.

NAGCAT violations and potential efficacy

Violations to the adult responsibility component of NAGCAT were present in 48.7% of the cases where there was clear documentation to make a determination (table 6) and this varied by injury outcome. Child development violations were identified in 36.6% of cases and were present among children in all age ranges (data not shown).

In the judgment of the research team, 59.6% of the injuries with an applicable guideline were completely preventable if the principles espoused by NAGCAT had been applied (table 7). Preventability varied by injury outcome with more fatalities being judged to be completely preventable (79.5%) than restricted activity injuries (35.0%).

Among the 42 (26.1%) cases with an existing guideline that were judged to be not preventable (100% certain), no substantial deficiencies were identified in the guidelines. Injuries associated with these cases resulted from hazards inherent to the job at hand (for example, being kicked by a large animal; slipping on uneven terrain) or were caused by the victim performing the job in an unusual manner (for example, cleaning equipment with a pitch fork; herding animals on all-terrain vehicles).

Farm work not covered by NAGCAT

Eighty seven work related cases (35.1%) had no applicable NAGCAT guideline (table 8). Of these cases, leading work categories were maintaining farm machinery (13.8%), using a tractor as a stationary power source (11.5%), and traveling to the worksite (10.3%).

DISCUSSION

Synopsis of main findings

Injuries involving children engaged in farm work are an important component of the pediatric farm injury problem. Within this context, our findings show there was an applicable NAGCAT guideline in the majority of work related injury cases and NAGCAT, if applied, would prevent 70% to 80% of the most serious work related injuries. For the injury cases judged not preventable (100% certain), it was the opinion of the investigators that the only way these injuries could be fully prevented would be to remove the child from these activities entirely, which is counter to the intent of NAGCAT. This speaks well for the content and effort afforded to the NAGCAT resource. Despite these encouraging findings, NAGCAT were applicable in only 20% (161 of the 823 cases where there was sufficient information to make a determination) of the full case series of childhood farm injuries. In most cases, NAGCAT were not applicable (567 of the 823 cases) because the child was not involved in farm work.

The preventability estimate is based upon efficacy (hypothetical complete application of the content of NAGCAT) rather than effectiveness (“real world” application practices). While there is general acceptance by farmers of NAGCAT as a resource,¹⁸ only modest proportions of farm parents will apply NAGCAT in actual practice.^18,23 Other recent work²⁴ shows that while farm parents can be quite knowledgeable about children and their development, simply providing parents with copies of NAGCAT is insufficient to remove many children from known occupational hazards; hence the efficacy estimates shown here do not necessarily imply practical effectiveness.

Other evaluations of NAGCAT

The present study is one of several ongoing efforts to evaluate the NAGCAT resource. Gadomski and colleagues (2003) are conducting a randomized trial in New York State evaluating the impact of active dissemination of NAGCAT on the incidence of childhood agricultural injury.²⁵ Interim analyses showed a decrease in childhood injury rates in both the intervention (active dissemination) and control group (usual care). The between-group differences in rate declines approached, but did not achieve statistical significance. Two additional NAGCAT related projects are noteworthy. Shutske et al are combining qualitative and quantitative methods to adapt NAGCAT for ethnic minority groups.²⁶ Wright is using both qualitative and quantitative research methods to assess parents’ attitudes and practices related to NAGCAT.²⁷ Results from these studies are pending, and in light of this, it is still premature to draw definitive conclusions about the general efficacy of NAGCAT.

Strengths and limitations

This study involved a large evaluation of a well known prevention resource and fills a dire need for evaluative research.³ We believe this to be novel work because it is the first rigorous evaluation of NAGCAT. Further, the results provide a realistic indication of the potential efficacy of NAGCAT and indicate how this resource might be refined.

Our study investigated a large case series, and two of its data components were national and population based. While we examined multiple windows of surveillance with the aim of generalizability, we were restricted to available data sources. The hospitalized and fatal injury cases were limited to Canada (with the exception of the 17 cases from the Fatal Assessment and Control Evaluation program), as an analogous case series could not be assembled in the United States. Likewise, the data on restricted activity injuries were limited to the United States. Our data were based upon records collected for other purposes. Thus, details surrounding the descriptions of injury events varied by data source and resulted in some missing information that could impact our estimates of the efficacy of NAGCAT. Irrespective of these limitations, our study was a practical, efficient, and cost effective way to begin evaluating the content and potential efficacy of NAGCAT.

Implications for NAGCAT

Our evaluation identified NAGCAT guidelines that are associated with recurrent injury events. This could lead to efficiencies in the presentation of the NAGCAT resource. To illustrate, 10/62 individual guidelines accounted for 63% of the cases observed. Certain categories of guidelines (specialty production, manual labor) accounted for very few of the observed cases and are unlikely to have a major influence on farm injury rates except for focused production circumstances. Condensed versions of NAGCAT might include only those guidelines/guideline categories that are most frequently associated with injury.

In addition, the NAGCAT team should re-examine guidelines most commonly associated with recurring patterns of injury. The potential for injury prevention through refinements to NAGCAT would seem greatest by focusing first on these specific guidelines. Adult responsibilities may need to be emphasized or clarified, the job may require more advanced developmental abilities than originally proposed, the level of supervision may need to be increased, and/or the NAGCAT job may need to be completely and openly restricted for children.

Currently, NAGCAT fail to cover some important work activities that commonly lead to childhood injury. Some of these are within the purview of developmentally appropriate farm work (some machinery maintenance) and the NAGCAT consensus team might consider developing new guidelines to cover this work. Conversely, the NAGCAT team will need to consider openly restricting some high risk activities to the adult work force only, analogous to the Hazardous Occupations Order for Agriculture.²⁸ Examples include work in confined spaces, application of chemicals, and certain animal husbandry activities.

CONCLUSIONS

NAGCAT do have the potential to prevent a significant proportion of the most serious work related injuries experienced by children in the correct age range (7–16 years). However, NAGCAT were applicable in only 20% of the farm injury cases, mainly because the children were not working. If NAGCAT are to address the leading causes of pediatric farm injury, their scope must expand beyond working children. NAGCAT were not intended to be, and clearly are not, the sole solution to the pediatric injury problem on North American farms.