METHODS

In 1998, New Mexico’s Department of Health and Traffic Safety Bureau commissioned a statewide restraint use observation study (RUOS). The purpose of the study was to obtain restraint use information on individuals in five age categories: infants (less than 1 year), toddlers (1–4 years), youth (5–12 years), teens (13–18 years), and adults (over 18 years). Six observers spent three months collecting data on 4893 occupants in 2013 vehicles in New Mexico. The six observers were recruited from a university community based on job experience requiring accuracy and attention to detail. They ranged in age from 29 to 48 years (mean age 33.8 years; SD 7.3). Although none of the observers were parents, all of them had previous experience caring for children and teens. The number of years of experience ranged from three to 33 (mean 14, median 7).

Each observer completed an eight hour training program that covered the rationale and protocol for conducting the survey, types of occupant restraints, age estimation, and state restraint laws. Training methods included classroom discussion, a photo identification exercise, and on-site practice. The photo exercise used 30 color photographs of 49 people who ranged from 3 months to adults, were of different racial/ethnic groups, and appeared in both sitting and standing positions (but not seated in vehicles). The observers were given 10 minutes to examine the photos and assign age categories. The categories were the same as those in the statewide RUOS data collection forms. Actual ages were then revealed to the observers, who “graded” their own estimates. The overall correct score for the six observers was 82.5% (range 71%–90%, SD 7.3). On-site practice involved restraint use observations in the field. Teams were formed consisting of two observers who shared the roles of “caller” and “writer”. Data elements on the observation form included vehicle type, seating position, gender of driver, occupant age category, and type of restraint.

After the field session and photo exercise, trainers and observers discussed issues encountered in estimating age and observing restraint use. For example, observers were alerted to not let car safety seat positioning (forward or rear facing) influence their estimation of whether a child was an infant or toddler, since car seats are often used incorrectly.^22,23 The observers were encouraged to rely on a constellation of factors (such as head shape, clothing, and age specific behaviors) to improve their accuracy in estimating occupants’ ages.

Two months after completing the RUOS, the six observers participated in the validation study to assess the accuracy of their age assignments. A validator joined the two person observation teams at the drive through area of three fast food restaurants. A sign informed patrons of a “child passenger safety” study. Observers were positioned near the order box. The validator was positioned between the order box and the pay window. Only vehicles with at least one child occupant (birth to 18 years old) were eligible to participate. Observations were conducted while vehicle occupants were placing their food orders. The observer called out the ages and seating positions to the writer, as in the RUOS study. When the vehicle moved forward to the pay window, the validator approached the driver, explained the purpose of the study, offered a discount coupon for participation, and asked the driver and occupants to report their ages. For matching purposes, the license plate number was noted on the data collection form by both observers and validator.

Analysis of proportions by χ² was used to determine the accuracy of the assignment of age categories. STATS VIEW 5 statistical software was used for the data analysis.²⁷

RESULTS

A total of 173 motor vehicles containing 449 occupants were observed during the validation study (table 1). Of the occupants, 391 (87.1%) were assigned to their correct age category (percent correct assignment = number assigned correctly divided by the total number of occupants in that age group, as reported by the driver). There was no statistically significant difference in the accuracy of age assignments by the age category of the occupants (p=0.83). The overall accuracy of age assignments by the three teams was 86%, 88%, and 89% (p=0.73).

Table 2 shows the observed frequencies of the 58 incorrect age assignments by the direction of error. The direction of error is necessarily unilateral for both infants and adults: adults could not be assigned to an older age category, infants could not be assigned to a younger one. Infants were somewhat more likely than individuals in other age categories to be estimated incorrectly: of nine infants, two (22.2%) were assigned to the toddler category. Errors in both over and under estimation of age categories occurred for the intermediate age groups (toddler, school age, and teens). Not surprisingly, many (53%) of the inaccurate assignments involved individuals whose ages were close to “break points”. For example, there were seven 4 year old children who were assigned to the 5–12 year old category and three 5 year old children assigned to the toddler (1–4 year old) category. Eight adults who were 19 or 20 years old were classified as teens (13–18 year old).

DISCUSSION AND RECOMMENDATIONS

Overall, our observer teams were very accurate in assigning age categories: 87% of the 449 vehicle occupants were assigned correctly. Possible factors for success were the observers’ innate ability to categorize ages, their experience working with children, the age estimation training, and three months of experience observing occupants during the RUOS. Being part of a team may have also enhanced the accuracy of age assignments by providing “second opinions” regarding occupant ages. That 83% of the age category assignments were correct in the photo exercise suggests that several of the observers were already very skilled in estimating ages before they received any specific training or experience in observation. This is consistent with results from experimental psychology, where untrained subjects often do well in age estimation tasks.^23,28,29

The high proportion (53%) of incorrect age assignments occurring at “break points” between age categories prompts a cautionary note for studies where specific ages, rather than age categories, are of critical importance. For example, observational studies of booster seats target 4–9 year olds^30,31; studies of rear versus front seat position focus on children under 12 or 13 years of age^32–34; and studies of graduated licensing target teenage drivers.^35–37 Interviewing drivers to obtain the correct ages of vehicle occupants is not a perfect solution. For example, the driver may not be the parent and therefore not report the correct age of a child occupant.

When selection bias is not a critical barrier, the accuracy of age assignments can be improved by using times and locations that attract specific age populations. For example, studies of teen seat belt use have been conducted at high school parking lots, infants can be observed at Women, Infants and Children’s Program sites, and toddler restraint use can be noted among children at day care programs.

The small number of infants (nine) observed during the validation study limits any conclusions regarding this important age group. Designers of observational surveys should consider over sampling infants and toddlers, since these groups were only 4% and 12%, respectively, of the occupant population in our statewide survey. That two of the nine infants were assigned to the toddler age category suggests that age assignments may be particularly challenging for older infants (10–12 months of age).

Another limitation of our study is that the fast food drive-through setting did not entirely duplicate field conditions. For example, observers were less subject to fatigue during the validation study because it required only two hours of observations, compared with six hours each day during the field survey.

Further age validation studies should determine how generalizable our results are to other observers, populations, and age categories. The independent impact of training on improving age assessment could be assessed by randomizing observers to groups with and without training and then validating their observations. The photo examination exercise is simple and inexpensive, and can identify observers who have particular difficulty in estimating ages. The exercise could be strengthened by having the children in the photographs seated in vehicles to simulate observation conditions. Discussion of relevant issues in child development (such as when infants can grasp objects or what ages young people begin to smoke cigarettes) and common mistakes in estimating ages (for example, assuming that a forward facing car seat guarantees its occupant is a toddler) should be included in the training, in addition to practice in the field. Car seat “clinics” (where NHTSA certified individuals check for car seat misuse) offer another valuable training opportunity.³⁸ At these clinics, observers can visit with many young children in different types of car safety seats and booster seats, and verify their ages with the parent or guardian.

It is likely that observers improve with experience. This hypothesis could be tested by conducting an age validation study before and after observers participate in a large scale restraint use survey. Finally, safety recommendations for appropriate vehicle restraint use are often based on the weight of the child, rather than, or in addition to, the child’s age.³⁹ The reliability of weight estimates of child occupants has never, to our knowledge, been investigated.

We recommend that NHTSA, the Centers for Disease Control and Prevention, state and local agencies, and other sponsors of observational surveys consider observer competence as an important variable. The validity of age assignments can be assessed by randomly interviewing a sample of drivers to attain the true age of the occupants. More accurate age estimates will improve decisions regarding prevention programs, funding, and policies.