Unintentional injuries are the leading cause of morbidity and mortality among children aged 1–18 years of age and up to 44% of injuries occur in the home.^1–3 Researchers have used many approaches to improve home safety practices including community campaigns and practice based interventions.^4–7 Studies evaluating the effectiveness of behavioral interventions to improve safety practices typically rely on self reported surveys to assess outcomes. However, social desirability could increase the reporting of targeted behaviors and bias estimates in favor of interventions.⁸ For example, observational studies of seat belt use reveal that it is overreported.^8,9,10 Moreover, it has been observed that rates of reporting safety practices differ between responders and non-responders to a safety survey.¹¹ These factors could all obscure the effects of safety intervention trials.

There are few studies that validate self report of home safety behaviors for pediatric populations and the results have not been consistent. One study found working smoke alarms were overreported by 22%.¹² Another study found that, out of 16 safety practices measured and validated, five of them were overreported.⁶ Yet, another found that disagreement between the questionnaire and the observation were the result underreporting the safe practice,⁵ while a recent report found 90% to 100% agreement across six safety practices (burns, poisoning, drowning, cuts, strangulation/suffocation/chocking, and falls) to prevent injuries to children.¹³ It should be noted that for the Morongiello study interviews were performed at the home and therefore the families may have been more inclined to report accurately in the expectation that their responses would be checked.

In the context of an ongoing randomized clinical trial to improve the provision of preventive care, we conducted a validation study to determine the reliability of selected self reported home safety practices.

METHODS

This was a cross sectional study conducted within the context of an ongoing prospective randomized trial. The University of Washington Institutional Review Board approved the study protocol. The study sample was drawn between April 2004 and February 2005. Telephone interviews after the intervention were conducted between May 2004 and March 2005.

Sample

Our sample was randomly drawn from parents of children 12 years and younger whose child had made at least one visit to a University of Washington Physician Network clinic between 2000 and 2003. Each clinic is part of a physician-run organization of eight primary care clinics located throughout King County, Washington, and affiliated with the University of Washington Medical Center. The clinics are located in diverse environments with respect to patient race, age, and socioeconomic status. These primary care clinics are staffed by both family physicians and pediatricians and are not resident teaching clinics.

Procedures

Parents were surveyed by phone 2–3 weeks after their child had made a well child visit and were asked about a variety of safety practices (table 1). We invited all families, who had children at least 1 year old and who had completed the follow up interview, to participate in a home visit. Recruitment did not depend on intervention or control assignment. We completed home visits for everyone who agreed and who was reached to schedule a visit. Families were first sent a letter thanking them for participation in the larger study and then inviting them to participate in a home visit. They were not told that the purpose of the visit was a validation of their self report. The letter was followed with a phonecall to schedule a visit. The home visitor was not aware of the phone interview responses; visits lasted no more than 15 minutes. The home visit recruitment began 25 May 2004 and ended 30 March 2005.

Safety practices assessed

The larger intervention trial identified 12 home safety practices that were age specific. For example, sleeping on the back to decrease the risk of sudden infant death is relevant only in infancy. In an effort to maximize the number of people to whom the practices applied we chose four safety practices: water heater temperature settings, smoke detectors, bike helmet use, and car seat use. We restricted home visits to families where the study child was over 1 year old. The home visit entailed a checklist of six items confirming the questions asked during the telephone interview (table 1). During the home observation the temperature of the kitchen tap water was measured and recorded after running hot water for 5 minutes. The observer checked all smoke detectors and noted whether one was present on each floor and whether the tester button was functioning. Observers recorded the presence of a child’s bike helmet and, when the child was present, checked to see if it fit snugly. The last observation was the type of car seat used for the child. The child was not placed in the car seat to test for fit, nor was the placement of the seat in the vehicle tested.

Possible outcomes

Home safety behaviors relating to hot water, smoke detectors, bike helmet, and car seat use were dichotomized as being safe or unsafe. For example, tap water temperature measured to be 120°F or lower was categorized as “safe” and higher temperatures were labeled as “unsafe” (table 1). Car seat type was compared with seat use previously reported by the parent, irrespective of the child’s age or weight.

Statistical analysis

We compared demographic characteristics and safety practices of parents who completed the home visit and those who did not participate in the home visit using t tests and χ² tests.

Sensitivities, specificities, positive predictive value, negative predictive value, and confidence intervals were calculated for the telephone interview responses and the home observations among those who completed the home visit. If a safety practice was not applicable to the child, then the data were not included in the calculations for that practice.

Sensitivity analyses

A time interval of nine weeks between the telephone interview and the home visit could allow changes to occur in the home before the visit; therefore we repeated our main analyses on the subset of families who completed the home visit in four weeks.

All analyses were conducted using Stata 8.0 (StataCorp, College Station, TX, USA).

RESULTS

Of the 212 eligible families, 70 refused (33%) and 64 (30%) completed the home visit. The remaining 78 families were either unreachable (8%) or a visit could not be scheduled (29%).

Characteristics of study children and parents

The median time between the interview and the home visit was 27 days (SD 11.2). The sex of the index child was evenly distributed with 32 boys and 32 girls. The mean age and weight were 5.2 years and 19.4 kg, respectively. Sixty four percent of the children were self identified as White and 78% of the households reported an annual income of $25,000 or more. Fifty percent of parents had graduated from college or attended graduate/professional school, and only 19% reported high school graduation as their highest education attained.

Comparison of selected characteristics of participants and non-participants

There were no statistically significant differences of demographics between participating and non-participating families (table 2).

Families that agreed to participate were more likely to report planning to attend a car seat safety clinic (p = 0.004) (table 3). However, there was no significant difference between participating and non-participating families and discussing car seat safety with the child’s doctor. Results were similar when participants were compared only with those who refused the home visit after excluding those who could not schedule a home visit from the non-participants (data not shown).

Prevalence of observed safety behaviors

Most homes had a smoke alarm on each floor (89%) and the majority were functional when tested by the observer (89%). Sixty nine percent of the children owned bike helmets, but only 44% actually fit the child’s head snugly when checked. Few homes (42%) had kitchen tap water that registered less than 120°F. The most common car seat noted was the forward facing child harness seat (39%) followed by a booster seat (31%) and an adult seat belt (27%). As the sample was restricted to children over 1 year of age, no rear facing infant seats were noted.

Sensitivity and specificity

The sensitivities and specificities between the parent telephone interview and the home visit are shown in table 4. Sensitivities were high, ranging between 0.78 and 0.98. Eighty nine percent of families were observed to have a smoke detector on each floor and 98% of them reported this accurately on the telephone survey. When a working smoke detector was found at the home visit, 78% of parents reported replacing the battery within the past six months. Ninety five percent of observed children with bike helmets accurately reported having one on the survey, while a properly fitting bike helmet had lower reliability between observations and self reports.

Specificities were variable. Car seat type ranged from 0.95 to 1.00; tap water temperature and owning a bike helmet had a specificity of 0.50. Other practices had specificities of zero. Specificities were calculated to be zero for smoke detectors and snugly fitting bike helmet because among those observed with the unsafe practice, none accurately reported the unsafe practice on the phone; they all reported the safe practice. For example, the seven families who did not have smoke detectors on each floor reported on the phone interview that they had smoke detectors on each floor. The proportion of 0 out of 7 gives the specificity of zero.

Sensitivity analysis

The accuracy of self report was similar and followed the same pattern when that sample was restricted to the 44 visits that occurred within four weeks after the telephone interview (data not shown).

DISCUSSION

This study adds to the sparse literature on the validity of self reported home safety practices. Our study supports evidence that self reports of car seat type, hot water temperatures, smoke detectors on each floor, and owning bike helmets are reliable, whereas self reports on other practices of working smoke detectors and properly fitting bike helmets may be overstated. A limitation of this study is that only 30% of eligible families completed the home visit and such a select sample could bias results. Families who agreed to a home visit may have been more safety conscious than parents who were not visited, as they were more likely to have been planning to attend a car seat safety clinic than were non-participants. The measurable effect of attending a car seat clinic would have been a change in the type of car seat between the interview and the home visit if the family learned that they were using the wrong car seat for their child. This change could decrease the reliability of car seat reporting; however we found high reliability across all estimates for the type of car seat used. Reliability was also high and not substantially different from the results in table 4 when the age was matched to the car seat type. For example, among children age 4 years and over the reliability estimates for booster seat were no different than those reported in table 4. We did observe a decrease in reliability when we compared the age and weight appropriateness of the self reported and observed car seat type. Ninety one percent of the observed car seats were the appropriate type for the child, yet only 44% of the correct self reported car seat types were observed at the home visit. A further limitation of this study is that the observation of a working smoke alarm is not an accurate validation of the self report for replacing the batteries within the last six months. If the battery did not need replacing, then this would lower the sensitivity. Also the last car trip asked about in the interview may not have occurred in the family car so the car seat observed at the home visit would not validate the interview question. This would lower the reliability estimates; however, we observed high accuracy for car seat type.

The sensitivities and positive predictive values were generally high across practices. Safe practices observed at the home were reported accurately on the phone interview just as the safe practices reported on the phone were confirmed at the home visit. The specificities and negative predictive values varied across safety practices. Unsafe practices concerning smoke detectors, bike helmet, and hot water temperature were observed at the home visit more than they were reported by the parents on the phone survey. Unsafe practices concerning smoke detectors were more commonly observed at the home visit than they were reported by the parents on the phone survey. In our study most parents reliably reported having a working smoke detector on each floor, in estimates that were higher than those reported by Chen⁷ but lower than estimates from a study in the United Kingdom.⁵ It should also be noted that the reliability was much higher for the presence of the smoke detectors than for their functionality. This is an important issue because many families may feel they are safe from home fires simply because they have smoke detectors in their home, yet if the smoke detectors are not checked and batteries are not replaced, they provide no protection.

Although parents correctly reported having bike helmets for their children, most helmets examined did not fit the child snugly despite parental report to the contrary. Parents may feel their child is protected with a bike helmet but will not get optimal protection if the helmet is poorly fitting. This is noteworthy as there is evidence that many bike crashes result in injury to the forehead despite the use of a helmet, implying the use of poorly fitting helmets.¹⁴

Car seat type and hot water safety practice were observed to be very reliable. However, it is a limitation that there were so few families who reported a temperature for tap water. Therefore, the reliability estimates for hot water are based on very small numbers and have large confidence intervals that are consistent with less than half agreement.

This report adds to the evidence in the injury prevention literature that there should be concern for overreporting from self reports for certain child home safety and behavior practices. We found that parents tend to significantly overstate the presence of working smoke detectors and properly fitting bike helmets. Our results suggest that studies relying on self reported functioning smoke alarms and properly fitting bike helmets may underestimate actual safety behaviors, and the inclusion of an observational component would strengthen study findings.