Discussion

This study adds to the literature on validity of self-report for measuring SA behaviour in populations. The PPVs described in the literature to date have clearly varied (from 26% to 90%), which could be a result of different measures of SA coverage, some, the present study included, with more strict definitions. The most comparable study reviewed above (conducted by Chen, Gielen and McDonald also in Baltimore City) found a PPV half that found by the present study (26% compared with 54.2%).9 The reason for this discrepancy could likely be the setting of the initial interview, via telephone in advance of the home observation in the 2003 study as opposed to in the home at the time of home observation in the present study. The validity of respondent-reported home safety behaviours has been seen to decrease when questions are asked over telephone rather than in the home.7 ,8 ,10 Overall, self-report of SA coverage has shown troublingly low validity, and research that needs to rely on self-report to evaluate programmes or monitor trends should bear in mind the degree to which there is error in such measurement, especially with regard to which population groups may be at increased risk for incorrectly reporting their levels of protection.

Despite our small number of follow-up respondents, the follow-up interviews do shed some additional light on these findings. The majority of those respondents we were able to follow-up with seem to have given the wrong information because they assumed the alarms were working for a variety of reasons. These respondents also had helpful suggestions for improving self-report measures in the future. In particular, asking respondents to test the alarm during the interview as well as better use of probes and encouragement are options that would be well worth trying in future survey research. A quarter of respondents, indicating that they were aware of their SA coverage, gave wrong information because they did not think they actually needed a working alarm on every level. Even though a very specific question about SA coverage was asked, it was interpreted by many respondents as ‘do you feel you have enough working smoke alarms?’ It is concerning that 35% of follow-up respondents indicated that they had reported greater than actual coverage because they knew that they should. This social desirability bias occurred despite the fact that respondents had been informed before the interview began that data collectors would be testing alarms in the home as part of the interview.

Although we interviewed a large and diverse sample, respondents were sampled from an urban inner city area that has high rates of poverty. Our sample reflected this, with about a half of respondents reporting a per capita income of $10 000 per year or less. Therefore, results should not be generalised to populations with very different characteristics. We also had a small number of participants in the qualitative follow-up survey. Nevertheless, the results shed new light on the interesting methodological challenges of obtaining valid self-report measures of important safety behaviours. In collecting data on home safety behaviours, we would recommend that researchers be mindful of the ways in which survey questions might be interpreted and answered in unintended ways. Such suggestions as asking residents to test alarms over the phone could be adapted to other safety behaviours to promote more accurate reporting. It is, however, important to note that the focus of the larger study was promoting complete SA coverage; reasons for under-reporting, which may have further implications for safety research design, were not explored here.

The results also have practical utility for promoting adequate SA protection. Given our findings that the high rates of over-reporting are likely due to inattention and incorrect perceptions regarding ‘safe’ SA coverage within the home, public education efforts should use messages that raise the salience of SA functionality. It seems that some people think SAs can be forgotten about once they are installed, and they would benefit from reminders about checking alarms as well as changing the batteries. Clinicians who counsel their patients about the need for SAs should consider how they can incorporate better assessment questions to guide their counselling. For example, paediatric anticipatory guidance about safety often starts with asking the parent if they are performing the behaviour (eg, Do you have smoke alarms?). As found in an earlier study, giving appropriate advice depends on having the correct information.9 Our results suggest that clinicians can fully rely on a negative answer to such assessment questions, but they should probe carefully any positive answer. Clinicians should ask about whether SAs are present specifically in the basement or attic, where some may feel they are not needed. The same implications apply to others who provide individually focused education such as public education firefighters and other public health practitioners engaging in fire safety counselling.

Furthermore, the lack of awareness or confusion shown by the over-reporting respondents in our follow-up sample regarding their SA coverage suggests that outfitting homes with more permanent alarms could be the best option to ensure that more households are protected from fire deaths. Hard-wired or sealed 10 year lithium battery alarms outfitted with a hush feature can be screwed into the wall or ceiling, making it easier to silence nuisance alarms and harder to remove the unit. Maintenance of lithium battery alarms is less demanding than the more common alkaline battery alarms, which require new batteries approximately every 6 months. When installed in homes, lithium battery alarms remain functioning longer than alkaline battery alarms.11 ,12 Given the low PPV seen for self-reporting SA coverage, efforts to equip homes with more permanent and tamper-proof alarms may prove worthwhile.