METHODS

The questionnaire which measured outcomes for the trial was in the form of a booklet with 49 questions spread over eight sides, including questions on safety practices, satisfaction with safety equipment, and accident occurrence. Safety practices included the storage of potentially hazardous items and safety equipment possession and use. For example, in relation to smoke alarms questions covered possession, whether they were fitted (attached to some part of the house) and how many were working.

The answers to 26 questions that could be assessed by observation were checked by a home visit. Altogether 2000 families were sent the questionnaire and a random sample of 300 of those that responded to the initial mail out were invited to take part in a “home safety check”. The first 32 families in each of the intervention and control groups who responded to the “home safety check” invite and could be contacted were visited. The visits were organised as soon as possible after the questionnaire was returned; the average length of time between return of questionnaire and the home visit was 37 days. Each family taking part in the “home safety check” received a £5 voucher. At the time of the visit the researcher was blind to the self reports in the questionnaires and the families were not told that the visit was part of a validation study.

The sample size was calculated based on an estimated sensitivity of 0.8. To obtain a 95% confidence interval for the sensitivity of 0.65 to 0.95, and assuming a minimum of 40% of families were observed to have a safe practice for a specific safety practice, then 64 questionnaires required validation.

Data were entered onto an ACCESS database, verified by double entry and analysed using SPSS version 11.0 and StatsDirect. The answers to some questions were combined in order to ascertain whether or not certain practices were safe, giving 16 safety practice outcomes. For example, the kitchen was considered safe in relation to medicines if there was none there or if they were stored at eye level or above or if they were stored at another level in drawers or cupboards that had safety catches or locks. Similar rules were applied to cleaning materials and sharp objects in the bathroom and kitchen. For each safety practice, the sensitivity, specificity, positive (PPV) and negative (NPV) predictive values were calculated and their 95% confidence intervals using the exact Clopper-Pearson method (fig 1).¹⁴

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Key terms.

Where more families under reported than over reported safe practice, PPV exceeds NPV. Where more families over reported than under reported safe practice NPV exceeds PPV.

In order to find out if there were any significant differences between the 64 families who had home visits and the rest of the families (n=1457) that completed questionnaires, comparisons were made of their self reported safety practices. The data were analysed using χ² tests and by calculating odds ratio and 95% confidence intervals for each safety practice.

RESULTS

The response to the initial mail out of the 2000 questionnaires was 57% and the final response was 77%. The response to the 300 invitations for the “home safety checks” was 32%. The total number of households visited was 64. Table 1 shows sensitivity, specificity, PPV, and NPV and percentage agreement between the questionnaire and observations. The PPV was high for most safety practices (78% or above for 15 of the 16 practices). For five safety practices a higher proportion of families over reported than under reported safe practice (NPV exceeds PPV). For 10 practices a higher proportion of families under reported than over reported safe practice (PPV exceeds NPV).

Table 2 shows that for most of the safety practices there appears to be no significant difference between the questionnaire responses of the home safety check families and the rest of the families who returned the questionnaire. However, in relation to safety on the stairs, home check families were more likely to report that they either had no stairs or had one or more stairgates.

DISCUSSION

This research has provided information about the validity of self reported safety practices from a home safety questionnaire and highlights which questions could be used in the future. This study found that self reported safety practices were good at predicting observed safe practice for the majority of practices. Where discrepancies occurred between self reported and observed practices, under reporting of safety occurred more commonly than over reporting, which in terms of using self reported practices as outcome measures is encouraging. Over reporting of safety occurred most commonly with storage of sharp objects in the kitchen, possession of window safety catches and fireguards.

Disagreements between the questionnaire and the home visits may have been due to a number of reasons including movement of hazards or safety equipment such as sharp objects or fireguards between completion of the questionnaire and the home visit or providing socially desirable responses such as stating possession of a fireguard but not having fitted the fireguard, or stating place of storage of sharp objects, not where they are left while, or after, being used. The discrepancy in responses to possession of window safety catches became clear to us at the observations as families found it difficult to distinguish between handles, security locks, and catches to limit the width of opening. Further work is required to devise improved questions in this area.

An important feature of our method that needs highlighting is that we kept the time period between the respondents completing the questionnaire and the home visits as short as was practical. This could be seen as a strength of the design. However, choosing from the people who responded to the initial mailing of the questionnaire could also lead to bias. Respondents who reply early may be more organised, more efficient, and may also be so in the application of safety practices in their homes. Our study shows that most self reported safety practices do not appear to differ between the early responders and the rest of the group that responded.

Another point that has implications for the generalisability of our findings is that our study only dealt with responders. The published literature suggests non-responders differ systematically from responders in terms of sociodemographic characteristics, ethnicity, and interest in the topic covered in the questionnaire.^15–28 In addition, there also appears to be differences in the reporting of safety practices.²⁹

Although our study is fairly small, which led to wide confidence intervals, it was larger than the previous general home safety validation study.¹² Although our study is in agreement with the other general home safety study,¹² it is out of line with the single issue smoke alarm studies.^8,9 The fact that in our questionnaire the smoke alarm questions were but a part of a general injury questionnaire might be one reason for the difference. It may also be due to the different research methods used.

In the field of injury prevention, the main concern in relation to the validity of self reported information has been the potential for over reporting. Our finding of more under reporting than over reporting is encouraging, but requires confirmation from larger studies.

Copies of the questionnaire used in this study are available from michael.watson{at}nottingham.ac.uk.