Article Text
PDF
Abstract
Objectives—To determine the effect of different methods of administering a diary to collect information from parents on near miss and minor injuries on responses, completeness and accuracy, the number of incidents reported, the effect of a financial incentive on response, and the cost of administering each method.
Setting—The study was set within the context of a cluster randomised controlled trial of injury prevention in 36 practices in Nottingham.
Methods—The study population comprised the 1594 parents who responded to the baseline questionnaire. Parents were allocated systematically to one of four groups: postal administration, with and without financial incentive, telephone administration, with and without financial incentive (102 in each group). A clinic visit method with and without financial incentive (50 in each group) was also used.
Results—A significant trend was found, with decreasing response rates with increasing degree of contact with the parent, such that administering the diary in the clinic had the lowest response (χ2 for trend = 5.54, 1 df, p = 0.02). Offering a financial incentive increased responses from 47% to 59% (χ2=5.78, 1 df, p = 0.016). The most complete recording was found in the diaries handed out at clinic visits. Importantly, parents were accurate in their recording of near miss and minor injuries, suggesting they understood the differences between the two types of incident. Postal methods were the least expensive method of administering the diary in terms of average cost per returned diary. Using a financial incentive resulted in a lower cost per returned diary for telephone and clinic visit methods.
Conclusions—Parents can accurately and reliably complete diaries recording near miss and minor injuries occurring to their preschool children. More work is needed to investigate methods of increasing response. Postal diaries achieve the highest response but have the least complete recording of data. Diaries administered through child health clinics were most complete but achieved the lowest response. The administration method chosen in future work should be influenced both by the response and completeness of recording that is required by the research.
- diary
- methods of administration
- near miss
- minor injury
Statistics from Altmetric.com
Currently there are a number of different outcome measures used to plan and evaluate childhood injury prevention programmes.1 Mortality statistics are commonly used, but the rarity of fatal childhood injuries limits their use as a measure of the success or failure of an injury prevention programme.2 Changes in the number of injuries receiving medical treatment, either at a hospital or by primary care team members, are also frequently used for planning and evaluating injury prevention programmes.3 Although such incidents occur more frequently than injury fatalities, using them as an outcome measure still requires large numbers of children to be studied over long time periods. Furthermore health care utilisation data are subject to selection biases introduced by factors such as distance from hospital, socioeconomic factors, admission policies, and bed availability.4–8
Using information on near miss and minor injuries as an outcome measure has a number of theoretical advantages. Firstly, the study of these more common injuries would enable the evaluation of injury prevention programmes to be carried out over shorter time periods using smaller study populations. Secondly, this additional information source would provide a more detailed picture of the complete spectrum of injuries occurring in a community.2
Collecting such information is difficult because it has to be collected using parental reports. Questionnaires and interviews have been used previously with parents,9,10 however these retrospective methods tend to be influenced by recall bias.11 The diary as data collection tool is being increasingly used in health service research, mainly because it has the advantage of collecting detailed information while reducing the effect of recall bias.12,13
The way in which a diary is administered does create issues that may affect costs and response rates and also the way in which information is recorded. The best responses are usually achieved by personal contact and delivery,12 however, this method is significantly more expensive to administer than any other method.14 An alternative to contacting subjects in person is to contact them over the telephone. The advantages of this method are the savings incurred in terms of time and consequently money, while maintaining many of the advantages of face to face contact. The potential disadvantage of using a telephone method, other than the increased administration costs, is the possibility of having a biased sample. Not having a telephone is more prevalent among lower social classes,14 the sample may therefore systematically under-represent poorer people. Posting diaries is by far the cheapest method, however, low response rates also reduce the generalisability of results.15 There are, however, techniques that have been shown to increase response rates such as the use of a financial incentive.15
The aim of the study therefore was to evaluate a variety of methods of administering a diary to collect information on near miss and minor injuries from parents of preschool children. This evaluation has assessed response rates, the effect of using a financial incentive on response, the completeness and accuracy of the information recorded, the number of incidents reported, and the cost of administering each method.
Method
DIARY DESIGN
As recommended in the diary literature,12,15 the first part of the diary contained a section by section set of instructions for completion and carefully worded definitions.
The definition of near misses used was: “A near miss incident is something your child does or that happened to your child which could have resulted in him/her being hurt, but fortunately it did not. For example, if your child placed a small object in his/her mouth but spat it out straight away rather than swallowing it and possibly choking”.
The definition for a minor injury was: “A minor injury is something which happens unexpectedly and results in the child being hurt in some way but it was not serious enough for you to go to a doctor's surgery or an accident and emergency department. For example your child cut his/her leg on a corner of a piece of furniture but it was only a small cut needing a plaster”.
The second section consisted of a 14 day calendar. Each day on the calendar was divided into three boxes; one for near misses, one for minor injuries, and one to record when none of these incidents had occurred. Parents were asked to place a tick in the appropriate box/boxes, each day, indicating how many incidents had occurred to the child. If parents realised they had failed to complete the diary on a particular day, they were asked to fill each box with a large cross.
The third and final section comprised eight questions identified as those routinely collected by the Home Accident Surveillance System (HASS)16 relating to the circumstances surrounding the incidents. The categories included the date and time, location, prior activities, mechanism of injury, objects involved, body part injured, and injury type. The near miss and minor injury recording sheets were colour coded, this was to reduce the possibility of parents recording information in the wrong section.17–19 The number of sheets to include (four minor injury and six near miss sheets) was established after the pilot study. The diary contained 12 pages in total.
VALIDITY
A systematic review of all the relevant literature was undertaken to identify circumstances commonly associated with injury events and the first draft of the diary was discussed extensively with a panel of experts (health promotion officers, NHS policy makers, primary health care team members, diary researchers, and a statistician).
PILOTING OF THE DIARY
The diary was piloted in its final format on a sample of 30 parents attending a child health surveillance clinic in Nottingham. Questions were asked during the piloting about the suitability, readability, layout, acceptability of questions, and question format. Responses from the pilot indicated that no changes needed to be made.
RELIABILITY
A one in nine sample of the 231 parents who completed the diary were contacted by telephone (within two weeks of completing the diary) and asked to recall an event recorded in the diary. The consistency of the responses was compared by calculating κ coefficients.
SETTING
The diary study was set within the context of a cluster randomised controlled intervention trial of injury prevention in primary care based in 36 practices in Nottingham.3
STUDY POPULATION
A total of 2152 questionnaires were mailed to the parents of children aged 3–12 months registered with the practices participating in the main trial. The 1594 who responded to the study questionnaire were used as the study population for the diary project.
STUDY SAMPLE
The sample size estimate was based on response rates from previous research. We estimated, based on 80% power a significance level of 5%, a response of 55% for postal method and 75% for hand delivered diaries,12,15,17 that a total of 100 families would be required in each group (with or without financial incentive) to detect a difference in response of 10%. After commencing the clinic visits it became clear that recruiting 100 families in both groups (with or without financial incentive) would not be feasible in the time scale, so the sample size was reduced to 50 in each group, which still allowed 100 families in the clinic group for comparison with the postal and telephone groups.
SAMPLING PROCEDURE
From the intervention group (n=823) every 16th parent was selected for the postal administration with financial incentive and every 15th for the postal group without financial incentive, without replacement. Every 14th and 13th parent in the intervention group was selected for the telephone administration method with and without financial incentive, respectively (a total of 102 for each method). The same selection process was used for the control group (n=771). Four practices and their matched control practices were selected for the clinic visits. These practices were chosen to include practices with low, medium, and high deprivation as measured by their Jarman20 and Townsend21 scores. A researcher visited the weekly child health surveillance clinics and handed out diaries to parents until 50 had been recruited to each group (those receiving financial incentive and those not).
METHODS OF ADMINISTERING THE DIARY
All parents selected for the postal groups were sent a diary with a covering letter explaining the purpose of the diary; the letter also assured confidentiality of the information. Parents selected for the financial incentive group were also informed that they would receive a £2 voucher to spend in a local children's store once the completed diary had been received. Parents in the telephone groups were contacted by telephone and provided with the same information, if the parents agreed to take part a diary was posted to them. A prepaid envelope was included with all of the diaries. The clinic visit method is described above, any parent who had already received a diary by post or by telephone was excluded from the study.
Non-responders from all groups were contacted four weeks after the first contact and asked if they would return the initial diary or if they would like another diary to complete.
DATA ENTRY AND CODING
The diary information was coded after completion of the diaries. Data were entered onto an SPSS database and validated by repeated entry. Any errors were corrected by checking the original diary recordings.
ANALYSIS
Categorical data were analysed using χ2 tests and the relative risk (RR) and 95% confidence interval (CI) were calculated. The data were analysed using the SPSS statistical analysis package.22
Results
(1) RESPONSE
Of the 102 parents who had been selected for each of the telephone method groups (with and without financial incentive) only 65 in each group could be contacted during the study time period. Response from the various methods ranged from 38% to 69% (table1).
Responses from the six methods of administering the diary
The responses were firstly compared between the methods of administration (table1). This demonstrated a significant difference in responses between the administration methods (χ2 = 7.8, 2 df, p=0.02) and a significant trend with a decreasing response with increasing degree of contact with the parent, such that administering the diary in the clinic had the lowest response (χ2 for trend = 5.54, 1 df, p=0.02). Comparing the individual methods demonstrated that both postal (χ2 = 6.76, 1 df, p=0.009) and telephone administration (χ2 = 5.73, 1 df, p=0.017) were superior to administration at clinic visits. These findings remained significant after correcting for multiple comparisons using the Bonferroni correction.
Responses were significantly higher when an incentive was offered (table 2), 59% versus 47% (χ2 = 5.78, 1 df, p=0.016). Analysing the response rates by individual administration methods demonstrated that the only significant difference found was within the telephone administration group (χ2=8.03, 1 df, p=0.005).
The effect of financial incentive on responses
(2) COMPLETENESS OF RECORDINGS
The item of information that was most commonly missing in all of the diaries was the object involved in the incident. Diaries issued to parents during clinic visits were significantly more likely to be complete (70% of returned diaries with all information recorded) when compared with those returned by parents in the telephone (χ2 =5.24, 1df, p=0.02, 49% complete) and postal (χ2 = 8.68, 1df, p=0.03, 46% complete) groups.
(3) ACCURACY OF RECORDING
“Near misses” in particular are difficult to study as they are based on subjective perceptions of how “near” an incident is, to be classed as a near miss.11 Two of the methods involved personal contact, one by telephone the other face to face, which allows for individual explanation.15 Recording of information using the postal method was reliant on parents reading and understanding the explanations and study definitions of such incidents. We therefore assessed the extent to which individual contact would affect the recording of information. With the exception of two minor injury events, which had been recorded by parents who had received a diary through the post as a near miss, all other descriptions met the study criteria and were recorded in the correct category (n=348).
(4) NUMBER OF INCIDENTS RECORDED
It was hypothesised that personal contact may have the adverse effect of influencing parents recording of the number of incidents experienced by the child—that is, by recording fewer incidents than actually occurred. The proportion of children reported as having experienced at least one near miss incident (χ2 = 0.43, 2 df, p=0.81) and at least one minor injury (χ2 = 5.08, 2 df, p=0.08) was less in the clinic group, but this did not reach statistical significance (table 3).
The number of children having one or more near miss and/or minor injury incident by method
(5) COST OF ADMINISTERING EACH METHOD
The costs of administering each method in terms of average cost per returned diary were calculated for each method by including the cost of postage (at 20 pence per mailing), telephone calls (based on an average call of 10 minutes, £1 per call), mileage at 41 pence per mile, time (in days) taken by a research assistant grade 1B, and the cost of offering a financial incentive (£2 voucher per returned diary).
The clinic visit method incurred the highest costs when compared with the telephone and postal methods. This was mainly because of the higher than expected number of visits, which had to be carried out to recruit the number of parents needed. The postal method was less costly in terms of both time and money when compared with the other two methods, at an average cost per returned diary of £1.73 without financial incentive and £3.54 with. The costs for the telephone administration without incentive were £14.98 per returned diary and with incentive £10.23. Clinic administration methods were substantially higher at £38.48 per diary returned without financial incentive and £35.24 with incentive. The reduced costs per returned diary for telephone and clinic visit groups (with financial incentive) were due to a higher proportion of parents in these groups returning the diaries without a reminder, lowering postal and telephone costs.
(6) CHARACTERISTICS OF RESPONDERS AND NON-RESPONDERS
The only significant difference shown between responders and non-responder characteristics was that non-responders were less likely to have access to a car (RR 0.72, 95% CI 0.54 to 0.96, p=0.02). No difference were found between the two groups in the proportion of families whose child suffered at least one medically attended injury over the two year follow up period (RR 0.84, 95% CI 0.66 to 1.06, p=0.17).
(7) RELIABILITY
κ Coefficients were calculated between responses recorded in the diary and those given by telephone for all of the categories (time of day, location, prior activity, mechanism of injury, injury, and body part injured); these ranged from 0.81 to 0.94.
Discussion
This study has demonstrated that parents will complete and return diaries recording near miss and minor injury incidents occurring to their preschool children, with a response that is low, but not dissimilar to that achieved by many postal surveys.12,15,23 Furthermore, the response varied by the method of administration. Surprisingly the method that involved the least contact with the parents produced the highest response. This was also the least expensive method of administration. Offering a small financial incentive was effective in increasing the response from 47% to 59%. Interestingly the completeness of data recording appeared to be related to the degree of contact with the researcher with the most complete recording occurring in the clinic administered diaries and the least complete recording occurring in the postal administered diaries.
One of the most cited disadvantages of using the diary as a research instrument is the likelihood of excluding a proportion of the population from certain social groups with low levels of literacy who feel they are unable to complete a diary.24–26 Similarly using a telephone administration method may also exclude the lower social classes. It could be argued that there may have been a selection bias towards parents who are literate, middle class, and who may also be less likely to be at risk of a future medically attended injury. Comparison of responder and non-responder characteristics showed that non-responders were less likely to be car owners. However, the frequency of medically attended injuries experienced did not differ between non-responders and responders. In addition all of the parents selected for the telephone method group had a telephone with a listed number.
It was interesting to find that the administration method that had the least contact with the parent produced the highest response. One possible explanation for this may be that parents find it easier to report near misses and minor injuries more anonymously. Previous work has shown that social desirability bias tends to be minimised using postal questionnaires14 and the results from the study suggest that this may be the case when using diaries. The use of clinic visits to administer the questionnaire may have made parents feel the researcher was associated with the clinic and that despite specifying that the information recorded was confidential, the parents may have been less likely to believe this to be the case. This may have important implications for the use of such diaries (or other methods of self report such as the parent held child record) in the future. Using diaries through child health clinics, and perhaps especially if administered by health care workers, may lead to a low response. A method of administration, which is perceived as more anonymous by parents, may be more successful.
The addition of a financial incentive (a £2 gift voucher for a local children's store) increased the overall response rate from 47% to 59%, and in the telephone group increased it from 45% to 69%. It would therefore seem sensible to include a small financial incentive in future work with diaries, especially as the cost per returned diary was lower when a financial incentive was used for both the telephone and clinic visit methods.
The differences in completeness of recording by the different administration methods are interesting. There appears to be a trade off between response and completeness, with postal diaries achieving the highest responses but being the most incomplete, and clinic diaries being most complete, but achieving the lowest response. The method of administration should therefore be considered along with the degree of detail required in future research. Where information is only required on the number of near miss or minor injuries incurred a postal method may be sufficient. Where more detailed information is required regarding the circumstances surrounding the incident, telephone or clinic administration may be preferable, perhaps because it allows a more detailed explanation of the information that is being requested.
Importantly, parents were accurate in their recording of near miss and minor injury incidents, suggesting they understood the difference between the two types of incident. It is, of course, extremely difficult to validate parental recording of near miss and minor injuries, but the finding of high reliability of the recording of incidents and the circumstances surrounding those incidents is encouraging.
In conclusion, this pilot study has demonstrated that parents will accurately and reliably complete diaries recording near miss and minor injuries in their preschool children. More work is needed to investigate methods for increasing response further. Postal diaries achieved the highest response but had the most incomplete recording of data. Diaries administered through child health clinics were most complete, but achieved the lowest response. Financial incentives were effective in increasing responses. The administration method chosen in future work should be influenced both by the response and by the completeness of recording that is required by the research.