Objectives—The aim of this study was to determine the validity of self report as a source of information on crashes and injuries.
Setting—This study was part of the Dunedin Multidisciplinary Health and Development Study (DMHDS), which is a longitudinal study of the health, development, and behaviour of a cohort of young New Zealanders.
Method—At the age 21 assessment DMHDS study members were asked to report serious injury and motor vehicle traffic crashes experienced over the previous three years. The self reported injuries were compared with the New Zealand Health Information Service (NZHIS) public hospital discharge file to determine the completeness of the self reported data. The traffic crashes were compared with the police traffic crash reports to determine the accuracy of self reported crash details.
Results—Twenty five (86%) of the 29 unintentional injuries, six (67%) of the nine assaults, and one (14%) of the six self inflicted injuries on the NZHIS file were self reported. The level of agreement between the self reported crash details and those recorded on the traffic crash report was high.
Conclusions—The results show that self reports can be a useful and valid source of injury and crash data.
Statistics from Altmetric.com
“No scientific discipline can be better than the quality of its raw data”.1 Since much of the data used in public health research are self reported, the quality of the data depends on the ability and willingness of study participants to accurately recall and report the relevant information. Despite this, there have been relatively few studies evaluating the validity of self report, and one area that has received little attention is the validation of self reported injury and crash histories.2
As part of a longitudinal study of child development, Langley and colleagues conducted two separate evaluations of self reported injury histories.3, 4 On both occasions injuries on a local hospital accident and emergency department record were checked against the self reported injuries. At age 9 years, 49% of those on the hospital record were self reported.3 At age 13, the comparable figure was 61%.4
In another study based on an adult population, self reports of hospital admissions for all causes, over a four year period, were obtained in a telephone interview, and compared with hospital records.5 The results showed that 58% of participants reported all admissions, 26% recalled some, and 16% none.
The reliance on self report in longitudinal studies, such as the Dunedin Multidisciplinary Health and Development Study (DMHDS), means that where possible it is important to examine reliability, validity, or both. For some assessments the reliability of the measures has been evaluated.6 For many studies it is not possible to assess the validity of self report because no “standard” is available. In New Zealand, however, traffic crashes and hospitalised injuries are recorded in national databases that can be used as a reference for comparing self reports. The main aims of the present study were: (1) to investigate the completeness of self reported injury requiring admission to hospital and (2) to determine the accuracy of the self reported information relating to traffic crash details.
The DMHDS is a longitudinal study of the health, development, and behaviour of a cohort of 1037 young New Zealanders.7 The members of this cohort were born at the only obstetric hospital in Dunedin, New Zealand between 1 April 1972 and 31 March 1973. They have been followed up regularly since birth, with assessments every two years up to 15 years of age, and then again at ages 18 and 21 years. The follow up rates have been consistently high (approximately 95%).
Since its inception, one of the main components of the DMHDS has been a study of injuries and their prevention. An important aspect of this has been obtaining personal injury and crash records from study members. At each assessment information has been requested regarding injuries experienced since the previous assessment. The self reported injuries and crashes included in the present study were the outcomes of interest for a much larger longitudinal study of predictors of crashes and injuries among the cohort.8 The period of recall was three years and among the outcomes sought were: motor vehicle traffic crashes, where someone sustained an injury for which medical attention was sought; and injuries that required an overnight stay in hospital. This information was obtained in a face-to-face interview, by a trained interviewer, using a structured questionnaire. As an aid to recall, prompt cards listing injury and crash definitions were used.
In New Zealand most acute injury is initially treated in public hospitals.9 All discharges (hereafter referred to as admissions) from these hospitals are recorded in a database maintained by the New Zealand Health Information Service (NZHIS). All injuries are coded according to the International Classification of Diseases injury and poisoning codes.10 This database, therefore, provides a record against which the self reported hospitalised injuries can be compared. Signed consent was sought from each of the study members to access their hospitalisation injury records. All but one gave this consent. Using a unique individual identification number, the NZHIS file was searched to obtain the relevant records. The details on these records were compared (by the first author) to the self reported injury forms to identify matched records, and determine the completeness of self reporting.
For traffic crashes there is no database equivalent to NZHIS file. Although all crashes involving injury are required to be reported to the police, it is known that there is significant under-reporting of these events. For example, for crashes involving hospitalised injury the reporting rate ranges from 54% (motor vehicle passenger) to 70% (motor vehicle driver).11 Conversely, some crashes involving minor untreated injury are included.12 While this file is not suitable to determine the completeness of self reports, for the crashes that are recorded it includes many details that can be used as a reference to assess the accuracy of self reported details.
The traffic crash report file did not include a unique identifying number that could be used for the purpose of matching, so this was done using Automatch.11 Automatch is a probabilistic and deterministic record linkage that has been used in a variety of settings including linkage of traffic crash records and public health databases. The details on the traffic crash report files identified as belonging to study members were compared with the details on self report crash forms to assess the accuracy of self report. These details included: vehicle type, year of crash, road user status, day of week, time of day, speed limit.
A search of the NZHIS file revealed 45 injury related public hospital admissions for the participants in this study, over the relevant time period. The number and percentage of these records matched to the self report are given in table 1. The percentage self reported ranged from 86% (unintentional) to 14% (self inflicted)
A search of the police traffic crash records revealed 51 crashes that involved a member of the DMHDS cohort, over the relevant period of time. Of these, 25 (49%) were matched to self reported crashes. A comparison of the details recorded in the traffic crash report file with those self reported is summarised in table 2. These results show that the agreement ranged from 100% for type of vehicle to 76% for time of day (within one hour).
The results of this study give us confidence that the data relating to unintentional injuries and motor vehicle crashes obtained by self report from the members of the DMHDS cohort are of good quality. Although the number of events examined was quite small, two aspects were especially encouraging. First, the reporting rate for unintentional injuries was high (86%). Second, the level of agreement between the details on traffic crash reports compared with the self reports suggest that self reports may be a valid source of crash information, without the biases associated with police reporting.
Although the numbers were very small, the level of self reporting of self inflicted injury was disappointing. Of the six unreported cases, however, five were drug overdoses. It is possible, therefore, that the study members involved did not consider these to be injuries. Alternatively, given the sensitive nature of these events, they may have chosen not to disclose them in the context of this interview.
Two points to be considered when interpreting the results of this study. First, because of the longitudinal nature of the DMHDS, the participants in this study may have developed better recall and are more willing to report socially undesirable behaviours than others who have not had this experience. Second, the procedure used to match the records may have lead to higher correlations than if this had been done “blind”.
This research was funded by the Accident Rehabilitation and Compensation Insurance Corporation of New Zealand (ACC). The Injury Prevention Research Unit is jointly funded by the Health Research Council of New Zealand (HRC) and the ACC. The Dunedin Multidisciplinary Health and Development Study is funded by the HRC, and also the United States NIMH. Thanks go to Mr Jonathan Alsop, the IPRU biostatistician, who did the data linkage for this study. Also, thanks to Dr Phil Silva, Director of the DMHDS, Ms Maria Hutchinson-Cervantes, Dr Tony Reeder, and Mrs Dianne Allnatt for their assistance with obtaining, coding, and checking the data. A very special thanks goes to the members of the DMHDS cohort for their cooperation over many years, and for providing good quality data.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.