METHODS

Data

In the United States, the National Highway Traffic Safety Administration collects data for the National Accident Sampling System Crashworthiness Data System (CDS), a systematic sample of crashes reported to police.^5–7 Specially trained investigators collect extensive data regarding about 5000 crashes per year.

I selected all CDS records from crash years 1988 through 2000 for passenger vehicles which had two occupants 16 years or older in the front seat, of whom at least one died: 1931 vehicles. Because information about age or sex was missing for at least one occupant, 29 vehicles were excluded. Belt use information was missing for 9.8% of the remaining occupants. Since risk ratio estimates are only possible in this study design when belt use information is known for both occupants, 213 vehicles were excluded due to missing belt use information, leaving 1689 vehicles for analysis. Excluded records generally differed little from included records in regard to mean occupant age (40 v 38 years), mean model year (1986 v 1986), mean crash year (1995 v 1994), proportion of occupants who were male (65% v 61%), or proportion of occupants that died (60% v 58%). However, omitted vehicles were less likely to have rolled over (18%) compared with included vehicles (28%).

Seat belt use and outcome

For information regarding seat belt use, CDS investigators used at least one of three sources: (1) post-crash inspection of the seat belt; (2) medical records, including autopsy records; or (3) interviews with survivors. Seat belt inspection can reveal evidence that the belt was used during the crash—for example, anchor points may be damaged or the belt may be stretched or abraded. Alternatively, there can be evidence that the belt was not used—for example, if it is loosely knotted, stuffed under the seat, or tightly retracted by a pretensioner. Medical records may be searched for evidence of bruising by a seat belt. Interviewed survivors were told that information was confidential and would not be shared with insurers or police.

Statistical analysis

I used a matched cohort study design, which only requires information from pairs in which at least one had the study outcome, death within 30 days of a crash.^8–11 I estimated the risk ratio for death of front seat occupants who wore a seat belt, compared with those who did not, using conditional Poisson regression.^12–14 The risk ratios were estimated by comparing occupants who crashed in the same vehicle, thereby controlling for vehicle and crash characteristics.^8,15 Agreement between CDS investigator classification of belt use and police report of belt use was estimated using the κ statistic.¹⁶ I used Stata statistical software.¹⁷

RESULTS

The 1689 CDS study vehicles had a median crash year of 1994 and a median model year of 1987. Belted occupants were older and more often female compared with unbelted occupants (table 1). Belted occupants were more often in a seat equipped with an airbag, or had an airbag deploy, compared with those unbelted. A greater proportion of unbelted occupants died.

During 1994 through 2000, CDS investigators recorded the most important source of their belt use information: vehicle inspection (91.8%), interview (4.9%), medical or autopsy records (2.6%), and other sources (0.8%). Police report of belt use was missing for 11% of the study records. Among the records for which belt use was recorded by police, expected agreement due to chance between CDS investigators and police was 51%, actual agreement was 91%, and the κ statistic for agreement beyond chance was 0.82. Among occupants classified by CDS as belted, police classified 7% as unbelted (table 2). Among occupants classified by CDS as unbelted, police classified 10% as belted. Assuming the CDS determination of belt use was correct, the sensitivity of police reports for use of a belt was similar for those who died (93%) and those who lived (93%) (table 2). The specificity (percent of those unbelted who were correctly categorized) of police reported belt use was somewhat greater for those who died (92%), than for those who lived (87%).

Using belt use data from the CDS investigators, the risk ratio for death, comparing belted with unbelted occupants and accounting for matching within vehicle, was 0.35 (95% confidence interval (CI) 0.28 to 0.44). The information from which the crude risk ratio is derived comes from those within-vehicle pairs which were discordant on seat belt use^8,11; this risk ratio can be calculated by dividing the number of belted persons who died (38 + 59) by the number of unbelted occupants who died (38 + 238) (table 3). Using conditional Poisson regression to adjust for age, age squared, sex, seat position (driver or passenger), changed the estimate only slightly to 0.36 (95% CI 0.29 to 0.46) (table 4). Airbag presence, airbag deployment, occupant weight, and occupant height had little confounding influence on the estimate for seat belts. When I used police reported belt use as the exposure, rather than belt use determined by the CDS investigator, the adjusted risk ratio was 0.36 (95% CI 0.28 to 0.47).

DISCUSSION

In this study of crashes, the adjusted risk ratio for death of a belted occupant, compared with one not belted, was 0.36. This was estimated from data that did not rely on belt use as recorded by the police. The estimate was similar, 0.36, when it was based upon police reported belt use for the same crashes. The close agreement between these two estimates suggests that false claims of seat belt use by survivors is not likely to be an important source of bias in estimates of seat belt effects.

One limitation of this study is that CDS belt use information may suffer from some degree of measurement error. While substantial differential bias in seat belt classification seems doubtful, some is possible. Another limitation is that information about belt use was missing from some records; 11% of the CDS study vehicles were omitted from the analysis for this reason. Given that the proportion of data omitted was small, and the omitted records were similar to those included, this is not likely to have had a major influence on the risk ratio estimate. A third limitation is that CDS investigators were aware of the restraint use information on the police report. Although they were specifically instructed not to use this information in their determination of seat belt use, they might be influenced by the police report. There is evidence, however, that National Accident Sampling System CDS investigators are willing to make judgments that are independent of those made by the police; in a study that compared CDS investigator and police reports, agreement about belt use was only 75%, much less than the 91% agreement I found for vehicles in which someone died.¹⁸

If the study sample of CDS crashes were systematically different from all crashes in the United States, in regard to factors that modify the effects of seat belts, the average risk ratio from this study might not apply to other crashes in the United States. The Fatality Analysis Reporting System (FARS) contains data on all crashes that involve a death on United States public roads from 1975 to the present.¹⁹ From FARS data I selected driver-passenger pairs just as I did for the CDS data, from calendar years 1986–98: 47 580 vehicles.²⁰ I compared these FARS vehicles and this study’s CDS vehicles with regard to occupant age and whether the vehicle rolled over or not, as the effectiveness of seat belts has been reported to vary by these factors.²⁰ Results were similar. Among FARS crashes, 25% involved a rollover, compared with 28% in the CDS sample. Mean and median occupant age were 38 and 29 years in the FARS data and 38 and 30 years in the CDS data.

A strength of this study was the use of a matched cohort design. Estimates were generated based on within-vehicle comparisons, thereby controlling for the potential confounding influence of either vehicle characteristics, such as weight and model year, or crash characteristics, such as speed, impact angle, and crash location.^8,15

These results suggest that police reported belt use information may be fairly accurate in severe crashes where at least one occupant died. It appears (table 2) that police reports had some non-differential misclassification of belt use which tended to bias the risk ratio estimate toward 1, and some differential misclassification which tended to bias the estimate toward 0. These modest amounts of misclassification bias were nearly balanced, so that risk ratios based on either CDS investigator information or police information were similar.

Surviving occupants in this study were seated next to someone who just died or was so seriously injured that they would die within a few days. Many of the survivors themselves were seriously hurt; 61% were hospitalized. It is possible that these survivors, compared with survivors in less serious crashes, were not concerned about the fine they might incur for not wearing a seat belt.²¹ Furthermore, police generally investigate fatal crashes intensively, which may enhance the quality of police belt use data in these crashes. Special police investigators are often used for fatal crashes and they may be aware of the belt inspection techniques used by CDS investigators. Police have the advantage of usually being at the crash scene soon after a fatal crash, so they may determine belt use by direct observation. Thus the matched cohort method, which requires data only from vehicles in which at least one occupant died, may make use of the most accurate seat belt information. In less serious crashes, false claims of seat belt use by survivors may be a greater problem.

Two studies, both using FARS data, have reported that estimates of the ability of seat belts to prevent death in a crash have changed over time.^4,20 An apparent increase in the protective effect of seat belts began around 1984; the first state seat belt law was passed in that year and some investigators have suggested that these laws have encouraged false reporting of belt use by survivors, resulting in a change in risk ratio estimates as seat belt laws became more common.⁴ As shown in fig 1, the average risk ratio estimate for the effectiveness of seat belts in preventing death was about 0.6 before 1985 and about 0.4 after that year. This present study, however, casts doubt on that theory; in serious crashes in an era when nearly all states had seat belt laws, the risk ratio based upon police crash reports was the same as the risk ratio based upon belt use data which did not rely to any important degree on the claims of crash survivors.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Prevalence of seat belt use for front seat occupants in cars with at least one front seat occupant death. Risk ratio for death of a belted occupant compared with an unbelted occupant in the same vehicle, according to calendar year of crash. Data from Cummings et al.²⁰

Changes in effect estimates over time could arise for several reasons.²² It is possible that seat belts used in recent years were more effective than those used in the past. This theory seems unlikely, as colleagues and I have reported evidence of a trend in risk ratio estimates related to the year in which a vehicle crashed, but no evidence that belt effects changed according to car model year.²⁰ If seat belts were more effective in certain crashes or for certain occupants, and if these crashes or occupants became more common, then belts would appear to become more effective over time. This theory also seems doubtful, because in vehicles that crashed there was little change over time in median speed, mean deformity, the proportion that had a frontal impact, the proportion that rolled over, or the median age of belted or unbelted occupants.²⁰

Another theory is that non-differential misclassification of seat belt use, unrelated to whether the occupant lived or died, might account for the changes in risk ratio estimates.^20,23,24 Non-differential misclassification refers to misclassification of an exposure (seat belt use) that is not related to the outcome (death in this study). Seat belt use might be inaccurately recorded on some police records or incorrectly entered into the computer files used for analysis. Non-differential misclassification of a binary exposure will bias risk ratio estimates toward 1.²⁴ If misclassification of this kind decreased over time, then the estimated risk ratio would move away from 1.0 toward its true value.

Alternatively, a constant amount of non-differential misclassification, combined with the substantial increase in seat belt use that occurred during the 1980s, could produce a change in risk ratio estimates over time.^20,22–24 Fewer than 4% of occupants were classified as belted in 1980, among driver-passenger pairs with at least one death (fig 1).²⁰ If just 2% of belted and unbelted occupants were misclassified due to coding errors, this could cause a great deal of bias in the risk ratio when belt use was rare, because misclassification of just a small proportion of unbelted occupants to the belted group would greatly increase the proportion of occupants classified as both belted and dead; colleagues and I have shown by numerical example that this could bias a true risk ratio of 0.35 to 0.61.²⁰ But with time, as belt use increased, the same amount of misclassification would cause only minimal bias. The present study offers support to this theory; if we accept the CDS investigator determination of belt use as accurate, the amount of non-differential misclassification in police data was about 7% in both directions.

Studies using United States crash data before 1986 have generally estimated a risk ratio for seat belt effects of about 0.55.^4,25–27 After about 1984, seat belt risk ratio estimates based upon the same methods and source of data have moved to about 0.40.^4,20 This change could, as described above, be due either to false claims of belt use by crash survivors which are recorded by police (in which case the older estimates would be less biased) or to a decrease in the influence of non-differential misclassification as belt use became more common (in which case the more recent estimates would be less biased). In this study I found evidence that supports the non-differential misclassification theory.

The risk ratio estimate from this study of CDS data was 0.36, similar to estimates of 0.39 (95% CI 0.37 to 0.41) and 0.35 (95% CI 0.33 to 0.36) from two matched cohort studies of recent FARS data conducted by myself and colleagues.^20,28 Based on the findings of this study, false reporting of belt use by crash survivors is probably not an important sources of bias in these estimates. If these estimates reflect a true benefit of seat belts, they suggest that use of belts can prevent about six of 10 deaths that otherwise would occur in an automobile crash.