Dear Editor

We thank Dr Carra for his comments[1] and we appreciate his attention to our work.[2]

Our paper was directed to a method for ranking potential safety problems that merit additional statistical and engineering review. We envisioned a surveillance process to develop a rank ordered problem list. A follow-up review process should start at the top of the problem list and work down through it, as resources permit. Problem ranking with our proposed statistic minimizes the "false positives" issue as it pushes these toward the bottom of the list. We understand this does not eliminate the issue. The only way we know to do that with certainty is to get rid of the surveillance.

We have never assumed that all relevant crash "factors" are coded perfectly in FARS and we said so in our conclusions. Throughout the paper, we also emphasized the relevance of this method to datasets other than FARS.

Like any good diagnostic test, the proposed methodology should be evaluated by a proven ability to detect a known issue early. That is why we studied an acknowledged problem. The FARS data show other potential issues of concern that were unknown, at least to us. See, for example, the fatal vehicle fire data shown in Figure 4.

Dr. Carra's letter points out that the first data from the agency's Early Warning Reporting system has been received by NHTSA and are currently being analyzed. The public may not be aware that these data for deaths, injuries, and property damage are currently being withheld from public health researchers outside the agency. As our paper points out, secret data were an essential ingredient in the original tragedy and ensuing scandal. This lesson seems not to have been learned.

Our proposed method to improve defect-related surveillance would have worked well to identify the Ford Explorer tire problem as a high priority concern for the agency. Our paper demonstrates this could have been known long before so many deaths and injuries resulted. See Figure 1.

Reference

1. Carra J. Unwarranted Assumptions about FARS data [electronic response to Whitfield and Whitfield; Improving surveillance for injuries associated with potential motor vehicle safety defects] injuryprevention.com 2004http://ip.bmjjournals.com/cgi/eletters/10/2/88#74

2. R A Whitfield and A K Whitfield. Improving surveillance for injuries associated with potential motor vehicle safety defects. Inj Prev 2004; 10:88-92.

Dear Editor

I read with interest the article by Whitfield and Whitefield that recently appeared in Injury Prevention.[1] I share their concern in identifying traffic safety issues as early as possible. Unfortunately the authors have made some unwarranted assumptions about the FARS data system and analysis based on it.

We do not find any problem with the statistics on tire problems or fires that were reported in the article. However, there is a major difference between identifying patterns in data of a known problem and identifying an unknown problem with patterns of data. It does not matter if a Poisson procedure [as used in this paper], control charts, or outlier analysis is used to identify the potential problems.

The authors have correctly pointed out that post-analysis of the FARS data, after a problem has been identified, shows the extent of a problem with tires and Ford Explorers compared to similar vehicles. A similar effort was completed by NHTSA.

However, any actual safety issues that could be identified a-priori through FARS would be scattered among literally thousands of combinations of vehicles and attributes. The authors examined two possible combinations knowing the problems were there. Not knowing that a safety defect exists in advance, the authors' recommended technique, or any other statistical technique, will produce many false positive results from the large number of possible safety issues. The authors have not addressed the problem of false positives.

The analysis also suffers from the assumption that if one of the related factors occurs in a crash, then it will be coded in FARS. This, however, is correct if and only if, the information is collected on police crash report and is available to the FARS analyst to code.

The FARS data is a national treasure that documents every fatal crash that has occurred on a public roadway since 1975. For close to thirty years, government, academia, and the private sector have used these data to evaluate a large variety of traffic safety related issues. However, FARS has its limitations. For this reason, in 2003 DOT undertook a new data collection effort, the Early Warning Reporting system to quickly identify safety issues within the fleet. The first data from that system have been received by the department and is currently being analyzed.

Reference

1. R A Whitfield and A K Whitfield. Improving surveillance for injuries associated with potential motor vehicle safety defects. Inj Prev 2004; 10: 88-92.