Methods

For this study, 1260 E coded hospital discharge records were sampled from the 1996 computerized hospital discharge data file maintained by the Washington State Department of Health. This computerized file included information on all inpatient discharges from acute care civilian hospitals in Washington. Computerized discharge records included in the study were those with an external cause of injury code (E800–E869, E880–E929, E950–E999), excluding codes for medical misadventures, complications or abnormal reactions related to medical care, or adverse effects of medication in therapeutic use (E870–E879, E930–E949). Sampling was made without regard to discharge status; thus fatal and non-fatal hospitalizations were sampled.

We restricted the sampling frame to 32 Washington hospitals that accounted for 80% of injury related hospitalizations in 1996. We selected discharge records from the computerized discharge data using probability proportional to size sampling.⁶

Injuries attributed to firearms and poisons were over sampled to ensure that the final sample contained enough intentional injuries to allow for meaningful analysis of intent coding. The vast majority of injuries are unintentional; firearm and poisoning injuries are the most likely to be intentional.

To assess the accuracy of hospital discharge data, a “gold standard” or “expert record” was developed by a medical record administrator trained in injury coding. Data for the expert record were obtained by the expert reviewer through on-site reviews of medical records at the 32 study hospitals. Information regarding the injury was abstracted from each medical record, including the face sheet, pre-hospital transport narrative, emergency department record, admission history and physical, nursing assessment, physicians' progress notes, and discharge summary. The record expert was blinded to the E codes reported in the hospital discharge records.

Injuries were assigned an E code based on ICD-9-CM guidelines, supplemental instructions issued by the Washington State Department of Health, and special rules in effect for 1996 as outlined in the official guidelines for coding and reporting published by the Department of Health and Human Services.^7,8

The expert records were keyed and linked with computerized hospital discharge records using a deterministic method that successfully matched all records. Twenty three hospital discharge records could not be located at the study hospitals and were eliminated from the sample. The probability weights of the sample were adjusted to reflect the elimination of these records.

The data available for analysis from the expert review and hospital discharge records included patient age, sex, discharge date, E code, and a response to the question “Is this the first hospitalization for this injury”. In addition, group codes for mechanism of injury and intent of injury were created for each hospital discharge and expert record using the complete E code and grouping guidelines recommended by the Centers for Disease Control (CDC).⁹

The list of E codes used to define the various categories of injury were identical to those presented in the CDC's recommended framework for injury classification.⁹ However, we made a separate category for E codes indicating late effects of injury: E929(.0.5), E929.8, E959, E969, or E989. For some categories of injury, the sample of records was too small to allow for meaningful analysis. Categories of injury were excluded from the results and discussion sections if the sample size was less than 15.

STATISTICAL ANALYSIS

We calculated (a) estimated counts of injuries by mechanism of injury; (b) incidence ratios by mechanism of injury; (c) estimates of sensitivity and predictive value positive and (d) inter-rater agreement (observed agreement and the kappa measure of agreement adjusted for chance).¹⁰ When appropriate, all calculations accounted for sampling weights, finite population corrections, and strata using linearization methods.⁶ STATA statistical software was used for the data analysis.¹¹

Results

We found that computerized hospital discharge data correctly ranked injuries in regard to both mechanism and intent (table 1). The 10 leading causes of injury hospitalization in rank order were falls, motor vehicle crashes, poisonings, late effects of injuries, injuries from being struck by or against an object, cuts, overexertion, transport injuries (other than motor vehicle traffic), machinery related injuries, and firearm injuries. The majority of these injuries were unintentional; a relatively small percentage was attributed to suicide attempts and assaults.

We compared the incidence of injuries from computerized hospital discharge records with the incidence based upon expert review of patient medical records. The estimated based upon computerized hospital discharge data was very similar to that based upon chart review: incidence rate ratio 1.0 (95% confidence interval (CI) 1.00 to 1.02) (table 1). Computerized hospital discharge data overestimated motor vehicle traffic injuries by about 10%, with most of these errors resulting from using motor vehicle traffic codes for injuries that should have been coded to some other mechanism, such as off-road injuries. Computerized data tended to overestimate “other” and “unspecified” injuries: approximately 20% of injuries assigned to these general categories could have been coded to a specific mechanism.

Nearly 20% of hospitalizations due to late effects of injury were misclassified in computerized discharge records as acute injuries, leading to inflated estimates of acute injuries. Among cases that were correctly classified as late effects of injury, the expert reviewer found that 95% of the patients had been previously hospitalized for the injury. Data needed to answer the question “Is this the first hospitalization for this injury” was available for 95% of the cases.

Computerized hospital discharge records under-reported injuries of undetermined intent by 48%. The incidence ratio comparing the computerized data with the expert review record was 0.52 (95% CI 0.15 to 0.90). These injuries were commonly misclassified as unintentional.

The overall amount of agreement between computerized hospital discharge data and records based upon expert review of actual medical charts was 87% for mechanism of injury and 95% for intent of injury. The amount of agreement was substantially lower (66%) for the complete E code. The sampling design employed for this study allowed in-depth analysis of coding for four categories of injury: falls, motor vehicle traffic injuries, poisonings, and firearm injuries. For each of these four injury mechanism groups, computerized hospital discharge records identified discharges for this mechanism, as defined by the expert review of medical records, with a sensitivity of 91% or better (table 2). Predictive value positive of the computerized data ranged from 88% for motor vehicle traffic injuries to 94% for poisonings. The most common type of error impacting the predictive value positive for all four subgroups was the misclassification of late effects of injury as acute injuries. The predictive value positive of motor vehicle codes was also impacted by misclassification of off-road injuries as motor vehicle traffic events. For poisonings, the predictive value positive of coding was further reduced by the use of accidental or intentional poisoning codes to classify adverse effects of medications in therapeutic use.

The amount of agreement for intent coding ranged from 84% (0.81 kappa) for firearm injuries to 99% (0.40 kappa) for falls (table 3). Disagreement on intent coding for firearm injuries reflected disagreement as to whether an incident was an accident, assault, or incident of undetermined intent. For poisonings the amount of agreement was 87% (0.72 kappa): the expert reviewer often classified a poisoning as unintentional when the computerized hospital discharge data classified the episode as suicide, despite the absence of evidence of either suicidal intent or depression in the medical record.

The complete E code contains details about an injury that are potentially useful for injury prevention and program planning. We found that the amount of agreement for coding of the complete E code ranged from 57% (0.50 kappa) firearm injuries to 72% (0.67 kappa) for poisonings. Based on these estimates, it appeared that computerized hospital discharge data lacked the precision needed to identify the status of the injured persons in motor vehicle traffic injuries (driver, passenger, pedestrian, etc); the circumstances of falls (for example, falls from one level to another, falls on the same level due to slipping or tripping); the drugs involved in poisoning incidents; and the role of handguns compared with long guns in firearm injuries.

Discussion

We found that computerized hospital discharge data can be used with confidence to determine how many injuries are treated in a hospital setting in Washington State and the relative magnitude of various categories of injury. We also found that E codes on hospital discharge records are a reliable source of information on the mechanism and intent of injury, the two types of information most often used for injury related analyses and priority setting. In comparing the coding of records by intent, we found that agreement regarding intent was excellent (98%) for injuries related to motor vehicle crashes, but the kappa statistic was zero, suggesting no agreement beyond chance. This arose because all of the computerized records for motor vehicle injuries were coded as unintentional; kappa will always be zero in this situation, regardless of the level of agreement. Similarly, agreement regarding the intent of fall related injuries was excellent (99%), yet kappa was only 0.40; in this extreme situation, where virtually all the injuries were unintentional, the kappa statistic is not very helpful.

We found that hospital discharge coders tend to overuse non-specific E codes. However, consistent with the findings of Langlois and associates,¹² we also found that deficiencies in chart documentation contributed to the use of non-specific codes. More complete documentation would enhance the quality and usefulness of injury data; specifically, more efforts need to be made to identify the status of the injured person in motor vehicle related incidents; establish the intent of injuries; and distinguish unintentional or suicidal poisonings from adverse effects of medications in therapeutic use.

The main limitation of the data was the lack of precision in the complete E code. This finding is consistent the study of Schwartz et al regarding the accuracy of E codes assigned to 108 emergency department records at a single hospital.¹³ Researchers who need injury data at the complete E code level should consider sources other than hospital discharge data. At a minimum, researchers must exercise caution in interpreting and using detail codes. It cannot be assumed that ICD-9-CM codes designated for very specific events and circumstances are used consistently or appropriately.

A recurring issue found in this study was the vague nature of the “late effects” category, and the impact that this has on injury estimates. The language in ICD-9-CM does not clarify which injuries are to be regarded as late effects and which are not. As a result, there was considerable interhospital variation in the use of these codes. Some hospitals coded any admission to a rehabilitation unit as a “late effects” admission, while others required some period of time to pass (anywhere from three to 12 months) before an injury admission was considered to be a late effect. A more useful distinction might be one that clearly identifies whether an admission is the first one related to a specific injury, as suggested by Smith and associates.¹⁴ This would allow a clearer identification of acute injuries, and therefore more accurate estimation of injury incidence from hospital discharge data.

The complexity of injury coding guidelines, combined with the fact that these rules are subject to change, suggests that medical record coders need periodic training on injury coding to stay abreast of current requirements and maintain a high level of coding accuracy. There are indications that training needs are not being met. During the course of our study the rules for coding intent of injury underwent a major change—before October, 1996 the instructions for coding intent stipulated that “if the intent (accident, suicide, assault) of an injury or poisoning is not specified or unknown, code the intent as accidental”. In October, the rules changed to require that such injuries be coded as intent undetermined. Subsequent discussions with medical record personnel revealed that most coders were unaware of these changes.

The findings of this study have implications for the quality of future injury data. In 1999, states implemented the ICD-10 classification scheme for mortality coding. In a few years, hospitals will adopt the ICD-10-CM for morbidity coding. The ICD-10 classification scheme for injuries will provide more injury codes, allowing injuries to be described in much greater detail than before. However, the existence of codes does not assure that these codes will be used consistently and accurately. To realize the promise of the ICD-10 injury classification scheme, work needs to be done to ensure that the creators of the classification scheme provide clear guidelines on their use; that health care professionals are trained and able to provide the needed documentation in medical records; and that hospital coders are trained to understand and apply the new injury coding requirements.