Article Text
Abstract
Background Classification of injuries and estimation of injury severity on the basis of ICD-10 injury coding are powerful epidemiological tools. Little is known about the characteristics and consequences of primary coding errors and their consequences for such applications.
Materials and methods From the Swedish national hospital discharge register, 15 899 incident injury cases primarily admitted to the two hospitals in Uppsala County between 2000 and 2004 were identified. Of these, 967 randomly selected patient records were reviewed. Errors in injury diagnosis were corrected, and the consequences of these changes were analysed.
Results Out of 1370 injury codes, 10% were corrected, but 95% of the injury codes were correct to the third position. In 21% (95% CI 19% to 24%) of 967 hospital admissions, at least one ICD-10 code for injury was changed or added, but only 13% (127) had some change made to their injury mortality diagnosis matrix classification. Among the cases with coding errors, the mean ICD-based injury severity score changed slightly (difference 0.016; 95% CI 0.007 to 0.032). The area under the receiver operating characteristics curve was 0.892 for predicting hospital mortality and remained essentially unchanged after the correction of codes (95% CI for difference –0.022 to 0.013).
Conclusion Errors in ICD-10-coded injuries in hospital discharge data were common, but the consequences for injury categorisation were moderate and the consequences for injury severity estimates were in most cases minor. The error rate for detailed levels of cause-of-injury codes was high and may be detrimental for identifying specific targets for prevention.
- Injury
- registry
- ICD-10
- healthcare surveys
- external cause of injury
- database
- e-code
- methods
Statistics from Altmetric.com
Introduction
Monitoring the burden of injury is an important public health issue.1 The nature and severity of injuries can be extracted from complete hospital discharge data.2 Such population-based data are of immense value in public health research, provided that the hospital discharge diagnoses generate reliable categorisation of injuries and accurate estimates of injury severity. Although misclassification and coding errors are likely, the crucial question is not only the extent of such errors, but also the consequences in the practical application of this type of data in injury research. Two major applications of these codes are injury categorisation and injury severity estimation, necessary for case-mix adjustments.
A limited number of studies have investigated the accuracy of injury coding based on the International Statistical Classification of Diseases and Related Health Problems (ICD), and the results from these studies are variable. In some settings, the validity of hospital discharge diagnoses has been disappointing.3 4 The frequency of errors in injury coding actually appears to be higher with ICD-10 than with ICD-9 when evaluated in the same setting.5 6 Special interest has been given to external-cause coding. Few studies have targeted ICD-10 coding, but the results indicate that external-cause codes may present specific problems.7–11
The aims of this study were to quantify the extent of coding errors of injury diagnoses and causes of injury, describe their nature, and estimate the consequence of these errors for injury categorisation and injury severity estimations.
Materials and methods
Data
The Swedish National Patient Registry (NPR) covers all in-patient care in Sweden and contains a principal diagnosis (which should reflect the main reason for hospital admission) and up to seven secondary diagnoses from all patients discharged from hospital. Coding of the diagnoses in the NPR is mainly performed by the physician responsible for the care of the patient at the time of discharge from hospital.
Injury hospitalisations were defined as hospital admissions with a principal diagnosis in the range S00–T80 but excluding adverse effects (T78) and poisoning (T36–T65, T96 and T97) as listed in ICD-10.12 Incident hospital admissions were selected on the basis of a validated prediction model.13 In order to eliminate observational stays due to poorly defined conditions of low severity, patients discharged alive with a hospital length of stay of only 1 day or less were excluded. The dominating principal diagnoses in this group were concussion (S060, 27%), fracture of lower end of radius (S525, 4%), and poisoning by other and unspecified drugs (T509, 2%).
These criteria identified 519 151 hospital admissions for incident injury in Sweden during the years 2000–2004. Of these admissions, 15 899 were treated primarily in either of the two hospitals in Uppsala County. Uppsala University Hospital is an 1100-bed tertiary care facility, and Enköping Hospital is a 90-bed local hospital. Together they serve a population of 302 564 (as of 31 December 2004). A random sample of 1000 admissions was drawn without replacement from the injury admissions to these two hospitals. Of these, 10 (1%) admissions that should have had a non-trauma principal diagnosis (infection or cancer), and seven (0.7%) admissions representing readmission for a previous injury, were excluded from the study. For 16 (1.6%) patients, we were unable to locate the patient records, and they were consequently also excluded from the study. The final study population therefore consisted of 967 hospital admissions for injury. The study was approved by the regional human ethics committee.
Verifying injury diagnoses in hospital records
One of the authors (MFB), who is an experienced intensive care unit nurse, member of the hospital trauma committee and certified ATCN (Advanced Trauma Care for Nurses) instructor, reviewed all patient records. The diagnoses recorded in the NPR were verified against the discharge summary, progress notes, x-ray results and nursing records. In the Swedish version of ICD-10, chapters 19 and 20 correspond strictly to the international ICD-10 version and its rules for coding were followed. The reassessment of coding made by MFB was considered to be the ‘gold standard’ for assessment of coding accuracy. Incorrect diagnoses were removed or changed as appropriate. Missing diagnoses were added. In cases of uncertainty (n=7), RG who is an experienced critical care physician and researcher in injury epidemiology, was consulted and a consensus decision was made.
Measuring consequences of misclassification
We assessed the consequences of misclassification by looking at the changes in classification of broader injury categories according to ICD-10 injury mortality diagnosis (IMD) matrix.14 The number of injury cases that were reclassified according to the IMD matrix because of changed, added or removed ICD codes was counted. In a similar manner, the consequences of corrected cause-of-injury codes were assessed by categorisation in the cause-of-injury matrix developed by the National Center for Health Statistics, Centers for Disease Control (CDC), USA.15 16 In ICD-10, the fourth position of external-cause codes W00–Y34 (except Y06 and Y07) is used to define the place of injury when relevant. In transport injuries, the fourth position instead defines traffic/non-traffic injuries for pedestrians, the role of the occupant in vehicle crashes, or the type of vehicle involved in water and air transport injuries. For codes V01–Y34, the fifth position can be used to define the person's activity at the time of injury.
The International Classification of Diseases Injury Severity Score (ICISS) has been shown to perform well compared with other injury severity scores17–20; it is calculated on the basis of diagnosis-specific survival probabilities (DSPs) for individual injury ICD codes. This ratio represents the proportion of patients with a specific injury code who survived until hospital discharge. To assess the impact of corrected codes on the predictive capacity of ICISS for hospital mortality, ICISS was calculated using DSPs based on all injury hospitalisations and prehospital injury deaths during 1998–2004 in Sweden by linking NPR to the Swedish Cause of Death Registry (CDR) using the unique personal identification number that is given to all Swedish citizens.21 22 The ICISS (survival probability) for the individual patient was calculated as the product of each of the DSPs corresponding to the patient's injuries (ie, the product of the probabilities of surviving each of their injuries individually). This was first done on the basis of the original codes, and then also after codes had been corrected after review of patient records. For the purpose of description, ICISS was categorised as critical (0–0.219), severe (0.220–0.354), serious (0.355–0.664), moderate (0.665–0.940) or minor (0.941–1.0). The choice of these cut-offs was based on an examination of a logit plot of a smoothed function of ICISS versus mortality in 50 000 injury events randomly selected from the NPR and CDR during the years 1998 to 2004 (General Additive Model, mgcv package in R version 2.7.0; R Foundation for Statistical Computing, Vienna, Austria). Cut-offs for categories were selected with the aim of minimising the difference in mortality risk within categories and maximising the change in mortality risk between categories.
Inter-rater reliability
Fifty patients were randomly selected from the hospital admissions where researcher MFB had made independent decisions on the accuracy of the codes. These patient records were reviewed and their diagnoses checked by RG, who was unaware of the results achieved in the main validation process by MFB.
In this inter-rater reliability assessment, there were 65 of 71 diagnoses (92%) considered correct by both researchers, two considered wrong by both researchers, and the researchers disagreed on the accuracy of four diagnoses (6%): correction of ‘Injury of multiple nerves at lower leg level’ (S847) to ‘Injury of peroneal nerve at lower leg level’ (S841); removal of an unspecific code (S729) when a more specific code (S720) was present as well; change of ‘Unspecified injury of head’ (S099) to ‘Concussion’ (S060); and finally correction of ‘Dislocation of patella’ (S830) to ‘Fracture of patella’ (S820).
A blind comparison of the two researchers' re-evaluation of cause-of-injury codes yielded a higher proportion of discordant assessments. Among 50 injury cases, 20 (40%) were considered correct by both researchers, 17 (34%) were considered wrong by both researchers, and the researchers disagreed on the accuracy of 13 causes of injury (26%). Six concerned classification of place and/or activity, six the type of fall, and one the type of motorcycle crash. None of these discrepancies between the two researchers affected the categorisation of cause according to the CDC matrix.
Statistical methods
Baseline characteristics were categorised and compared using the χ2 test. A 95% CI was calculated for proportions. The distribution of ICISS is skewed, and results are therefore presented as medians with IQR. The difference in median ICISS resulting from correction of injury diagnoses was calculated based on the Wilcoxon signed-rank statistic for paired data.
The ability of ICISS to predict hospital mortality was assessed in logistic regression models adjusted for age and sex. Age and ICISS were entered as restricted cubic splines in these models. The area under the receiver operating characteristic curve (c-statistic) was calculated as a measure of predictive capacity. Bootstrap samples from the original dataset with 1000 replications and the percentile method were used to calculate a 95% CI for the difference in c-statistics. The same method was used to calculate 95% CIs for the differences in proportions of specified place and activity derived from cause-of-injury codes.
The statistical packages SAS V.9 and R V.2.7.0 (R Foundation for Statistical Computing) were used for data management and statistical analyses.23
Results
The 967 hospital admissions for injury generated a total of 1370 unique injury codes, and 10% (95% CI 9% to 12%) of these were corrected after review of the patient record. Most commonly, the correction of a diagnosis was in the fourth position in the code (table 1).
Of the injury codes, 95% were correct to the third position. In 91% (43/47) of those where a correction was made in the first position, this correction represented removal of a diagnosis code. Secondary diagnoses had a higher error rate than principal diagnoses (table 1). As an example, 96% of principal diagnoses were correct to the third position, compared with 92.1% of secondary diagnoses (95% CI for the difference 0.8% to 7.0%).
When divided into major injury categories, codes for thoracic injuries were most prone to errors, with almost 18% judged to be completely incorrect (table 2).
In 207 of 967 (21%; 95% CI 19% to 24%) hospital admissions, at least one ICD-10 code for injury was changed or added. These cases were on average younger patients with more severe injuries (table 3). If the 10 excluded admissions that should have had a non-trauma principal diagnosis (infection or cancer) had been included in the study population, the estimate of this proportion would have been identical.
A high proportion of transportation injuries had several injury diagnoses, and they had on average higher injury severity. At least one (and up to six) new ICD-10 injury codes were added in 11% (104/967) of the hospital admissions.
Owing to the nature of the errors in the corrected diagnoses, with most errors occurring in the fourth position in the code, only 13% (127) of the hospital admissions had some change made to their IMD matrix classification. The corrections necessitated the use of at least one additional cell of the IMD matrix to classify the injuries in 12% (95% CI 10% to 14%) of the cases, and negated the use of the cells in the matrix in 5% (95% CI 3% to 6%) of cases.
Although approximately one-fifth of the hospital admissions had some correction to injury diagnoses, owing to the nature of the errors, they had only minor impact on the ability to predict mortality. In some cases, correction of injury codes resulted in a substantially increased ICISS (reflecting a lower estimated injury severity), but, overall, the changes in ICISS appeared to be minor (figure 1). In the subpopulation of 207 cases where the review of the patients' records resulted in a changed, removed or added injury diagnosis code, injury severity changed from the median ICISS of 0.947 (IQR 0.865 to 0.987) before recoding to 0.963 (IQR 0.899 to 0.986) after recoding (difference 0.016; 95% CI 0.007 to 0.032). This difference did not, however, affect the discrimination in a logistic regression model of hospital mortality using age, sex and ICISS as independent variables. The c-statistic was 0.892 for predicting hospital mortality both with the original coding and after correction of codes (95% CI for difference –0.022 to 0.013).
Regarding cause of injury, only one out of all 967 admissions had a missing cause-of-injury code. However, in only 50.6% (489) of the hospital admissions were the causes of injury deemed to be correctly coded. The majority of these errors were in the fourth and fifth position of the code, which should reflect place and activity, except for transport accidents, where the fourth position instead defines traffic/non-traffic injuries for pedestrians, the role of the occupant in vehicle crashes, or defines the type of vehicle involved in water and air transport injuries. Thus 82% were correct to the third position of the code (table 2). This pattern appeared to be largely consistent when examined for different cause-of-injury categories (table 4). The most common error was to code falls as unspecified (W19) even though such information was present in the patient record. Another common error was failure to provide coding concerning whether the occupant of a vehicle in a traffic crash was a driver or a passenger (fourth character in the code for motor vehicle traffic crashes) (table 4). These errors had only minor effects on the classification of cause of injury according to the CDC matrix. Only 6% (55/966) were reclassified to a different ‘cause’ category and 0.6% (6/966) to a different ‘intent’ category. The proportion with a specified entry in the fourth position of the code (place/type of pedestrian injury/occupant role in vehicle crash) was substantially increased after review of records, while the increase in specified activity was less substantial (table 5). The most common errors in the fourth and fifth position of the code were that place and activity or the role of the vehicle occupant were coded as unspecified when information existed in the patient record to determine this. This was especially evident for machinery-related injury but was also prominent for falls and motor vehicle and cyclist injuries.
Discussion
In this reassessment of patient records from a random subset of hospitalised injuries, one-fifth of the hospital admissions had at least one ICD-10 code for injury changed or added. The majority of errors were in the fourth character position of the code, and the consequences for broader injury classification were therefore moderate. The median level of the ICD-based injury severity score changed slightly among those where codes were corrected, but not the discriminative ability of the score. While errors in the more detailed levels of the codes have less impact on broad categorisation of injuries or injury severity estimates, such errors may be quite detrimental to specific research objectives that require a high level of detail.
When categorised by body region, codes referring to thoracic injuries had notably lower accuracy. The most common error was failure to code for multiple fractures of thoracic vertebrae or ribs (coded as S220/S223 instead of S221/S224). The finding that hip fractures had a higher error rate to the third position than other types of extremity fractures was expected. A detailed classification of femoral neck, trochanteric and subtrochanteric fractures (ie, hip fractures) is associated with higher apparent error rates, probably because of difficulties in classifying fractures located on the border between poorly defined anatomical regions.24
Cause-of-injury codes had an even higher proportion of errors, with only half being correct. Because errors in the fourth and fifth position of the cause-of-injury code dominated, the effects on the broader classification according to the CDC matrix were only minor. Only 6% resulted in a different ‘cause’ category and it was rare that the ‘intent’ category was changed. While the majority of errors in the external-cause codes may be considered minor and without serious consequences in many applications, the errors may be detrimental for in-depth studies aiming to identify targets for prevention. In a large proportion of cases, there was information on place, in particular, and also activity in the patient record; however, this was not reflected in the code.
The present study is a thorough record review of population-based, randomly selected hospital injury admissions. The inter-rater check indicated overall consistency in the assessments by the coders, and the discrepancies identified were mainly in the fourth position of the code. A particular strength of this study was the reporting of not only the extent and characteristics of ICD-10 coding errors to both principal and secondary diagnoses, but also the description of the consequences of these errors to injury classification and injury severity estimates.
There may be notable differences in the way coding is carried out in different countries. In Sweden, it is mainly the physician responsible for the discharge of the patient that performs the coding of discharge diagnoses, not professional coders. The fact that the proportion of errors appeared to be lower among injury diagnoses and higher for coding of the external cause may be related to these differences, in that the clinician has good first-hand knowledge of clinical data but may pay less attention to the cause and mechanism of injury.
Our decision to exclude 10 cases that turned out to have a non-trauma primary diagnosis can be questioned. Inclusion of these cases does not, however, change the estimated proportions to any meaningful degree. Another possible limitation of our study is that regional variation in coding practices cannot be excluded. While a record review of 967 injury admissions is a substantial commitment, the diverse types of injuries represented in a mixed ‘all injury’ population means that some types of injuries may be poorly represented. Specific patterns of misclassification and poor coding may not be revealed. The fact that coding in Sweden is mainly performed by the physicians without the aid of professional coders may limit extrapolation to settings with other systems. However, despite such fundamental differences in the organisation of coding, there are similarities in the findings to a comparable study from New Zealand that focused on principal injury diagnoses and identified 14% with errors in the first, second or third position.5 This appears lower than the overall proportion of 21% in our study; however, when limited to errors only in the first, second or third position, only 5.2% of all injury diagnoses in the present study had such errors. If we further limited the analysis in the present study to the principal injury diagnosis, the admissions with a changed diagnosis in the first three positions were 4% and 10% if extended to the fourth position.
Previous studies on ICD-9 have shown external cause-of-injury codes from hospital discharge records to be reliable for categorisation of mechanism and intent of injury, but also that there is a higher error rate in the part of the code that identifies detailed information on the circumstances of injury.25 In the New Zealand studies on injury diagnoses, the proportions of codes with errors were higher with ICD-10 than those reported for ICD-9.5 6
In an Australian study on ICD-9-coded injuries, the overall proportion of injury diagnosis codes with errors was higher than that found in our study; however, similarly to our findings, most errors were minor.26 The proportion of external-cause codes with errors was substantially lower in the Australian study. Comparisons with studies performed with ICD-9 are, however, difficult because of the fundamental differences between the two systems concerning injury coding.27 The ICD-10 system for coding circumstances of injuries is not without problems and contains inbuilt ambiguities such as for coding of place and activity.7 A more detailed coding system may also apparently result in more coding errors simply because there are more choices to make,28 but it is then of greater interest to compare the consequences with practical applications such as broader injury classifications and injury severity estimates.
On the basis of the findings in this study that coding errors could be detected and corrected by a thorough review of the patients' records, we envisage that coding accuracy in healthcare data could be improved. Automated coding assistance built into computerised systems for routine documentation of healthcare and real-time verification of coding accuracy may be the most cost-efficient ways to improve the quality of routinely collected healthcare data. In conclusion, errors in ICD-10-coded injuries in hospital discharge data were common, but most were not severe. The consequences for categorisation were moderate and the consequences for injury severity estimates were in most cases minor. The error rate for place and activity injury codes was very high and may be detrimental for identifying specific targets for prevention.
What is already known on this subject
ICD-10 coding of injuries in hospital discharge data is a powerful epidemiological asset for classification of injuries and estimation of injury.
What this study adds
While errors in ICD-10-coded injuries in hospital discharge data are common, the consequences of these errors for categorisation of injuries are moderate and the consequences for injury severity estimates are in most cases minor.
The error rate is high for detailed levels of cause-of-injury codes. This has little consequence for broader classification of cause of injury but may be detrimental for identifying specific targets for prevention.
References
Footnotes
Funding Uppsala University. The study sponsor had no role in the study design, in the collection, analysis and interpretation of data, in the writing of the report, or in the decision to submit the paper for publication.
Competing interests None.
Ethics approval This study was conducted with the approval of the Regional Human Ethics Committee, Uppsala, Sweden.
Provenance and peer review Not commissioned; externally peer reviewed.