Article Text
PDF
Abstract
This retrospective study examined the accuracy of the International Classification of Diseases, Clinical Modification (ICD-10-CM) coding for physical child abuse among patients less than 18 years of age who were evaluated due to concern for physical abuse by a multidisciplinary child protection team (MCPT) during 2016–2017 (N=312) in a paediatric level I trauma centre. Sensitivity, specificity, predictive values and diagnostic OR for ICD-10-CM coding were calculated and stratified by admission status, using as a reference standard the abuse determination of the MCPT recorded in a hospital registry. Among inpatients, child physical abuse coding sensitivity was 55.6% (95% CI 41.4% to 69.1%) and specificity was 78.6% (95% CI 59.0% to 91.7%), with diagnostic OR of 4.58 (95% CI 1.64 to 12.70). Among outpatients, sensitivity was 22.2% (95% CI 15.5% to 30.2%) and specificity was 86.3% (95% CI 77.7% to 92.5%), with diagnostic OR of 1.80 (95% CI 0.89 to 3.64). Use of ICD-10-CM coded data sets alone for surveillance may significantly underestimate the occurrence of physical child abuse.
- child abuse
- epidemiology
- surveillance
- child
- hospital care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Introduction
Child maltreatment is a significant public health problem that affected an estimated 674 000 children in the US in 2017.1 Surveillance has historically relied on substantiated case rates from Child Protective Services (CPS), although hospital discharge data sets based on the International Classification of Diseases, Clinical Modification (ICD-CM) coding are also used as data sources for epidemiological studies of child abuse seen in hospital settings.2–4 Evaluating the accuracy of this hospital discharge data is important because abusive injury may be more prone to ICD coding errors than accidental injury.4 5 Published studies of the accuracy of ICD-9-CM coding for physical child abuse indicated a sensitivity of 61%–92%.5–8 Previous studies have identified at least two contributors to low sensitivity in child abuse coding: coder error and insufficient or unclear documentation in the medical record.5 6
Child abuse diagnosis codes were expanded under ICD-10-CM to include both confirmed and suspected abuse. What was previously one code in ICD-9-CM (995.54 Child physical abuse) became two codes under ICD-10-CM (T74.12 Child physical abuse, confirmed and T76.12 Child physical abuse, suspected).9 10 No published study to date has investigated the accuracy of ICD-10-CM coded hospital discharge database data for physical child abuse to evaluate the suitability of these databases for research or surveillance purposes. This study aimed to evaluate the accuracy of hospital discharge data, by comparing ICD-10-CM codes assigned after hospital discharge with known case outcomes documented by a multidisciplinary child protection team (MCPT).
Methods
Study design
This cross-sectional study was conducted using data from all patients evaluated for possible physical child abuse in 2016 and 2017 at a paediatric level I trauma centre. The study hospital uses an MCPT to evaluate the likelihood of abuse among patients brought to their attention through hospital protocol, clinician concern or outside referral. The MCPT is composed of a hospital-based team of child abuse clinicians and hospital social workers, as well as CPS caseworkers, law enforcement representatives and district attorney’s offices. Child abuse determinations made by this entity were considered the reference (‘gold’) standard for evaluation of abuse in the hospital (MCPT determination) and are maintained in a child abuse care team database separate from the study’s comparison outcome (ICD-10-CM codes). ICD-10-CM codes are assigned by the Health Information Management (HIM) department using medical record review following discharge.
The study included all patients in a hospital child abuse registry of MCPT evaluations who were less than 18 years of age and evaluated for possible physical child abuse, with hospital arrival dates during 2016–2017. Patients evaluated for other types of maltreatment (no physical abuse concern) were excluded.
Patient race and ethnicity as reported by the family and entered into the medical record were categorised for the study into four groups, with Hispanic patients of any race categorised as Hispanic and the remaining patients categorised as non-Hispanic white, non-Hispanic black or non-Hispanic other. Patients classified as inpatients included those admitted to any hospital ward or intensive care unit. Outpatients included those seen in the outpatient child abuse clinic and those seen in the emergency department and not admitted.
The MCPT determination was recorded as one of five options: abuse, concerning for abuse, indeterminate, not abuse and no opinion (or insufficient information to render determination). A dichotomous study variable was created by collapsing the abuse and concerning for abuse categories into a single category called abuse and categorising all other options as not abuse. This dichotomised MCPT abuse determination (abuse/not abuse) was used as the reference standard for all analyses.
ICD-10-CM codes were obtained from a hospital discharge database by matching records from the same hospital encounter in which the MCPT evaluation was performed. ICD-10-CM codes are entered into the discharge database by HIM department staff based on their reading of the medical record. The list of all ICD-10-CM codes assigned to each eligible patient was used to create a dichotomous variable for whether there was any code indicative of physical child abuse. Patients were categorised as positive for ICD-10-CM coding for physical abuse if they had any of the following predetermined codes: T74.12, T74.4, T74.92, T76.12 or T76.92, Y07 or Y09. The following accuracy measures of ICD-10-CM coding for physical child abuse in the discharge database were calculated, using the MCPT determination as the reference standard: sensitivity, specificity, PPV, NPV and diagnostic OR. Accuracy analysis was stratified by admission status.
Statistical analyses
Distributions of demographic variables were compared by final MCPT determination using χ2 tests for categorical variables or the Wilcoxon rank-sum test for continuous variables. Point estimates and 95% CIs for the overall sensitivity, specificity, PPV, NPV and diagnostic ORs were calculated using the Clopper-Pearson exact method for estimation of binomial proportions.11 12 Prevalence of abuse was reported because this number affects the PPV and NPV. However, this was the prevalence only within the narrowly defined study population, which was defined by hospital evaluation protocols. It is not the prevalence within the general population of the hospital or community. Diagnostic OR was chosen as a single measure of overall accuracy because it is relatively easier to interpret than some other related measures and is not influenced by prevalence.13 Exploratory analysis found similar accuracy measures between outpatients seen in the emergency department and those seen in the child abuse clinic, so they were not stratified further to avoid unnecessarily limiting subgroup sample size. All statistical analyses were performed using Stata V.12.1 software.14 A p value of 0.05 or less was considered significant for all statistical tests. The 2015 Standards for Reporting Diagnostic Accuracy guidelines were used in preparing this report.15
Results
Descriptive statistics
As shown in table 1, there were 312 patients identified, with 189 (60.6%) categorised by the MCPT as victims of physical abuse and 123 (39.4%) deemed non-abuse. There was a statistically significant difference in abuse categorisation by age group, with patients 4 years of age or older more likely to be categorised as abused and those less than 1 year of age more likely to be categorised as not abused (p=0.009). A significant difference in abuse categorisation was also noted by insurance type, with privately insured patients more likely to be categorised as not abused and those with all other insurance types more likely to be categorised as abused (p=0.003). No statistically significant difference in abuse categorisation was noted based on sex, race/ethnicity or admission status (table 1).
Descriptive statistics of patients evaluated by the multidisciplinary child protection team for possible physical abuse during 2016–2017 (n=312)
ICD-10-CM coding for physical child abuse
The frequencies of ICD-10-CM codes of interest for physical child abuse are detailed in table 2. The most frequently used ICD-10-CM code related to physical child abuse was T76.12 (Child physical abuse, suspected, n=56). Only 4.2% of the cohort received the supplementary ICD-10-CM code to show they had been examined or observed for possible abuse or neglect (Z04.72).
Frequency of ICD-10-CM code usage related to physical child abuse, 2016–2017
Among inpatients, ICD-10-CM coding for physical child abuse sensitivity was 55.6% (95% CI 41.4% to 69.1%) and specificity was 78.6% (95% CI 59.0% to 91.7%). Among outpatients, sensitivity was 22.2% (95% CI 15.5% to 30.2%) and specificity was 86.3% (95% CI 77.7% to 92.5%). Table 3 summarises the findings of the accuracy analysis.
Analysis of sensitivity, specificity, PPV and NPV for ICD-10-CM coding of physical child abuse in 2016–2017
Eighteen out of 19 false positives (94.7%) were classified as positive for ICD-10-CM coding based on the presence of a code for suspected child abuse (ICD-10-CM code T76.xx). One other false positive occurred when the patient was given a Y07 cause of injury code (which designated the perpetrator). Although this child was evaluated for physical abuse, only sexual abuse was confirmed. Reasons for false negative results (cases classified as physical abuse based on MCPT determination that did not receive an ICD-10-CM code for physical abuse) were not evaluated.
Discussion
Among inpatients, the ICD-10-CM coding sensitivity of 55.6% in this study was lower than the 60%–90% ICD-9-CM coding sensitivity previously reported from other institutions. The coding sensitivity for outpatients was even lower at 22.2%. There is little to compare this result with because similar literature is sparse; where outpatients are included in coding accuracy studies, they have not been reported separately from inpatients.7
Specificity in this study was also subjectively lower than previously reported using ICD-9-CM.5–8 The new ICD-10-CM codes for suspected physical abuse were the primary contributors to lower specificity. The addition of suspected abuse codes in ICD-10-CM was likely intended to provide more granularity to abuse diagnoses. However, in this hospital the suspected abuse code was often used for confirmed cases and the confirmed abuse code was rarely used. The suspected code was also used in cases where abuse was ruled out. These findings may indicate both hesitancy of coders to use the term ‘confirmed’ and confusion about whether the suspected abuse codes should be used any time a patient is evaluated for possible abuse or only when abuse is still suspected after that evaluation. Related to this is the finding that only 5% of the study population received ICD-10-CM code Z04.72 (Examination and observation following alleged physical abuse). Based on ICD-10-CM Official Guidelines for Coding and Reporting, all patients in this study population were expected to receive that code.10 Further scrutiny of the data revealed that an additional 18.5% of patients received a code Z02.9 (Administrative examination, unspecified). This generic code appears to have been used in place of the more specific examination code.
Given that the final MCPT determination was used as the reference standard, timing may be important in interpreting coding specificity in this study. Child abuse could have still been suspected at the time of patient discharge when ICD coding occurred, but later ruled out. It can thus be argued that some false positives were related to the study design. Regardless, any resulting effects on specificity are not as problematic from a surveillance standpoint as the low observed sensitivity which would result in significant undercounting of cases.
Study limitations
Despite efforts to ensure rigour in our study, limitations do exist. (1) This study did not attempt to fully ascertain what factors contributed to incorrect coding in each case. (2) Misclassification bias could have occurred in both the ICD coding and MCPT determinations. However, bias in the MCPT evaluation process that was used as the reference standard was minimised by the large-sized, broadly inclusive composition and high level of training of the team members. The choice of ICD codes for inclusion was similar to that of previous studies.5–8 Exploratory analysis not reported found that changing the list of codes did not substantially affect the findings. (3) Generalisability of findings is limited because the study population included patients seen in only one hospital and only those evaluated for possible abuse. (4) Finally, sample size was small for inpatients.
Conclusions
The low sensitivity observed should prompt caution before using ICD-coded hospital discharge data alone to estimate incidence of physical child abuse. The findings of this study may help justify the maintenance of hospital registries containing detailed information on physical abuse cases. Such registries require commitment of resources to maintain but may yield worthwhile improvements in data quality. It is important to note that any data sourced from hospitals, including registries, will be limited to cases where an injury resulted in the patient being taken for medical evaluation or treatment. Ongoing quality improvement efforts within hospital record-keeping and ICD coding systems or algorithms may help improve data accuracy. High-quality data and surveillance methods are critically important to guide child abuse prevention programmes within the hospital and the community to where they are needed most.
What is already known on the subject
Child abuse surveillance presents unique challenges, with reported cases often acknowledged as only the ‘tip of the iceberg’ or the ‘worst of the worst’.
International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) inpatient coding sensitivity from hospital discharge data for physical child abuse ranged from 61% to 92% in previous studies.
What this study adds
Accuracy of ICD coding for physical child abuse remains a concern with ICD-10-CM, despite creation of separate codes for confirmed and suspected abuse.
Ongoing quality improvement efforts and incorporation of multiple data sources are important to optimise child abuse surveillance.
Footnotes
Contributors There were no contributors who did not meet the criteria for authorship.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The contents of this publication are solely the responsibility of the authors and do not necessarily represent the official views of the CDC in the United States Department of Health and Human Services, the State of Colorado, the State of Kentucky, the State of Maryland or the State of Massachusetts.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Ethics approval Approval to conduct this study was obtained from Seton and University of Texas Institutional Review Boards.
Provenance and peer review Not commissioned; externally peer reviewed.