Illness severity adjustment for outcomes analysis: Validation of the ICISS methodology in all 821,455 patients hospitalized in North Carolina in 1996

doi:10.1016/S0039-6060(98)70119-9

Surgery

Volume 124, Issue 2, August 1998, Pages 187-196

Presented at the Fifty-ninth Annual Meeting of the Society of University Surgeons, Milwaukee, Wis, Feb 12-14, 1998.

https://doi.org/10.1016/S0039-6060(98)70119-9 Get rights and content

Abstract

Background: Previous work has demonstrated that the International Classification of Diseases 9th Revision (ICD-9) Based Illness Severity Score (ICISS) methodology developed by Rutledge and Osler can perform well in this role as a severity adjustment tool in trauma patients. The purpose of the present study was to extend this previous work to determine the ability of ICISS to predict outcomes in all types of hospitalized patients. Methods: The ICISS methodology was used to derive predictions of survival, length of hospital stay, and hospital charges in the entire study population. Results: A total of 821,455 hospitalized patients in North Carolina in 1996 had complete data available for analysis. The overall hospital mortality rate was 2.9%. ICISS was an accurate predictor of hospital survival in all hospitalized patients (accuracy 95.9%, sensitivity 97.2%, and specificity 52.7%.) The area of the receiver operator characteristic curve was 0.93. By adding age to the model, the area under the receiver operator characteristic curve increased to 0.95. ICISS also explained a large amount of the variance in hospital stay and charges (R² = 0.38 and 0.56, respectively, P < .0001). Conclusions: This study extends previous work suggesting that ICISS may be an important improvement over other presently available severity adjustment models. If these findings are confirmed in comparison with other predictive tools, ICISS may find an important place in assessing illness severity. (Surgery 1998;124:187-96.)

Section snippets

Data source

Data for this study were obtained from a commercial medical data and information company that develops and markets clinical and financial decision support systems used by hospitals, integrated delivery systems, managed care organizations, employers, and pharmaceutical manufacturers. This database has been used in many studies to benchmark clinical performance and outcomes, profile best practices, and manage the cost and delivery of health care.

Patient selection

The patients selected for this study included each

Prediction of survival

Analysis of the association of ICISS with survival was performed using the SAS logistic regression procedure. The results of the SAS logistic regression analysis are shown in Fig 1 and Table I.

. SAS LOGISTIC procedure results for ICISS.

There were 821,455 patients hospitalized during 1996 who had complete data available for analysis in the study population. There were 798,142 survivors (97.2%) and 23,313 deaths (2.8%) in the study population. The measures of model fit indicate that ICISS is an

Discussion

In many health care marketplaces, outcomes assessment is central to monitoring quality while controlling costs. Increasingly, health care providers are being evaluated and held accountable for their patients' outcomes.¹⁹ Outcome assessments need to be adjusted for patient illness severity, with the goal of accounting for pertinent clinical characteristics before drawing inferences about the effectiveness or quality of care. Risk adjustment, case-mix adjustment, or severity adjustment methods

Conclusions

The national drive to assess the quality of health care necessitates accurate methods of quantitating and controlling for illness severity to assure fair comparisons and to avoid incorrectly concluding that poor outcomes are the result of substandard care, rather than severe illness. Previous work has demonstrated that ICISS can perform well as a severity adjustment tool in trauma patients. The purpose of this study was to determine the ability of ICISS to predict outcomes in all types of

References (35)

BR Motheral et al.
The use of claims databases for outcomes research: rationale, challenges, and strategies
Clin Ther
(1997)
LI Iezzoni
Using risk-adjusted outcomes to assess clinical practice: an overview of issues pertaining to risk adjustment
Ann Thorac Surg
(1994)
E Faltermayer
Will the cost cutting in health care kill you?
Fortune
(1994)
S Greenfield
The state of outcome research: Are we on target?
N Engl J Med
(1989)
AR Tarlov
Multiple influences propel outcomes field
Med Outcomes Trust Bull
(1995)
FA Connell et al.
The use of large databases in health care studies
Annu Rev Public Health
(1987)
G Kolata
New frontier in research: mining patient records
New York Times
(1994 Aug 9)
NJW Lewis et al.
The role of insurance claims databases in drug therapy outcomes research
PharmacoEconomics
(1993)
KN Lohr
Use of insurance claims data in measuring quality of care
Int J Technol Assess Health Care
(1990)
J Green
Problems in the use of outcome statistics to compare health care providers
Brooklyn Law Rev
(1992)

JP Kassirer

The use and abuse of practice profiles

N Engl J Med

(1994)

Office of Technology Assessment

(1994)

LI Iezzoni et al.

Predicting in-hospital mortality for stroke patients: results differ across severity-measurement methods

Med Decis Making

(1996)

M Shwartz et al.

Do severity measures explain differences in length of hospital stay? the case of hip fracture

Health Serv Res

(1996)

R Rutledge et al.

Injury severity grading in trauma patients: a simplified technique based upon ICD-9 coding

J Trauma

(1993)

EW Lang

Outcome after severe head injury: an analysis of prediction based upon comparison of neural network versus logistic regression analysis

Neurol Res

(1997)

R Rutledge

Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients

J Trauma

(1997)

Cited by (13)

Trauma Scoring
2008, Current Therapy of Trauma and Surgical Critical Care
The National Pediatric Trauma Registry: A Legacy of Commitment to Control of Childhood Injury
2004, Seminars in Pediatric Surgery
The National Pediatric Trauma Registry represents almost 15 years of effective collaboration among hospitals committed to improving care for the injured child. Its design of providing a “physiologic snapshot” of the injured child on presentation has supported numerous studies that have helped define the epidemiology of childhood injury and refine principles of management. Global analysis of the 103,434 records included in this database suggest that mortality is significantly higher in the very young, that vehicular injury remains a major pediatric public health challenge, and that shock is just as devastating in the child as the adult. Based on this foundation of collaborative commitment, future versions of a pediatric trauma database must harness the emerging internet technology that combines information accrual with human thought, and must extend this effort to include all the children of our world.
A multicenter evaluation of whether gender dimorphism affects survival after trauma
2001, American Journal of Surgery
Background: The frequency of women who have sustained severe injuries has increased over the past 30 years. The purpose of this study was to evaluate whether severely injured women have a survival advantage over men. To address this issue, we undertook a multicenter evaluation of the effects of gender dimorphism on survival in trauma patients.
Methods: Patient information was collected from the databases of three level I trauma centers. We included all consecutive patients who were admitted to these centers over a 4-year period. We evaluated the effects of age, gender, mechanism of injury, pattern of injury, Abbreviated Injury Score (AIS), and Injury Severity Score (ISS) on survival.
Results: A total of 20,261 patients were admitted to the three trauma centers. Women who were younger than 50 years of age (mortality rate 5%) experienced a survival advantage over men (mortality rate 7%) of equal age (odds ratio 1.27, P <0.002). This advantage was most notably found in the more severely injured (ISS >25) group (mortality rate 28% in women versus 33% in men). This difference was not attributable to mechanism of injury, severity of injury, or pattern of injury.
Conclusions: Severely injured women younger than 50 years of age have a survival advantage when compared with men of equal age and injury severity. Young men have a 27% greater chance of dying than women after trauma. We conclude that gender dimorphism affects the survival of patients after trauma.
Establishing an injury indicator for severe paediatric injury
2017, Injury Prevention
Systematic review of risk adjustment models of hospital Length of Stay (LOS)
2015, Medical Care
Intramedullary lesion expansion on magnetic resonance imaging in patients with motor complete cervical spinal cord injury. Clinical article
2012, Journal of Neurosurgery: Spine

View all citing articles on Scopus

^☆: Reprint requests: Robert Rutledge, MD, FACS, Associate Professor of Surgery, Department of Surgery, University of North Carolina at Chapel Hill, Campus Box 7210, Burnett-Womack Building, Chapel Hill, NC 27599-7210.

View full text

Society of University SurgeonsIllness severity adjustment for outcomes analysis: Validation of the ICISS methodology in all 821,455 patients hospitalized in North Carolina in 1996☆

Abstract

Section snippets

Data source

Patient selection

Prediction of survival

Discussion

Conclusions

Clin Ther

Ann Thorac Surg

Will the cost cutting in health care kill you?

Fortune

The state of outcome research: Are we on target?

N Engl J Med

Multiple influences propel outcomes field

Med Outcomes Trust Bull

The use of large databases in health care studies

Annu Rev Public Health

New frontier in research: mining patient records

New York Times

The role of insurance claims databases in drug therapy outcomes research

PharmacoEconomics

Use of insurance claims data in measuring quality of care

Int J Technol Assess Health Care

Problems in the use of outcome statistics to compare health care providers

Brooklyn Law Rev

The use and abuse of practice profiles

N Engl J Med

Predicting in-hospital mortality for stroke patients: results differ across severity-measurement methods

Med Decis Making

Do severity measures explain differences in length of hospital stay? the case of hip fracture

Health Serv Res

Injury severity grading in trauma patients: a simplified technique based upon ICD-9 coding

J Trauma

Outcome after severe head injury: an analysis of prediction based upon comparison of neural network versus logistic regression analysis

Neurol Res

Comparison of the Injury Severity Score and ICD-9 diagnosis codes as predictors of outcome in injury: analysis of 44,032 patients

J Trauma

Society of University Surgeons
Illness severity adjustment for outcomes analysis: Validation of the ICISS methodology in all 821,455 patients hospitalized in North Carolina in 1996☆