Article Text
PDF
Introduction
Firearm injuries are a public health crisis in the US. The National Death Index (NDI) is a well-established, comprehensive database managed by the National Center for Health Statistics at the CDC. In this methodology paper we describe our experience accessing and linking data from the NDI to our regional, hospital-based violent injury database to identify out-of-hospital deaths from firearms.
Methods We outline the key steps of our submission to the NDI. Data were collected from research team meeting notes, team member emails with NDI staff, and information provided from the NDI website and supplementary guides. Few of our collaborators or university partner investigators had accessed or used data from the NDI. We discuss the online NDI Processing Portal data request, data preparation and receipt from the NDI, troubleshooting tips, and a timeline of events.
Results Our query to the NDI returned 12 034 records of 12 219 firearm-injured patient records from 2010 and 2019. The record match rate was 98.5%.
Discussion Linking hospital-based data sets with NDI data can provide valuable information on out-of-hospital deaths. This has the potential to improve the quality of longitudinal morbidity and mortality calculations in hospital-based patient cohorts. We encountered logistic and administrative challenges in completing the online NDI Processing Portal and in preparing and receiving data from the NDI. It is our hope that the lessons learnt presented herein will help facilitate easy and streamlined acquisition of valuable NDI data for other clinical researchers.
What this study adds:
- A step-by-step guide for clinical researchers of how to apply to access data from the National Death Index (NDI).
- Advice and lessons learned on how to efficiently and effectively access data from the NDI.
- A well-described methodology to improve the quality of longitudinal morbdity and mortality calculations in hospital-based cohorts of firearm injured patients.
What is already known on this subject:
- There is a need for robust, longitudinal data sources that reliably track morbidity and mortality among firearm injured patients in the United States.
- The NDI is a well-established, comprehensive database that holds death records for all 50 states, which provides valuable mortality data to the public health and medical research community.
- public health
- firearm
- process/impact evaluation
- recidivism
- epidemiology
Statistics from Altmetric.com
Background
Firearm injuries are a public health crisis in the US.1–6 Between 2010 and 2019, the CDC reported 358 946 deaths resulting from firearm injuries, at a rate of 11.25 per 100 000 people.7 During this 10-year period, the CDC estimated a total of 981 135 non-fatal firearm injuries.8 However there is significant variability in the precision of these data due to data quality issues and fragmentation of data across multiple databases at the national, state and regional level.8 9 Victims of assault and penetrating trauma, such as stabbings and shootings, are likely to be revictimised and to perpetrate violence against others.10 Additionally, prior studies have found surviving a firearm injury is one of the leading risk factors for suffering a subsequent firearm injury.11 12 A recent California state-level population-based retrospective cohort study found that 4.1% of patients who presented to the emergency department (ED) or hospital with a non-fatal firearm injury suffered a recurrent firearm injury.13 These recurrent firearm injury calculations may have been limited by a lack of longitudinal data linkage to patients who died in the prehospital environment.
Central to this issue is that a comprehensive national database for patients with repeat firearm injuries who survive transport to the hospital does not exist. State-wide trauma registries do not reliably capture patients who are discharged directly from the ED without a trauma service evaluation, nor do they regularly link with other data sources that can reliably identify patients who die from firearms outside the hospital setting. These limitations in extant data collection highlight the need for a comprehensive national data infrastructure to longitudinally track fatal and non-fatal firearm injuries. As things currently stand, the true burden of acute and recurrent firearm injuries in the US is unknown. Development of accurate baseline data in this field is essential to accurately assess the impact of regional, state and national violence prevention interventions.
The National Death Index (NDI) can help fill this gap in out-of-hospital death data. NDI data are critical to assess the trajectory of non-fatal and fatal firearm injury and to design effective interventions, and to assess their impact on recurrent injury prevention. The NDI is a well-established, comprehensive database managed by the National Center for Health Statistics at the CDC.14 The purpose of the NDI is to provide the public health and medical research community with an opportunity to obtain mortality follow-up information on their study participants, without having to go from jurisdiction to jurisdiction. It holds all death records from 1979 onwards for all 50 states and contains data on patient identifiers, dates of death, International Classification of Diseases code cause of death and names of states where those deaths occurred. Prior studies have highlighted the benefits of linking law enforcement and criminology databases with the NDI to investigate gang-related mortality in the US and in linking single-centre hospital data with the NDI to investigate overall survival rates in patients with cancer and diabetes.15 16
Objective
In this methodology paper we describe our experience accessing and linking data from the NDI to our regional, hospital-based violent injury database to evaluate out-of-hospital deaths from firearms. To accurately evaluate recurrent firearm injury rates and complete time-to-event (reinjury) analyses, we queried the NDI for death data for select firearm-injured patients in the St Louis Hospital-Based Violence Intervention Program Data Repository (STL-HVIP-DR). The NDI recently launched an online submission portal and our research team was among the first to apply for data access in this way. Our objective in sharing these findings is to have our experiences serve as a primer for other clinical investigators and institutions seeking to link their data with the NDI.
Methods and findings
In the following section, we outline the key steps and timeline of our submission to the NDI. Data presented herein were collected between 2019 and 2021 from field notes and minutes from research team meetings, team member emails (including correspondences with NDI staff), and information provided from the NDI website and supplementary guides.
The St Louis Hospital-Based Violence Intervention Program Data Repository
To address the need to better identify and track violent injuries in the St Louis region, our research team helped design and implement the STL-HVIP-DR.17 This repository contains patient-level data for all patients who received care for a violent injury at one of the four STL-HVIP-DR partner level I trauma hospitals with a violent injury from 2010 onwards. This includes over 10 000 visits for acute firearm injuries between 2010 and 2019. This repository relies on a data sharing agreement (DSA) that is in effect to facilitate data sharing across all hospital sites, their affiliate universities and the Missouri Hospital Association (MHA). The MHA already collects hospital data on behalf of its member hospitals and agreed to provide quarterly data updates to the repository. Additionally, MHA provides a unique person-level identifier to link data across hospital sites. Broadly, the repository was designed to support research projects that would help to fill the local epidemiological and research gap related to violence and violence prevention. This includes calculation of region-wide recurrent firearm injury rates and time-to-event (reinjury) analysis.
Planning to apply to the NDI
In pursuit of a complete data set to calculate recurrent firearm injuries, our research team queried data from the NDI to account for out-of-hospital deaths in patients who presented to an STL-HVIP-DR partner hospital with a firearm injury from 2010 through 2019. At the end of 2019, the project principal investigator (PI), KM, emailed multiple contacts among STL-HVIP-DR partner hospitals and universities to identify colleagues who had previously acquired data from the NDI. KM also contacted the Health Alliance for Violence Intervention Research and Evaluation Committee through email and inperson Zoom meetings; this organisation coordinates hospital-based violence intervention programmes nationally.18 No members of this group reported recent or regular access of data from the NDI. Through these efforts it became clear that none of our university or hospital partners, including KM’s affiliated university, regularly access data from the NDI.
Three contacts were found who had accessed NDI data in the previous 10 years; the process was noted to be fruitful, but labour and cost intensive. In 2020, our team explored linking STL-HVIP-DR firearm injury data with medical examiner data, Missouri and Illinois state vital records data, and other out-of-hospital death data. Non-NDI data sources were deemed not feasible given the multicounty, multistate geographical catchment applicable to the STL-HVIP-DR firearm-injured cohort. Additionally, in 2020 due to the COVID-19 pandemic, email correspondence with the Illinois quality control inspector indicated that the Illinois Department of Public Health was not accepting any new research or data requests unrelated to COVID-19 in response to the pandemic health crisis. For this reason, our team decided to continue with the original plan to acquire out-of-hospital death data from the NDI.
NDI application
Online application
While specifics of the online application may change as the NDI application is revised over time, we provide the following stepwise guide to completing the online application to search NDI data. This online application is a preapproval vetting process that must be completed and approved by the NDI before transfer of data will be accepted to the NDI to complete the data query. A step-by-step guide to complete the online application based on our experiences is detailed in table 1. A sample copy of the online NDI Processing Portal application is available on the NDI website.19
Step-by-step guide and tips for successful completion of the online NDI Processing Portal
Timeline of activities
In June 2021 our research team initiated the online application for a search of the NDI. To do this, KM opened the Secure Access Management Services (SAMS) Public Health Partner Portal and completed the portal registration. The next day NDI application processing portal access was granted. Our team then completed the application to the best of our ability. This initial application was followed up with several emails and Zoom meetings with contacts from the CDC’s NDI team and multiple rounds of application resubmission based on their feedback over the following 2 months. Final recommendations for each section of the online application are described in table 1. Of note, it took 2 weeks to complete the confidentiality agreements for all research team members, as each of these forms needed to be signed by the project PI, the research team member, the data steward and a representative from the primary site university grant’s office. After multiple rounds of revisions and discussion with NDI staff, our online application to the NDI was approved in August 2021.
After online data application approval, it took 1 month to finish preparing the STL-HVIP-DR data for submission to the NDI (this process was also started in June 2021). It took an additional 3 weeks to arrange a purchase order through the primary institution to process the cost of the application. The NDI does have a credit card payment option, but the cost of this data request was outside credit card limit at our primary site. Payment receipt was required by the NDI before the NDI would start processing the data request. The total cost of our data request was $11 500 for 12 219 patient records.
Data were then returned from the NDI to our team in late November 2021. All told, from time of initiating the online application to receipt of data from the NDI, this process took 23 weeks. It took our data team an additional 2 weeks to review the extensive documentation and convert the NDI data into a form compatible with the STL-HVIP-DR for analysis.
Notes on data preparation
A significant amount of time and skill is needed and should be invested to clean data prior to the submission of the NDI data request. After searching the NDI Frequently Asked Questions (FAQ) site and conducting other internet searches, no template code in any common programming languages could be found. Our research team had most experience programming in Stata and used that platform to clean and format data for the NDI submission.20
While the NDI instructions provide a substantial amount of detail on how the submitted data should be formatted, it should be noted that data are requested in a fixed-width format, a less-common format that many software programs may not natively support. We used a freely available add-on module for Stata called OUTFIX2. Additional cleaning and formatting of critical data fields (such as name or date of birth) are required to maximise the likelihood of data matching and reduce the likelihood of unnecessary expense.
As mentioned elsewhere, requesting NDI data is not free. The NDI team provides an algorithm, embedded within an Excel spreadsheet, to assist researchers in computing an approximate cost for the data request. This fee is based on the number of records and the last year of contact for each record request. Thus, older records will cost more to request than more recent records. Finally, after corresponding with the NDI team, we confirmed that charges for the NDI request would be based on the initial number of records provided, irrespective of how many subsequent records might be removed upfront due to missing or improperly formatted data. Therefore, it greatly benefits the researcher (both in terms of a high sample match rate and financially) to provide a thoroughly cleaned data request that meets NDI specifications.
The NDI’s process for matching file(s) of records submitted for an NDI search has been detailed elsewhere.14 15 As described on the NDI website, and detailed in our Results section, this process includes matching submitted records to NDI files along several patient identifiers, which include combinations of US Social Security Number (SSN); month and year of birth; first, middle and last name; and father’s surname (eg, first and last name and SSN, or first and last name and month and year of birth).14 The matching criteria are intended to maximise the number of true matches that can be found, especially for records when SSN is not available. The success of the NDI matching process is determined by the effectiveness of the matching criteria, the quality and completeness of the data on study subjects, the quality and completeness of the death certificate data in the NDI file, and the investigator’s ability to assess the quality of the resulting matches.14
On receiving the results data, significant time was needed to properly import the data and review the accompanying documentation to understand the match results. Our team used the readr package in R to import the results file. Other researchers have also used SAS for import and conversion. Once raw data were converted, data in the raw file were tabulated and checked for agreement with the match summary reports provided by NDI.
NDI contact information
Our research team found NDI staff to be helpful and accessible throughout this process. They can be reached for questions by email at ndi@cdc.gov or by phone at +01-301-458-4444. Additional information regarding the NDI application process and supplementary materials can be found on the NDI website (https://www.cdc.gov/nchs/ndi/index.htm).
Results
Our data query to the NDI returned 12 219 firearm-injured patient records submitted from the STL-HVIP-DR between 2010 and 2019 (table 2). Of these, 12 034 (98.5%) records were accepted by the NDI as a match. One hundred and eighty-five records were rejected by the NDI for invalid or missing data among identifier variables such as last name, first name, SSN or birth date. In our patient cohort, 7.5% of records were flagged as a true match or ‘presumed dead’.
Summary of 12 219 matched results returned from the NDI
Discussion
Linking hospital-based data sets with NDI data can provide valuable information on out-of-hospital deaths. This has the potential to improve the quality of longitudinal morbidity and mortality calculations in hospital-based patient cohorts. Regarding our research team’s work, linking identified NDI data to the identified STL-HIVP-DR has the potential to greatly increase the rigour of recurrent firearm injury calculations. As there are no national databases that longitudinally track and link non-fatal firearm injuries with recurrent injury or death, and state-level fatal and non-fatal firearm injury data capture is incomplete in our region, these data will fill an important knowledge gap regarding the scope of acute and recurrent firearm injuries. Linking these data facilitates more accurate calculations of firearm injury recidivism and time-to-event (survival) analysis.
For our initial query, we requested data from the NDI as a single study where data may be stored for analysis up to 5 years. The STL-HVIP-DR has a DSA in place for all our universities, hospitals and the MHA, which did not meet use criteria for the NDI registry use. We plan to develop a companion NDI registry to the STL-HIVP-DR in future for long-term storage of NDI data on out-of-hospital firearm injury deaths. Based on our experience creating the STL-HVIP-DR, we anticipate this process will take at least a year as we will need to obtain data sharing approval from the legal departments of all affiliate universities and hospitals to create the companion NDI registry.
Our research team, which includes data managers, epidemiologists, medical clinicians, public health workers and statisticians, encountered several logistic and administrative challenges in completing the online NDI Processing Portal and in preparing and receiving data from the NDI. Adding a dedicated administrator at the university institutional level to help manage NDI data applications and data processing could improve efficiency in NDI application and data receipt in the future. This university-level NDI administrator would gain experience by completing multiple request across disciplines, and future researchers would benefit from this cumulative experience.
Several of the delays in portal approval, data submission and data acquisition were the result of competing demands for time and resources among research team members in public health and those with patient care responsibilities during the ongoing COVID-19 pandemic. Additionally, there were unanticipated staff turnovers at the CDC, including the retirement of the person primarily responsible for NDI processing during our application window. The NDI data application and submission process took our team 23 weeks from start of the application to receipt of data from the NDI. The greatest time delays occurred during completion of the online portal and ensuring processing and receipt of the purchase order from our university to the NDI. We believe that by following the steps outlined in this paper the application time could be truncated significantly and potentially be completed in as little as 6–8 weeks based on a 2-week turnaround time for the initial online application, 2 weeks for data preparation while the application is being processed for approval, and 2 weeks for data return from the NDI.
Lessons learnt include starting the NDI data application process early in your research project and clearly detailing your data sources, data storage methods, security methods and team members. We also recommend having a cleaned data set at the time of data submission to the NDI and closely following the NDI user’s guide in reformatting of data in preparation of submission to the NDI. Additionally, for university-based investigators, early coordination with your regulatory and billing offices may expedite completion of DSA with the NDI and payments to the NDI.
In conclusion, linking hospital-based data sets with NDI data can provide valuable information on out-of-hospital deaths. This has the potential to improve the quality of longitudinal morbidity and mortality calculations in hospital-based patient cohorts across disciplines. While NDI data access process may change over time, it is our hope that the lessons learnt presented herein will help to facilitate future easy and streamlined acquisition of valuable NDI data for other clinical researchers.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by the Washington University in St Louis Institutional Review Board (IRB #201806054). This study does not involve animal subjects.
Footnotes
Twitter @KMueller_MD
Contributors All authors contributed to study concept and design and critical revision of the manuscript for important intellectual content. KM and BPC completed the acquisition of data, analysis and interpretation of data, and drafting of the manuscript. KM, the guarantor for this work, accepted full responsibility for the overall content of this work, had access to the data, and controlled the decision to publish.
Funding KM is supported in part by the Emergency Medicine Foundation and the American Foundation for Firearm Injury Reduction in Medicine at Aspen Institute.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.