Article Text
PDF
Abstract
Large administrative databases provide powerful opportunities for examining the epidemiology of injury. The National Coronial Information System (NCIS) contains Coronial data from Australia and New Zealand (NZ); however, only closed cases are stored for NZ. This paper examines the completeness of NZ data within the NCIS and its impact upon the validity and utility of this database. A retrospective review of the capture of NZ cases of quad-related fatalities held in the NCIS was undertaken by identifying outstanding Coronial cases held on the NZ Coronial Management System (primary source of NZ Coronial data). NZ data held on the NCIS database were incomplete due to the non-capture of closed cases and the unavailability of open cases. Improvements to the information provided on the NCIS about the completeness of NZ data are needed to improve the validity of NCIS-derived findings and the overall utility of the NCIS for research.
Statistics from Altmetric.com
Background
Large databases of administrative data provide powerful opportunities for examining the epidemiology and outcomes of injury. For example, cross-country databases with common coding frameworks offer considerable promise for research by making data readily available, retrievable and compatible for comparative research and benchmarking of safety performance. The utility of large databases for research purposes is determined by the (1) completeness of the database (ie, What cases are available to the database and does the database capture all available cases?) and (2) accuracy of the information within the database.1 Both aspects should be assessed to identify if either limitation is associated with any individual database, and methods for improving data should be considered to address any concerns.1
The National Coronial Information System (NCIS) is an internet database and user-retrieval system that contains information on all deaths reported to a Coroner in both Australian and New Zealand (NZ) Coronial jurisdictions.2 Coroners are notified of a death when a person has died in an unnatural or unknown way, including all intentional and unintentional injury deaths, and in-depth inquests are held in about 6% of cases.3 The NCIS platform was designed for Australian state Coronial jurisdictions and has been operational since 2000, with the NZ jurisdiction added in 2013. The Coroners Act 2006 & Official Information Act 1982 limit the release of personal details and Coronial documents with regard to open Coronial cases in NZ; therefore, NZ data included within the NCIS database differ from the Australian data. NZ data, derived from the Coronial Services of New Zealand's Coronial Management System (CMS), are only uploaded to the NCIS database when Coronial cases are officially closed following completion of the Coronial review process, while Australian state Coronial jurisdictions upload open cases to the NCIS platform and manage these cases on the NCIS database until case closure (Dean Skatchill, Coronial Services of New Zealand, Pers Comm, 2014). Given the operational differences for NZ Coronial cases held on the NCIS database could potentially influence the validity of population-based estimates of fatal injury and cross-country comparisons, it is important to understand the completeness of NZ data on this database.
This paper examines the completeness of NZ data within the NCIS database by examining case capture and case availability using a subset of Coronial cases of current interest—fatalities involving a quad bike—for a 5-year period covering July 2007–December 2012. Our study assesses the validity of the NCIS database for estimating the number of quad-related fatalities in NZ, and the overall utility of NZ data on the NCIS platform for research purposes. In particular, this study answers the following research questions: (1) Does the NCIS platform capture all the closed NZ Coronial cases? and (2) What proportion of NZ Coronial cases remain open and thus unavailable to the NCIS database?
Methods
A retrospective review was undertaken to estimate the completeness of NZ cases of quad-related fatal injuries in the NCIS. All Coronial cases involving a quad bike were extracted from the CMS for the period July 2007–December 2012. The CMS, maintained by the Coronial Services of New Zealand, is the sole source of NZ Coronial data uploaded to the NCIS database. The CMS-generated list of quad-related fatalities was matched to NCIS-held Coronial files using the local Coronial case number, which is a shared feature of both databases.
To assess closed case capture on the NCIS database, three data matches were conducted over a 12-month period: first in January 2014, then in July 2014 and finally in January 2015. Closed Coronial case capture in the NCIS system was calculated as the proportion of the CMS-identified quad-related fatality closed cases that were subsequently matched to the NCIS database.
To estimate the number of open cases excluded from the NCIS database, the numbers of closed and open Coronial cases for quad-related and all-cause fatalities were obtained from the CMS in July 2014. The distribution of open and closed cases on the CMS was calculated for quad-related and all-cause fatalities. The proportion of open Coronial cases identified on the CMS estimates the extent to which data are unavailable to the NCIS database.
Results
We identified 39 quad-related fatalities on the Coronial Services of New Zealand CMS database for the period July 2007–December 2012: 35 cases were closed and 4 remained open by the end of our data collection period (January 2015). Table 1 provides a summary of the CMS-identified closed Coronial cases able to be matched to records held on the NCIS database at the three time points. At the first search, 18 (51%) of the 35 closed NZ quad bike-related Coronial case files were held in the NCIS database. Case matching undertaken 6 and 12 months later identified 94% (33 cases) and 100% (35 cases), respectively, of all closed quad bike-related fatality cases were held on the NCIS platform. Closed cases from the most recent years of our study period were most likely to be absent from the NCIS. For example, in July 2014, all closed cases from 2007 to 2010 were available, but only 75% (3) of 2011 cases and 71% (5) of 2012 cases.
New Zealand quad-related fatal injury closed case capture on the NCIS platform by year
Table 2 summarises the proportion of open NZ Coronial cases on the CMS in July 2014. A third of quad-related cases from 2012 remained open and were therefore unavailable to the NCIS database. For all-cause fatalities, the percentage of closed cases increased with increasing time since the fatal injury event with those occurring in the most recent years (2011–2012) most likely to remain open, with 2–12% (58–402) of cases unavailable to the NCIS database.
Proportion of open Coronial cases by year (data obtained from CMS, Coronial Services of New Zealand in July 2014)
Discussion
This study provides a snapshot of the utility of the NCIS platform for research purposes by examining the completeness of NZ Coronial data on the NCIS. We sought to establish, using the example of quad-related fatalities, to what extent (1) the NCIS database captures all the closed Coronial cases for NZ and (2) NZ Coronial cases remain open and therefore unavailable to the NCIS database. In this study, we found that both absent closed cases and the unavailability of open cases contributed to incomplete Coronial data being held on the NCIS for NZ. These gaps in the completeness of the NCIS database were time dependent with the likelihood of absent closed cases or of unavailable open cases decreasing with increasing time since the fatal injury event. Separate inquiries with Coronial Services of New Zealand identified significant 3-month to 6-month, or greater, delays in uploading closed cases to the NCIS during our data collection period due to backlogs in the preparation and uploading of large volumes of historical data as far back as 2007 to the NCIS (Dean Skatchill & Gemma Baigent, Coronial Services of New Zealand, Pers Comm, 2014). While these delays have been rectified and uploads of recently closed cases now occur weekly, at the time of this study the delays were not indicated by the NCIS on either the database or their website.
Our study's findings highlight the importance of checking the completeness of a database. To effectively prevent injury and inform prevention efforts, researchers need to ensure the data used to determine the epidemiological burden of injury within a specific population capture all relevant cases. Our first attempt at establishing the rate of quad-related fatal injuries in NZ only captured 18 of 39 cases, which could have led to the publication of fatality figures for NZ that underestimated the true number of fatalities by half. The impact of incomplete NZ Coronial data within the NCIS could lead to the unwitting generation of potentially misleading epidemiological findings.
Differences in the completeness of NZ data within the NCIS database is a limitation that needs to be clearly conveyed to NCIS users. While researchers using the NCIS have only limited access to open Australian Coronial cases, these users are clearly informed about outstanding Australian data via operational statistics (updated monthly) of the number of closed and open cases. No such information exists for NZ data. The NCIS reports a 100% case closure operational statistic for NZ cases. While technically correct (ie, 100% of NZ cases held by the NCIS are closed), this statistic is potentially misleading to uninformed users. We identified that at the time of our first search the NCIS only captured 51% (18 cases) of the expected closed quad-related cases. Since there are no future plans to include open NZ Coronial cases to the NCIS (Gemma Baigent, Coronial Services of New Zealand, Pers Comm, 2014), it is important accurate closed case data are provided to NCIS users.
To reduce the possibility of producing potentially misleading fatality figures using NZ data, we propose greater transparency regarding the completeness of NZ data on the NCIS database. We recommend the NCIS ensures the date of the most recent upload of closed NZ cases is available and that the NCIS accounts for open cases excluded from the database by providing regularly updated figures derived from the CMS on the number of open cases outstanding. Future research should investigate the likelihood of, and nature of, changes to data between the opening and closing of Coronial files to ascertain if the benefits of making open cases available for public health research and surveillance outweigh any potential negative consequences.
Conclusion
This study highlights the importance of ascertaining the completeness of data held on large cross-jurisdictional databases, such as the NCIS. Furthermore, this study serves as a word of warning to current and future users of NZ data on the NCIS database. Our study indicates improvements are needed in the information provided about the completeness of NZ data in the NCIS database to improve the accuracy of NCIS-derived findings and the overall utility of the NCIS for research. Until this information is available, we recommend that NCIS users of NZ data ascertain directly with the Coronial Services of New Zealand any delays in uploading of closed cases to the NCIS and the number of outstanding open cases.
What is already known on this subject
Large administrative databases provide powerful opportunities for examining the epidemiology of injury; however, the completeness of a database can limit the utility of the database for research purposes and the validity of research findings.
The National Coronial Information System (NCIS) is an internet-based storage and retrieval platform for Australian and New Zealand (NZ) Coronial records; however, unlike Australian cases, only closed (completed) Coronial cases are stored for NZ.
What this study adds
The completeness of NZ data held on the NCIS database was deficient due to the incomplete capture of closed cases and the unavailability of open cases. Our first attempt at establishing the number of quad-related fatalities underestimated the true number of fatalities by half.
This study raises questions about the validity and utility of NZ data on the NCIS given these data are incomplete. At a minimum, documentation should be provided to NCIS database users that alerts them to these deficiencies.
Smartphone app to prevent scalds
Injury prevention researchers at The University of Queensland and a digital health company, iPug, are collaborating to prevent childhood scalds from hot drinks. Cool Runnings is a smartphone app intended to educate parents about how to prevent these injuries and how to apply first-aid treatment. Although the app is basically educational, it involves choosing ‘the right message for the right person on the right device, (and) then reward them for engaging in the message.’ Comment: Once again, I suspect education alone will be relatively ineffective.
Prevent cricket injuries digitally?
The Board of Control for Cricket for India (BCCI) hopes to use a digital monitoring system to prevent strain-related cricket injuries. The system is especially aimed at helping fast bowlers. It will also measure the workload to which players are subjected. A BCCI official said it is putting “more emphasis on preventing injuries rather than just rehabilitation”.
Acknowledgments
We thank Gemma Baigent and Dean Skatchill of the Coronial Services of New Zealand, Ministry of Justice, for providing information on the relationship between the Coronial Services of New Zealand and the NCIS, and for data supplied from the Coronial Services of New Zealand, Coronial Management System. We also thank Joanna Constanios, National Coronial Information Service, for providing information on operational aspects of the NCIS.
Footnotes
Contributors RL conceived and designed the study, oversaw the acquisition of data, undertook the analysis and interpretation of data, drafted and revised the manuscript. GD contributed to the study conception and design, analysis and interpretation and manuscript revisions. SW undertook the data acquisition, contributed to data analysis and interpretation of findings and manuscript revisions. All authors approved the final version for publication.
Funding This study was funded by the University of Otago, Dunedin School of Medicine Dean's Bequest fund.
Competing interests None declared.
Ethics approval University of Otago human ethics committee and the Chief Coroner of New Zealand via the NCIS ethical process.
Provenance and peer review Not commissioned; externally peer reviewed.