Background The US Child Death Review (CDR) multi-disciplinary team consensus process uses tiered maltreatment definitions that allow for a degree of uncertainty. Individual heterogeneity is thought to normalise within the context of the CDR team resulting in reliable consensus determinations. This assumption has never been tested. The Alaska CDR tiered maltreatment designations were used to assess the polytomous maltreatment classification.

Methods We used a mixed-methods inter-rater reliability framework to test the reliability of abuse, neglect, and gross-negligence classifications through CDR consensus review. All children born in 2009–10 who died in Alaska through 2014 were eligible for inclusion (n = 116); after exclusions 101 cases were re-reviewed. Approximately half of the cases were included with normal monthly reviews, the rest were reviewed at an annual committee “blitz” review.

Results Total percent agreement was 64.7% with a weighted Kappa of 0.61 (95% CI: 0.51, 0.70). The percent agreement was 69.3% for abuse, 64.4% for neglect, and 60.4% for gross-negligence. Agreement was much less among infant deaths compared to child deaths (0.48 vs 0.77; p = 0.001) and unmarried mothers at birth (0.47 vs 0.74; p = 0.005). Incidence estimates per 1,000 live births are statistically equivalent between the initial and subsequent review for yes/yes probably any maltreatment and by maltreatment type.

Conclusions This is the first known reliability study of CDR team classification of tiered maltreatment death classification. Not unexpectedly, substantial agreement was observed for abuse and only moderate agreement for neglect and negligence classifications. Discordance is related to missing or unknown information and the interpretation of identified risk factors. Due to these limitations, standardised death scene investigations for all infant and child deaths are invaluable and implementing a systematic decision tree metric is necessary to produce more reliable estimates.