Article Text
PDF
Abstract
Background This study aimed to simultaneously evaluate the effectiveness of a range of interventions to increase the possession of safety equipment or behaviours to prevent falls in children under 5 years of age in the home.
Methods A recently published systematic review identified studies to be included in a network meta-analysis; an extension of pairwise meta-analysis that enables comparison of all evaluated interventions simultaneously, including comparisons not directly compared in individual studies.
Results 29 primary studies were identified, of which 16 were included in at least 1 of 4 network meta-analyses. For increasing possession of a fitted stair gate, the most intensive intervention (including education, low cost/free home safety equipment, home safety inspection and fitting) was the most likely to be the most effective, with an OR versus usual care of 7.80 (95% CrI 3.08 to 21.3). For reducing possession or use of a baby walker: education only was most likely to be most effective, with an OR versus usual care of 0.48 (95% CrI 0.31 to 0.84). Little difference was found between interventions for possession of window locks (most intensive intervention versus usual care OR=1.56 (95% CrI 0.02 to 89.8)) and for not leaving a child alone on a high surface (education vs usual care OR=0.89 (95% CrI 0.10 to 9.67)). There was insufficient evidence for network meta-analysis for possession and use of bath mats.
Conclusions These results will inform healthcare providers of the most effective components of interventions and can be used in cost-effectiveness analyses.
Statistics from Altmetric.com
Introduction
Across the world, falls are a leading cause of morbidity and mortality in children.1–4 Mortality rates from falls in childhood are highest in children under 1 year old1 and, among 0–4-year-olds, they are the 12th leading cause of disability-adjusted life years lost.5 Most falls occur at home in the 0–4-year-olds.6 Falls place a considerable burden on healthcare systems globally as they are the most common injury among emergency department attenders, comprising between 25% and 52% of all treated child injuries.7 ,8 Falls are the leading cause of injury-related healthcare costs in the USA, accounting for just over one quarter of all injury-related healthcare costs in childhood, totalling US$81 billion in 1996.9
Despite the high burden of injury attributable to falls, there is little evidence that home safety interventions reduce falls rates or promote falls prevention practices. A recent overview of systematic reviews on preventing childhood falls within the home identified one meta-analysis and 13 systematic reviews, and included a total of 29 relevant primary studies10 Evidence of the effect of interventions on falls or fall injuries was sparse, with one of three primary studies reporting this outcome finding a significant reduction in falls.11 Interventions were effective in promoting the use of safety gates and furniture corner covers. There was some evidence of a reduction in baby walker use. The effect on the use of window safety devices, non-slip bath mats/decals and the reduction of tripping hazards were mixed. One meta-analysis was included in the overview12 which found that families receiving home safety interventions were: significantly more likely to have a fitted stair gate; less likely to use a baby walker; not significantly different in their possession of window locks, non-slip bath mats or decals; or reporting leaving a child alone on a high surface. There was significant heterogeneity between effect sizes for the fitted stair gate and baby walker outcomes. This meta-analysis evaluated any intervention against a ‘usual care or no intervention’ comparison group. The interventions, and in some studies the control arm, comprised various combinations of education, home safety inspection, provision of free or low-cost safety equipment, and fitting of safety equipment. Some interventions were aimed at only preventing fall-related injuries, while others aimed to prevent a range of injuries. In reality, healthcare commissioners, and housing providers, among others, have to make policy decisions on the ‘best’ intervention(s) for preventing fall-related injuries, so lumping together interventions is not particularly useful.
Standard (pairwise) meta-analyses are usually restricted to finding a pooled estimate of effectiveness comparing two groups, often an intervention group with a control group and, hence, only identifies one as being superior to the other. This can make only a limited contribution to policy decisions.13 An extension to this that enables comparison of all evaluated interventions simultaneously within a single coherent analysis is network meta-analysis (NMA),14 also known as mixed treatment comparison.13 ,15 NMA allows all interventions to be compared with one another, including comparisons not directly evaluated within any of the primary studies. Interventions can also be ranked in order of effectiveness. Such an approach is being increasingly used in health technology assessment when deciding on the optimal intervention strategy for a given medical condition.16 ,17
Suppose we have studies providing effect estimates for control versus an intervention A, and for intervention A versus an intervention B. NMA allows us to estimate the pooled effects where pairwise evidence exists (direct comparison between control and A and between A and B), and also allows us to estimate effects where interventions are not directly compared but are linked through a connected network of studies (indirect comparison between control and B). If evidence is available on all comparisons between control, A and B, the indirect evidence is pooled with direct data from the studies, hence, inference is based on more evidence and uncertainty should be reduced. All evidence is combined in a single model and details can be found in the Nice DSU Technical Support Document 2.18 NMA is particularly relevant to the field of injury prevention, where interventions are often complex and multifaceted, and the number of studies evaluating the same comparisons is small.
The objective of our research was to evaluate the effectiveness of different interventions to increase the possession of safety equipment by households or increase falls prevention behaviours. This application is part of a series of NMAs evaluating a range of interventions to prevent injuries in preschool children in the home. The first paper to be published in this series reported NMA to evaluate the effectiveness of interventions to increase the uptake of smoke alarms.19
Methods
Study identification
A recently published overview of systematic reviews and a systematic review of primary studies published since the most recent comprehensive review10 sourced studies for the NMA. The primary studies included in the published review reported non-legislative interventions aimed at primary or secondary prevention of falls at home among children aged 0–19 years, and reported medically or non-medically attended falls, possession or use of home safety equipment to prevent falls, or other falls prevention practices. The sources searched for the review are summarised in online supplementary appendix table 1A, and more details (including the search strategies used, and the inclusion and exclusion criteria applied) are available in Young et al.10
Study quality of the primary studies was assessed using: allocation concealment, blinding of outcome assessment, and completeness of follow-up for randomised studies; blinding of outcome assessment, completeness of follow-up, and balance of confounders between treatment arms for non-randomised studies; and the Newcastle-Ottawa scale20 for controlled observational (case-control and cohort) studies.
Statistical methods
The main outcomes analysed for the NMAs were: possession of a fitted stair gate; possession or use of a baby walker; possession of a bath mat or decals; possession of window locks; and never leaving a child alone on a high surface. Possession of a fitted stair gate could refer to a gate at the top and/or bottom of the stairs or a safety gate preventing access to an unsafe area; most of the original studies did not specify and only looked at possession. For the baby walker outcome, we considered it appropriate to combine baby walker possession or use as one of the included studies21 found that 94% of those owning a walker used it and 96% of those who used a walker owned one.
NMA13–15 was implemented to compare a range of different interventions including ‘usual care’ and to include studies where the control arm was another intervention. It allows us to estimate the pooled effects where pairwise evidence exists (direct evidence) and also to estimate effects where interventions are not directly compared but linked through a connected network of studies (indirect evidence). For example, if we wanted to compare the following four interventions—usual care, education, safety equipment giveaway, home safety inspection— this could be achieved by using studies containing the following direct pairwise comparisons: usual care versus education, education versus safety equipment giveaway, and safety equipment giveaway versus home safety inspection (figure 1A), and by tracing a comparison pathway through the direct pairwise comparisons to estimate, for example, the indirect comparison of usual care versus safety equipment giveaway. However, the network would be disconnected, and the analysis impossible, if only studies of usual care versus education, and safety equipment giveaway versus home safety inspection existed (figure 1B).
(A) Connected Network; (B) Disconnected Network. Line represent available direct evidence.
For our analyses, a standard NMA random-effects model with a binary outcome13–15 was fitted to the data that allows us to include trials with three or more arms by accounting for the correlation structure.22 ,23 If studies did not adjust for clustered allocation of intervention, the effective sample size was estimated based on the design effect using published intraclass correlation coefficients (ICC),24 or ICCs estimated from individual participant data where the author(s) provided it. We obtained pooled estimates of intervention effects, expressed as ORs, and 95% credible intervals (CrI) for all combinations of pairwise comparisons from the NMAs using a combination of direct and indirect evidence, and indirect evidence only. For completeness, pooled estimates from the direct evidence only are presented for each pairwise comparison where study data was available (using a fixed effect meta-analysis model when only two studies were available for a particular pairwise comparison, and a random effects model where three or more studies were available, and where only one study had evaluated a particular pairwise comparison, the results from this study alone). From the NMA, intervention effectiveness was ranked based on absolute intervention effects (derived by using an underlying rate based on the usual-care arms) and the probability that each intervention was best for a particular outcome was calculated.14
To assess the goodness of fit of the model to the data, the posterior mean residual deviance25 ,26 was calculated. For an adequately fitting model it will be approximately equal to the number of treatment arms across all studies.27 ,28 Heterogeneity of the network (variability in treatment effects within pairwise comparisons above that expected by chance) was quantified by using the between-study SD parameter where a SD of below 0.5 indicated fairly low heterogeneity, and above 1, substantial heterogeneity.23 ,29 ,30 This model assumes that the degree of between-study, within-comparison heterogeneity is constant across all intervention comparisons in the network.
Inconsistency, where direct and indirect evidence are available and do not agree (beyond chance), was assessed using the method based on node splitting31 which compares the results from the direct evidence to the results from the NMA with the direct evidence excluded.
To explore the effect of the variable study quality of the included studies, analyses were repeated restricted to randomised clinical trials (RCT) and, to explore the variability of the interventions, analyses were repeated by splitting the interventions further.
The NMAs were conducted using a Markov chain Monte Carlo method14 with minimally informative prior distributions fitted in the WinBUGS software28 and tests for inconsistency were carried out in R.31 Further technical details of the analysis, together with the WinBUGS and R code, are available from the corresponding author. The analysis and subsequent reporting adhere to the PRISMA statement guidelines.32
Results
The published overview of systematic reviews and systematic review of primary studies10 identified 29 primary studies eligible for inclusion in the NMA (24 from the overview of systematic reviews and 5 from the systematic review of primary studies). Of these, 13 were excluded from the NMA (figure 2). Online supplementary appendix table 1B shows the bibliographic databases in which the articles appeared. The characteristics of the 16 studies included in the NMAs are reported in table 1 together with their study quality which was observed to be variable across studies. In total, seven interventions were evaluated (although not all interventions had been trialled for each outcome):
-
usual care
-
education
-
education+low cost (ie, voucher)/free equipment
-
education+low cost (ie, voucher)/free equipment+home safety inspection
-
education+low cost (ie, voucher)/free equipment+fitting
-
education+home safety inspection
-
education+low cost (ie, voucher)/free equipment+fitting+home safety inspection.
Summary of Studies and their data included in the NMA of the interventions to prevent falls injuries in children under 5 (Numbers adjusting for clustering in parentheses)
PRISMA flow chart for the systematic overview of reviews and systematic review of primary studies.
Possession of a fitted stair gate
Ten (83%) of the 12 studies reporting the possession of a fitted stair gate outcome were RCTs, and 2 (17%) were non-RCTs (table 1). Figure 3A displays a connected network diagram for this NMA.
Network diagrams of interventions to increase safety practices to prevent falls in pre-school children in the home. Each intervention is a node in the network. The links between the nodes are pairwise intervention comparisons. The numbers along the link lines indicate the number of studies or pairs of study arms for that link in the network. The network for bath mats is presented in the online supplementary appendix. * Babul three-arm trial (A) Stair gates (B) Baby walker (C) Window locks (D) High surfaces.
The NMA estimated the 21 possible pairwise comparisons between the seven interventions. The pooled estimates, along with the available direct within-trial estimates are reported in table 2A.
Results of NMA models for interventions to increase safety practices to prevent falls in preschool children in the home expressed as ORs. Direct comparison estimates are also displayed.*,†
The most intensive intervention (education+low cost/free equipment+home safety inspection+fitting) was most likely to be effective (probability best=0.97, table 3) in increasing possession, with, for example, families in the intensive intervention group more likely to possess a fitted stair gate compared with those in the usual care group (OR=7.80 (95% CrI 3.08 to 21.3)).
Assessment of best intervention and model fit
The effect of study design on the NMA results was assessed by repeating the above analysis using only data from the 10 RCTs. The result was similar with the most intensive intervention identified as being the most likely to be effective (probability best=0.87), with an estimated OR for possession versus usual care of 7.93 (95% CrI 2.76 to 23.6).
Possession or use of a baby walker
Seven (78%) of the nine studies for the possession or use of a baby walker outcome were RCTs and two (22%) were non-RCTs (table 1, figure 3B).
The NMA estimated the 21 possible pairwise comparisons between the 7 interventions (table 2B). The education-only intervention was the most effective (probability best=0.65, table 3), with families in the education-only intervention group less likely to possess or use a baby walker compared to usual care (OR 0.48, 95% CrI 0.31 to 0.84).
The effect of study design on the results of the NMA results was assessed by repeating the above analysis using only data from the seven RCTs. The result was similar with the education-only intervention identified as being the most likely to be effective (probability best=0.45), with an estimated OR versus usual care of 0.58 (95% CrI 0.21 to 1.87).
Possession of window locks
Five (83%) of the six studies for the possession of window locks outcome were RCTs and one (17%) was a non-RCT (table 1, figure 3C).
The NMA estimated the 15 possible pairwise comparisons between six interventions (excluding the education-only intervention) (table 2C). Education+Home safety inspection was most likely to be effective (probability best=0.26, table 3), but there was very little difference between any of the interventions (OR for Education+Low cost/free equipment+Fitting vs Usual care of 1.56 (95% CrI 0.02 to 89.8)). Repeating the analysis using only data from the five RCTs gave similar results.
Child not left on a high surface
Only three studies reported the numbers who left a child on a high surface; two (67%) were RCTs and one was a non-RCT (table 1, figure 3D).
The NMA estimated the six possible pairwise comparisons between four interventions. The pooled estimates, along with the available direct within-trial estimates are reported in table 2D. There was very little difference between any of the interventions, but education-only was the least likely to be effective in preventing children being left on high surfaces (probability best=0.10, table 3 and OR vs Education+Low cost/free equipment of 0.56 (95% CrI 0.06 to 4.65) and versus Education+Low cost/free equipment+Home inspection of 0.50 (95% CrI 0.03 to 8.76).
Possession of bath mats
Four studies reported the possession of bath mats (table 1). Three (75%) of the studies were RCTs and one (25%) was a controlled before-and-after study (CBA). Online supplementary appendix figure 1 displays a network diagram for this NMA showing two unconnected networks of three interventions, so we were unable to use NMA for this outcome.
Evaluation of models
Overall, the NMA models fitted the data well with the posterior mean residual deviance being close to the number of data points in each network (table 3).
The between-study SDs for each of the NMA models are reported in table 3. The uncertainty in their estimation reflects the relatively low number of studies providing direct evidence for each pairwise comparison, especially for the not leaving a child alone on a high surface outcome. Consistency was checked between the direct and indirect evidence by using node-splitting methodology. This can only be done when a pair of interventions is part of a closed loop in the network. Any closed loops in the networks were checked for consistency between the direct and indirect evidence. There was no evidence of inconsistency for either the stair gate (see online supplementary appendix table 4) or baby walker outcomes (see online supplementary appendix table 5). There were no closed loops for the possession of window locks, high surfaces and bath mat outcomes.
Sensitivity analysis
The network for the stair gate outcome was extended to comprise nine different intervention groups by splitting the low cost/free safety equipment giveaway included in interventions into relevant and not relevant/not stated (see online supplementary appendix figure 2). The findings from this analysis (see online supplementary appendix table 4) were similar in that the most intensive intervention clearly was the most effective in increasing the possession of a stair gate, and it also showed that there was very little difference between the interventions with low cost/free relevant or not relevant equipment. There is no relevant equipment for reducing the possession of baby walkers, and the networks were too sparse for the other outcomes to be extended further.
Discussion
NMA was used to compare and evaluate the different interventions to increase a range of safety practices to prevent falls in preschool children in the home. Using this method enabled all strategies, including those not addressed in any of the individual primary studies, to be compared, and interventions could be ranked to identify the most effective intervention(s) in promoting safety practices to prevent falls. The findings showed that the most intensive intervention was most effective in increasing stair gate possession, and education-only was most effective in reducing baby walker possession and use. The findings were inconclusive for the possession of window locks, child not left on a high surface, and possession of bath mats.
An updated Cochrane review24 based on pairwise meta-analyses of the same studies found that families in the home safety intervention group were more likely to possess a fitted stair gate, less likely to have or use a baby walker and more likely to possess window locks than families in a control arm. Because of the complexity and number of different interventions in the intervention and control arms and the number of studies available for the different combinations, the results from the NMA are more likely to be useful to policy makers, service commissioners and providers when making choices between multiple alternatives, than those from the pairwise meta-analyses.
A key limitation of our analyses is that, although we were able to categorise the interventions we studied to a greater degree than in previous meta-analyses,12 there is potentially still some ‘lumping’ of interventions within these categories. For example, the intensity of education interventions may differ markedly between studies (ie, from providing only educational information, such as leaflets, to providing intensive face-to-face education on home safety), and the low cost/free safety equipment may not have been relevant to the outcome concerned (ie, equipment may have included socket covers and smoke alarms that would not prevent fall injuries, and there was no directly relevant safety equipment for the possession and use of a baby walker outcome). Similarly, different levels of usual care exist across the populations recruited in the primary studies. Additionally, the definitions of low-cost equipment used by our included studies varied between studies. Costs may also not be comparable between studies conducted in different countries, or between populations within one country with very different income levels and economic conditions. Insufficient detail was presented in many of primary reports to enable us to subcategorise the interventions further. It would be helpful if future studies provided sufficient details of interventions to enable more detailed NMAs to be conducted. Sensitivity analyses showed that findings were robust to splitting the interventions further and to study quality. Due to small numbers of studies, we were not able to explore the impact of allocation concealment, blinding, and percentage follow-up separately; however, rerunning the NMA analysis using data from RCTs only had minimal impact on the results.
Potential extensions to the NMA modelling could be explored including examining differential effects by child and family factors, exploring in more depth the effect of study quality, and categorising educational interventions more finely. However, such analyses would be severely limited by the quantity of data currently available. The NMAs presented in this paper relied on only a small number of studies. To enhance the evidence base, further studies are required to increase precision of effect estimates, along with more details on the interventions trialled to reduce heterogeneity. Also, methods have been developed to incorporate individual-level data into NMA analyses,48 which would greatly increase the power of analyses to explore the subject-level covariates identified above.
Knowing which interventions are the most effective is important, but cost-effectiveness is an essential part of any decision-making process. The effect sizes from this NMA will be used in subsequent decision analyses to determine the most cost-effective interventions for preventing falls in preschool children in the home. For example, our analysis found the most intensive interventions to be the most effective for increasing stair gate possession; however, as these interventions will also be the most costly, it is crucial to establish which interventions provide the best value for money.
What is already known on this subject
-
Falls at home are a leading cause of injury in children under age 5 years, and a major burden on healthcare costs.
-
Interventions to prevent falls have shown to be effective in promoting the use of safety equipment.
-
Because interventions are often complex, direct comparisons are sparse and not easily combined in a standard (pairwise) meta-analysis.
What this study adds
-
Using a network meta-analysis, a range of complex interventions to prevent falls in children under age 5 years have been compared, including those comparisons not directly evaluated in primary studies.
-
The interventions have been ranked according to their effectiveness, thus providing directly useful information for decision makers to inform falls-prevention policy.
Fall prevention programmes save money
A paper in The Journal of Safety Research found three fall prevention programmes to be effective and to save medical costs. The programmes were: Tai Chi: Moving for Better Balance; Stepping On; and The Otago Exercise Program. Importantly, effectiveness was only based on randomised controlled trial results. The return on investment ranged from 36% for Otago to 509% for Tai Chi.
End to drug overdose epidemic predicted
Farr’s law on epidemic patterns may apply to infectious diseases and drug overdoses. A study in Injury Epidemiology predicts that the tenfold increase in overdose deaths since 1980 will peak in 2017 and then decline to about 6000 deaths in 2035.
Prolonged rest for concussions
A new study in Paediatrics indicates that following a concussion prolonged rest until acute symptoms are gone may and actually worsened them. Patients were randomised to strict-rest and short-rest groups. Strict rest failed to improve symptoms and also worsened them.
Acknowledgments
This paper presents independent research commissioned by the National Institute for Health Research (NIHR) under its Programme Grants for Applied Research funding scheme (Reference Number RP-PG- 0407–10231). The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Contributors DK was the principal investigator for the NIHR funded grant, contributed to the systematic review and draughting of the paper. SH undertook the analysis and draughted the paper. NC contributed to the analysis, plan, and interpretation of the results and draughting of the paper. BY contributed to the systematic review and draughting of the paper. PW and ZH contributed to the systematic review. PM contributed to the systematic review and critically reviewed the paper. FA and AS contributed to the analysis and interpretation of the results. All authors approved the final version of the paper.
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement This is a secondary analysis of previously published findings and references are given throughout.