Article Text
PDF
Abstract
Objective To identify, summarise and critically assess studies reporting costs and consequences of sport and recreation injury prevention strategies among children and adolescents.
Design Systematic review.
Methods and data sources We searched MEDLINE (Ovid), EMBASE, CINAHL, Pubmed, Econlit and SPORTDiscus and PEDE. Included studies were peer reviewed full economic evaluations or cost analyses of sport/recreation injury prevention among children and adolescents≤18 years of age. The Pediatric Quality Appraisal Questionnaire was used for quality assessment.
Results The initial search yielded 1896 unique records; eight studies met inclusion criteria. Six studies were related to injury prevention in the context of recreation, two were related to sports. For recreation studies in cycling and swimming: costs per head injury averted was US$3109 to $228 197; costs per hospitalisation avoided was US$3526 to 872 794; cost per life saved/death avoided was US$3531 to $103 518 154. Sport interventions in hockey and soccer were cost saving (fewer injuries and lower costs). Global quality assessments ranged from poor to good. Important limitations included short time horizons and intermediate outcome measures.
Conclusions Few rigorous economic evaluations related to sport and recreation injury prevention have been conducted. The range of estimates and variation in outcomes used preclude specific conclusions; however, where strategies both improve health and are cost saving, implementation should be prioritised. Future economic evaluations should incorporate time horizons sufficient to capture changes in long-term health and use utility-based outcome measures in order to capture individual preferences for changes in health states and facilitate comparison across intervention types.
- recreation / sports
- economic analysis
- systematic review
- adolescent
- child
Statistics from Altmetric.com
Introduction
Participation in sport and recreational activity promotes healthy physical, cognitive and social development in children and adolescents. While physical activity is increasingly promoted as part of a healthy lifestyle, yielding multiple long-term benefits, participation in sports and recreation activities is a leading cause of injury, with up to 40% of all children and adolescents sustaining sport related injuries anually.1–4 Recreation and sport-related injuries result in financial costs of care and treatment, economic costs from lost productivity, diminished health-related quality of life5 6 and poor health outcomes.7 Productivity impacts may be short term, for parents who need to miss work to care for injured children and long term, for individuals whose future capacity to work may be affected by injuries sustained in their youth.8–11
Injury prevention strategies have the potential for reducing recreation and sport-related injuries among children and adolescents.12 13 However, as with all resource allocation decisions, choices between competing alternatives must be made. In order to ensure that available resources are used optimally, a method for systematically assessing the costs and outcomes associated with competing alternatives is required. Economic evaluation provides a framework for such assessment and is widely used to guide resource allocation and priority setting within the health sector.14 Currently, evidence about effectiveness of strategies to prevent sport and recreation injuries among children and adolescents is available.15–19 However, evidence related to cost effectiveness is lacking.20–23
The aim of this systematic review is to examine economic evaluation literature on sport and recreation injury prevention strategies for children and adolescents. The specific objectives are to: (1) identify cost analyses and full economic evaluations focusing on prevention of recreation and sports-related injuries in children and adolescents (aged≤18 years); (2) summarise the results of identified studies and (3) critically assess the quality and reporting of identified studies.
Methods
Six indexed online databases were searched: MEDLINE (Ovid), EMBASE, CINAHL, Pubmed, Econlit and SPORTDiscus. The search consisted of MeSH terms and keywords related to injury prevention; injury types; sports and recreation; economic evaluation and cost analysis. The search was limited to English language but had no limits on date of publication. Reference lists of final included studies were hand searched to identify additional eligible studies. The initial search, conducted in July 2016, was updated in November 2017 which yielded no additional eligible studies. The Paediatric Economic Database Evaluation (PEDE) was also searched in January 2018. All but one of the final studies was retrieved and no additional eligible studies were identified. A sample search strategy is provided in online supplementary table S1. PROSPERO registration: CRD42016042405.24
Supplemental material
Studies were eligible for inclusion if they met the following criteria:
Publication type—peer-reviewed articles presenting original research.
Study type—partial economic evaluations including cost analyses comparing two or more intervention strategies (not including cost of injury studies measuring the economic burden of injuries) and full economic evaluations assessing both costs and outcomes of two or more alternative injury prevention strategies (cost effectiveness analysis (CEA), cost utility analysis (CUA) or cost benefit analysis (CBA)); modelling studies and evaluations conducted alongside randomised controlled trials were included.
Prevention strategy—injury prevention in a sport or recreation setting.
Study population—studies focusing on children and adolescents ≤18 years of age, or broader studies with disaggregated results to allow the separate consideration of results for children and adolescents ≤18 years of age.
Key definitions used to guide inclusion criteria are included in online supplementary table S2. Duplicates were removed electronically and titles and abstracts were screened by two reviewers for all unique records (RL and CMI). Full-text review was completed by two reviewers (RL and CMI) and discrepancies resolved in discussion with a third party (GC).
Supplemental material
Data extraction from full-text articles included: intervention type(s) and comparator, intervention outcome(s), target age range, sample size, type of economic evaluation, perspective of economic evaluation, costs included, analytic approach and main results. All cost estimates were converted to 2016 US dollars (US$) using the GDP deflator25 of the original currency and then converted to US$ using the average 2016 rate.26
The Pediatric Quality Appraisal Questionnaire (PQAQ) was used to assess the methodological quality of each study.27 28 The PQAQ is a 57-item checklist designed to assess the quality of economic evaluations that focus on interventions and health services in a paediatric population. Each of the items relates to 1 of 14 domains: economic evaluation, comparators, target population, time horizon, perspective, costs and resource use, outcomes, quality of life, analysis, discounting, incremental analysis, sensitivity analysis, conflict of interest and conclusions.27 The assessment was conducted independently by two reviewers (RL and CMI) and discrepancies resolved through discussion and consultation with a third party (GC).
Results
Literature search
The initial search yielded 2380 records, from which eight articles were included in the final review (figure 1).
PRISMA flow diagram of study selection.
Study characteristics
A summary of the characteristics and results of included studies is presented in table 1. Three studies were published prior to 2000,29–31 two were published between 2001 and 2010,32 33 and three were published after 2010.34–36 Studies originated in Canada,34 35 New Zealand,29 33 USA,30 31 Norway32 and Bangladesh.36 Six studies29–33 36 focused on injury prevention in recreational activities and two on injury prevention in sports.34 35 Of the six recreation strategies, five were related to cycling29–33 and one was a swimming intervention.36
Characteristics of included studies
All eight studies were full economic evaluations; in seven studies, the authors stated that they performed CEAs and one stated they conducted both CBA and CEA.33 Five articles presented modelling studies whereby cost and effectiveness data were drawn from a variety of sources,29–33 two presented CEAs performed alongside a trial35 36 and 1 conducted a CEA alongside a prospective cohort study.34 All studies used intermediate outcomes related to injury prevention; one also used a final outcome measure, reporting the cost per Disability Adjusted Life Year (DALY) averted.36 Modelling studies used time horizons extending 3–5 years beyond the intervention29–33 and the studies using trial or cohort data used a 1 year time horizon.34–36
Summary of study results
Recreational cycling strategies
Five studies examined the cost effectiveness of strategies to increase bicycle helmet use as a means to decrease head injuries,29–33 prevent hospitalisations29 and save lives.29 These studies, published between 1993 and 2002, modelled the impact of regulatory approaches to increasing helmet use,29 30 33 helmet subsidies,31 and a hypothetical change in helmet use.32
Three studies reported costs per head injury avoided. Estimates ranged from $57 868 to $228 197 per head injury avoided for legislative, community-based or school-based interventions to increase helmet wearing in the USA.30 A compulsory helmet law in Norway was estimated to cost $9756 per head injury avoided for children 5–12 years of age and $31 514 per head injury avoided for children aged 13–18 years.33 A hypothetical increase in helmet wearing in Norway was estimated to cost $3109 per head injury avoided for children 3–13 years of age.32
Cost per hospitalisation avoided was reported by two studies.29 32 An increase in helmet wearing in Norway was estimated to cost $9893 per hospitalisation avoided among individuals aged 3–18 years.32 In New Zealand, the helmet wearing law was estimated to cost $3526–$4538 per hospitalisation avoided among children 5–12 years of age and $18 362–$21 640 among adolescents 13–18 years of age.29
Cost per life saved associated with the helmet wearing law in New Zealand was estimated as $94 314–$212 382 for children 5–12 years of age and $740 616–$872 794 for adolescents 13–18 years of age.29 Legislative, community-based and school-based interventions in the USA cost $28–$103 million per death avoided and $1.4–$5.3 million per year of life saved.30
A helmet subsidy study found that a subsidy of $9 per helmet for children aged 5–9 years would result in cost savings to society of $156 708–$796 143 over 5 years if helmet use was 50%, assuming a discount rate of 2% for savings accrued in the future.31
Sensitivity analyses indicated that results were sensitive to parameters related to helmet use,30 helmet efficacy,29 the type of healthcare costs included,31 the cost of helmets33 and the expected useful life of helmets.33
Recreational swimming strategies
One study examined the impact of three approaches to reducing swimming injury and drowning deaths in Bangladesh.36 Anchal, a day care programme for children aged 1–5 years aimed to reduce injury and drowning through increased supervision, and SwimSafe provided water safety and rescue skills for children aged 4–12 years. Combined, these two programmes are referred to as Prevention of Child Injuries through Social-Intervention and Education (PRECISE). Anchal alone was associated with an average cost per death averted of $32 398 and $953 per DALY averted. SwimSafe alone was estimated to cost an average of $3531 per death averted and $100 per DALY averted. The overall PRECISE programme was associated with an average cost of $14 782 per death averted and $425 per DALY averted. The results of the analysis were reported as being robust to alternative analytic approaches, but details were not provided.36
Injury prevention in sports
The two injury prevention studies assessed the cost effectiveness of disallowing body checking in hockey34 and a neuromuscular training (NMT) intervention for soccer players 13–18 years of age.35 Compared with body checking, a policy disallowing body checking among 11–12-year-old hockey players was associated with fewer injuries and cost savings of $219 per 1000 player hours over the course of one season.34 Compared with standard warm-up, NMT was associated with fewer injuries and cost savings of $499 per 1000 player hours over one season.35 Probabilistic sensitivity analysis showed that for 100% of simulated estimates the hockey intervention was both less costly and more effective.34 For the soccer study, 90% of simulated estimates showed the intervention was both less costly and more effective.35
Quality assessment
A summary of PQAQ scores is presented in figures 2–5. The full results of the quality assessment are presented in online supplementary table S3.
Supplemental material
Breakdown of the overall PQAQ instrument categories related to economic evaluation design and methods and the number of studies satisfying each criterion. PQAQ, Pediatric Quality Appraisal Questionnaire.
Breakdown of the overall PQAQ instrument categories related to study inputs and data sources and the number of studies satisfying each criterion. PQAQ, Pediatric Quality Appraisal Questionnaire.
Breakdown of the overall PQAQ instrument categories related to analysis and results and the number of studies satisfying each criterion. PQAQ, Pediatric Quality Appraisal Questionnaire.
Breakdown of the overall PQAQ instrument categories related to conclusions and conflict of interest and the number of studies satisfying each criterion. PQAQ, Pediatric Quality Appraisal Questionnaire.
Economic evaluation design and methods
All studies posed a research question that included costs and consequences, stated the approach to economic analysis, provided a rationale both for the interventions and the comparators selected and clearly stated the target population (figure 2). All but one study34 described the alternatives in detail. No study provided a clear event pathway, nor was a clear outline of the formal decision analysis provided in any of the studies. Five studies used data that were representative of the target population,29 33–36 one used study subjects that were not representative,30 one did not report on study subjects32 and for one study this item was not applicable for the analytic approach.31
The time period covered by an economic evaluation may be determined by the nature of the intervention or health condition, or data considerations in terms of available evidence related to long-term health outcomes. Seven studies reported the time horizon for both costs and outcomes;29–35 however, only six provided a justification for the time horizon used.29 31–35
Seven studies included a statement about the perspective of the analysis or provided enough information that perspective could be inferred.30–36 Five studies stated that they adopted a societal perspective.29 30 32 33 36 Two studies adopted both a healthcare system payer perspective and a ‘partial societal perspective’ which included publicly funded healthcare costs and some private costs and presented these results separately.34 35
It is standard practice to adjust for the differential timing of costs and consequences through the process of discounting or adjusting to present value.14 Of the included studies, four discounted costs and consequences where appropriate,29–31 33 one did not32 and one discounted benefits, but did not specify if costs were also discounted.36 Two studies considered costs and consequences only for 1 year and therefore discounting was not necessary.34 35
Inputs and data sources
The results of an economic evaluation depend on the type and quality of data and inputs used for both costs and consequences. Two studies were judged to have included all relevant costs for each alternative (figure 3).34 35 One study considered the impact of the intervention strategy on productivity costs or school and community resources where relevant.30 None of the eight studies included considered changes in future productivity or long-term health outcomes. All but one study described sources used to estimate the volume of resource use36 and all but two described the sources used to estimate costs.29 36
Analysis and results
All included studies described their primary outcome, and six provided a justification for the choice of outcome (figure 4).30 32–36 Six studies used data related to intervention effectiveness, rather than efficacy, and provided details of the cited studies.29 30 33–36 For one study, it was not clear whether outcome data were related to effectiveness or efficacy and no source was clearly cited.32 None of the included studies used utility-based outcomes measures, therefore none of the PQAQ items related to assessment of quality of life were applicable.
All studies measured both costs and outcomes in units appropriate for the indicated analytic technique. In five studies, it was clear that costs were aggregated correctly;30–32 34 35 for the remaining three, this was unclear.29 33 36 Seven studies provided estimates of incremental cost and benefit or provided enough information that this could be inferred.29–31 33–36 Two studies presented the results as ratios of incremental cost and benefit.29 36 For two studies, the intervention strategy was both less costly and more effective, making it a dominant alternative, so a ratio was not calculated.34 35 Three studies provided confidence intervals to characterise uncertainty around estimates of costs and intervention effectiveness.34–36 Six studies discussed important study assumptions.30 31 33–36 All performed sensitivity analysis to investigate the impact of assumptions or uncertainty around key parameters on study results, but only three provided a justification for alternate values used in these analyses.29 35 36
Conclusions and conflict of interest
All studies provided an answer to the study question posed (figure 5). In the discussion, all but one mentioned important study limitations33 and three discussed the generalisability of results to other settings or populations.31 32 36 Authors of six of eight studies provided information on their relationship with study sponsors29–31 34–36 and in five studies, the authors indicated that they had full control over methods and right to publish.29–31 34 35
The PQAQ scoring system does not involve generating a quantitative final score, instead a global assessment of the quality of the article is assigned. Of the eight studies reviewed, two were classified as poor quality,29 32 four were classified as fair30 31 33 36 and two were good.34 35 Global assessment scores were first assigned qualitatively and scores subsequently calculated as follows: a response of no or unclear was coded as 0, items fully reported in the text were scored as 1 and items that were inferred from the text or table were scored as 0.5. Thus, the total of all scoreable items for included studies ranged from 20.5 to 30.5 with a maximum score of 47 (given that no studies used a utility-based outcome measure). The qualitative scoring corresponded to scores of ≤21 for studies ranked as poor, 22–29 for studies rated as fair and 29–30 for studies rated as good.
Discussion
This review, based on a broad and comprehensive search strategy and including both full economic evaluations and cost analyses of recreation and sport-injury prevention interventions for children and adolescents, identified only eight studies for inclusion. The large range of estimates and mixed results make it difficult to draw strong conclusions about how results may be applied to different settings. On an individual study basis, an intermediate outcome such as injury prevented is difficult to interpret as the type and severity of injuries encompassed may vary widely. It is also challenging to apply this in a policy context where a decision maker is choosing between other types of injury prevention or health promotion strategies without a common measure of outcome. An outcome expressed in terms of cost per life saved or death averted can be useful in that many analyses across intervention types and sectors may use this approach. However, only three studies reported this type of outcome29 30 36 and debate remains over an acceptable threshold.
It is noteworthy that no studies used final utility-based outcome measures (though one used a DALY), which would facilitate comparisons across intervention types and could be more easily used to inform resource allocation decisions. Further, given the short time horizons, none included longer term impacts of injury such as future salary and productivity changes or future diminished health-related quality of life as a result of injury. These may be substantial, particularly when considering traumatic head injury or the development of osteoarthritis after knee injury which may require surgical intervention. Similarly, no study considered productivity costs to parents, such as days of missed work when a child is injured. Failure to consider health outcomes and consequences over a longer time horizon or household costs is likely to underestimate the benefits of injury prevention.
While most studies clearly described the alternatives being evaluated, none clearly described the event pathway or decision model used in the analysis. Improved reporting of these elements would lend credibility to model assumptions and facilitate replication of studies in other settings or populations, which is critical to understanding the generalisability of results. Similarly, few included provided confidence intervals for model inputs or incremental cost effectivess ratios (ICERs)34–36 and only half provided a justification for alternative values used in sensitivity analyses.29 34–36 Where health gains are achieved with the use of additional resources, it is particularly important to be clear about data uncertainty and variability and to understand the likelihood that an intervention will be cost-effective.
Quality assessment revealed deficits in both conduct and reporting of studies. Although the ability to identify patterns is limited by the small number of studies, there may be a general improvement in quality over time, mirroring the general trend across paediatric economic evaluations.28 Therefore, while the scores of the eight studies shows substantial room for improvement, this may reflect changes in methodological approaches and reporting standards over time.
Other systematic reviews of injury prevention interventions have found that such interventions can be cost saving and suggest they may represent good value for money.21 23 For both reviews, the majority of included studies relied on time horizons ranging from 1 to 5 years and cost utility studies were the minority. Both reviews reported substantial variation in methodological quality and reporting and called for improved study design and adherence to established reporting guidelines for economic evaluations. However, neither of these reviews focused exclusively on children and adolescents ≤18 years of age, a key contribution of the present review. There is a broad literature evaluating the effectiveness of injury prevention interventions in children across a variety of sports and settings. Indeed, several systematic reviews focusing on sport-based injury prevention indicate that a variety of approaches may be used to effectively reduce the risk of injury.13 15–18 The relatively low yield and low quality of studies included in our review suggests a need for more methodologically sound economic evaluations in this field in order to generate evidence to identify appropriate strategies for scaleup based on where there is good value for money. Where interventions are shown to be both effective in reducing injuries and cost saving, implementation and scale-up should be prioritised.
Our study has both strengths and limitations. A robust search strategy was used across a number of electronic databases. However, our search focused only on peer-reviewed academic journals and therefore some relevant grey literature or items published in books may have been missed. The quality assessment tool PQAQ was used because it is tailored towards assessing economic evaluations of interventions with a target population under the age of 18 who may rely on parents or caregivers to facilitate access to care and report on health states and resource utilisation. Our inclusion criteria required that results be reported separately for children and adolescents aged≤18 year in order to facilitate comparison across studies and ensure that results were applicable to our target age group. This resulted in the exclusion of four studies that included youth but did not have separate results available, and therefore a smaller number of final included studies and restriction of the range of topics covered. However, given our focus on assessing economic evaluations in a paediatric population, where there are some distinct methodological considerations related to the assessment of costs and consequences, we felt that this was justified.
Identifying priorities in resource allocation for injury prevention programming presents a unique challenge since costs of treatment and cost savings are seen in the health sector, but the cost of implementation is borne in other sectors. To build a case for the adoption of injury prevention programing, policy makers may need to consider innovative approaches to financing and develop new partnership models to fund effective interventions. Recent calls to intensify efforts to prevent and mitigate the effects of sport and recreation injuries are based on a growing body of evidence linking injury to adverse health outcomes. Where policy lags is in the adoption of evidenced-based approaches. Here, rigorous economic evaluations may provide the additional piece of evidence that policy makers require to advance the injury prevention agenda.
Conclusion
Strategies to prevent injuries in sport and recreation among children and adolescents can reduce injury and show potential to be cost saving or good value for money, even when only considering the short run. Taking a longer term view of the value of prevention is likely to amplify these conclusions. Our findings point to a dearth of evidence available to inform the injury prevention agenda and highlight the need for more work to assess the costs and consequences of strategies that are known to be effective. Future economic evaluations in this area should use time horizons sufficient to capture long term changes in health. Cost utility analyses should be prioritised to ensure that individual preferences for changes in health are captured and to facilitate comparison across intervention types.
What is already known on the subject
A variety of established and new strategies to prevent sport and recreation injuries among children and adolescents have been successfully implemented and shown to reduce injuries.
What this study adds
There is a lack of evidence around the cost-effectiveness of sport and recreation injury prevention interventions.
The current evidence base relies on short time horizons and therefore is likely to underestimate the impact of long-term changes in health outcomes associated with injury and prevention.
Future economic evaluations in this area should be based on longer time horizons in order to capture the full impact of long-term changes in health outcomes.
Acknowledgments
We would like to acknowledge the support of Karen V MacDonald in her role as research manager with Dr Deborah Marshall’s team. We would also like to acknowledge the advice provided by Dr Diane Lorenzetti, research librarian, which helped inform our search strategy.
References
Footnotes
Contributor RL, GC and DAM designed the study and search strategy. RL and CM-I completed screening, data extraction and quality assessment with input from GC and DAM. CM-I conducted analysis and drafted the manuscript. RL, GC and DAM provided input on the manuscript. All authors approved the final manuscript.
Funding CM-I is funded by a Postgraduate Fellowship from Alberta Innovates. RL received an undergraduate summer student scholarship from the Alberta Children’s Hospital Research Institute, supplemented by the Alberta Program in Youth Sport and Recreational Injury Prevention funded by an Alberta Innovates Collaborative Research and Innovation Opportunities grant, to support initial work on this project. DAM is supported by a Canada Research Chair in Health Systems and Services Research and the Arthur J.E. Child Chair in Rheumatology.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.