Article Text

Predictors of health-related quality of life following injury in childhood and adolescence: a pooled analysis
Free
  1. Joanna F Dipnall1,2,
  2. Frederick P Rivara3,
  3. Ronan A Lyons1,4,5,
  4. Shanthi Ameratunga1,6,7,
  5. Mariana Brussoni8,9,
  6. Fiona E Lecky10,11,
  7. Clare Bradley12,13,
  8. Ben Beck1,
  9. Jane Lyons4,
  10. Amy Schneeberg9,14,
  11. James E Harrison15,
  12. Belinda J Gabbe1,4
  1. 1 School of Public Health and Preventive Medicine, Monash University, Melbourne, Victoria, Australia
  2. 2 Institute for Mental and Physical Health and Clinical Translation, School of Medicine, Deakin University, Geelong, Victoria, Australia
  3. 3 Departments of Pediatrics and Epidemiology, and the Harborview Injury Prevention and Research Center, University of Washington, Seattle, Washington, USA
  4. 4 Health Data Research UK, Swansea University, Swansea, UK
  5. 5 National Centre for Population Health and Wellbeing Research, Swansea University, Swansea, UK
  6. 6 School of Population Health, University of Auckland, Auckland, New Zealand
  7. 7 Kidz First Hospital and Population Health Directorate, Counties Manukau District Health Board, Auckland, New Zealand
  8. 8 Department of Pediatrics, School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
  9. 9 British Columbia Injury Research and Prevention Unit, British Columbia Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
  10. 10 Centre for Urgent and Emergency Care Research, School of Health and Related Research, University of Sheffield, Sheffield, UK
  11. 11 Emergency Department, Salford Royal Hospital, Salford, UK
  12. 12 South Australian Health and Medical Research Institute, Adelaide, South Australia, Australia
  13. 13 College of Medicine and Public Health, Flinders University, Adelaide, South Australia, Australia
  14. 14 School of Population and Public Health, University of British Columbia, Vancouver, British Columbia, Canada
  15. 15 Flinders Institute for Health and Medical Research, Flinders University, Adelaide, South Australia, Australia
  1. Correspondence to Dr Joanna F Dipnall, School of Public Health and Preventive Medicine, Monash University, Melbourne, VIC 3004, Australia; jo.dipnall{at}monash.edu

Abstract

Background Injury is a leading contributor to the global disease burden in children and places children at risk for adverse and lasting impacts on their health-related quality of life (HRQoL) and development. This study aimed to identify key predictors of HRQoL following injury in childhood and adolescence.

Methods Data from 2259 injury survivors (<18 years when injured) were pooled from four longitudinal cohort studies (Australia, Canada, UK, USA) from the paediatric Validating Injury Burden Estimates Study (VIBES-Junior). Outcomes were the Paediatric Quality of Life Inventory (PedsQL) total, physical, psychosocial functioning scores at 1, 3–4, 6, 12, 24 months postinjury.

Results Mean PedsQL total score increased with higher socioeconomic status and decreased with increasing age. It was lower for transport-related incidents, ≥1 comorbidities, intentional injuries, spinal cord injury, vertebral column fracture, moderate/severe traumatic brain injury and fracture of patella/tibia/fibula/ankle. Mean PedsQL physical score was lower for females, fracture of femur, fracture of pelvis and burns. Mean PedsQL psychosocial score was lower for asphyxiation/non-fatal submersion and muscle/tendon/dislocation injuries.

Conclusions Postinjury HRQoL was associated with survivors’ socioeconomic status, intent, mechanism of injury and comorbidity status. Patterns of physical and psychosocial functioning postinjury differed according to sex and nature of injury sustained. The findings improve understanding of the long-term individual and societal impacts of injury in the early part of life and guide the prioritisation of prevention efforts, inform health and social service planning to help reduce injury burden, and help guide future Global Burden of Disease estimates.

  • disability
  • public health
  • longitudinal

Data availability statement

Data are available on reasonable request. The data included in this project are not freely available. Requests for access to data from the participating datasets would need to be directed to the relevant data custodian.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Introduction

For over 10 years, there have been urgent calls to address issues surrounding injuries in children and adolescents across the globe.1 2 Injury is a leading contributor to the global disease burden in this population.3 Tens of millions of children around the world are hospitalised every year for non-fatal injuries, representing a high burden on healthcare worldwide and placing children at risk for adverse and lasting impacts on their health-related quality of life (HRQoL) and development.4

The measurement of HRQoL following injury is key in quantifying longitudinal pathways to recovery and subjective burden in survivors. However, studies of HRQoL postinjury in children and adolescents are uncommon, often restricted to single centres5 or subgroups of the population (eg, traumatic brain injury (TBI), spinal cord injury, sports-related concussion or fracture6 or multiple trauma). These studies have often involved small sample sizes, and wide variation in the number and timing of follow-up assessments. A number of studies have experienced issues with loss to follow-up7 8 resulting in potential biases8 9 and thereby restricting the capacity of studies to identify important predictors of postinjury HRQoL across a broad range of injuries in children and adolescents.

The opportunity to pool longitudinal data from multiple sources can increase sample size, enabling a broader coverage of injuries, improved generalisability, more precise estimates and greater statistical power to identify predictors of HRQoL following injury for children and adolescents. The aim of this study was to characterise and identify the key predictors of HRQoL following injury in childhood and adolescence across demographic groups and categories of injuries using pooled data of injury survivors from cohorts from four high-income countries.

Methods

Included datasets

Data from four longitudinal cohort studies of paediatric injury survivors associated with the paediatric Validating Injury Burden Estimates Study (VIBES-Junior)10 were pooled for this project (table 1) and integrated using the Data Integration Protocol in 10 Steps.11 The VIBES-Junior project is an international collaboration of injury research experts aimed at quantifying the burden of injury in children and adolescents. The pooled studies were all prospective cohort studies and included multiple injury types, collected outcomes at multiple time points after injury and included a standardised measure of HRQoL. Injury diagnoses coded using International Classification of Diseases (ICD) enabled the characterisation of the injuries sustained and aid characterisation of external cause of injury codes. Two studies included a standardised measure of preinjury HRQoL.

Table 1

Summary of injury-specific cohort studies included in research*

The Victorian State Trauma Registry (VSTR) is a population-based trauma registry that captures data about all major trauma patients in the state of Victoria in Australia.12 Children who survived to hospital discharge were followed up postinjury (N=996). The US Children’s Health After Injury (CHAI) included children with mild, moderate and severe TBI or with upper extremity injuries who presented to a set of US hospitals.13 The study collected the recalled preinjury HRQoL measures at baseline and postinjury HRQoL at follow-up among 924 children in the pooled study. The UK Burden of Injury (UKBOI) was a study of injured individuals with children recruited from emergency department (ED) presentations and hospital admissions in four UK centres (N=181).14 The British Columbia Children’s Hospital Longitudinal Injury Outcomes (BCCH-LIO) study included children who attended the British Columbia Children’s Hospital in Canada for an injury5 and collected baseline recalled preinjury HRQoL measures and were followed up postinjury (N=365).

Measures

The Paediatric Quality of Life Inventory (PedsQL) was used to measure HRQoL at baseline (ie, preinjury status as recalled postinjury), 1, 3–4, 6, 12 and/or 24 months following injury. To allow for non-linearity, time was treated as categorical and the PedsQL scores at times 3 and 4 months for the CHAI and BCCH-LIO cohorts were grouped together to represent 3–4 months. The PedsQL is considered an appropriate instrument for assessing HRQoL in paediatric trauma populations across a broad age range,15 can discriminate between HRQoL postinjury for injuries of varying severity for injured children,16 and exhibits good reliability and construct validity.17

The PedsQL is a 23-item generic instrument formulated to measure physical, mental, social, emotional and school functioning. A five-point response scale from never to almost always is used to assess the extent to which the child has been affected in the previous month. For consistency across the cohorts, parent item scores were used and reversed and transformed to produce the three PedsQL health summary scores: total health (all 23 items); physical health (physical functioning: 8 items) and psychosocial health (consisting of emotional, social and school functioning: 15 items). Each summary score was calculated by averaging across the relevant score items, but only if more than 50% of the items on the scale were available. Summary scores ranged from 0 to 100, with higher scores representing better function. A difference of ≥4.5 has been shown to be clinically meaningful.18

Demographic characteristics were collected at baseline and included sex, and age group separated into five groups to align with the WHO classification2 (0–4 years, 5–9 years, 10–14 years, 15–17 years). Differing socioeconomic status (SES) quintiles for each cohort were collapsed into three groups (low=1–2, moderate=3–4, high=5) (online supplemental table S1). Mechanism of injury was harmonised to a binary measure (transport vs non-transport) to ensure this measure was consistent across the cohorts. The transport group included those who were a motor vehicle occupant, pedestrian, or on a motorcycle or bicycle at the time of injury. The non-transport group included children injured in falls, struck by/against an object or person and other mechanisms. Care was grouped into ED presentation and discharge vs hospital admission. The distribution of the Injury Severity Score (ISS) was asymmetric so was collapsed into tertiles and the intent of injury grouped into intentional (including self-harm, maltreatment and interpersonal violence), unintentional and intent not known. Diagnoses and external cause codes were classified using the ICD 10th Revision (ICD-10). The CHAI data were mapped from the ICD ninth Revision (ICD-9) to the ICD-10. All diagnosis codes were mapped to the 2013 Global Burden of Disease (GBD) study injury health states.19 Due to small cell counts, these injury health states were then collapsed into 17 binary variables (online supplemental table S2). Comorbidities present at the time of injury were categorised into no comorbidities vs 1 or more comorbidities based on the 27 health conditions described by Mitchell et al.20

Statistical analysis

Data were summarised using frequencies and percentages for categorical variables. Mean and SD were used for continuous variables. Independent t-tests were used to compare the mean PedsQL total, physical and psychosocial scores with published norms for healthy children.21 Bonferroni correction was used for the t-test analysis for healthy populations with a p<0.003 considered significant.

The effect of demographics and injury related specific risk factors on the PedsQL total, physical and psychosocial scores were modelled separately for each measure using a mixed effects linear regression with random intercepts and slopes at the patient level to adjust for the correlation between patients. Nested random effects within the cohort were tested and not found to significantly improve the models, so the more parsimonious solution was used. Missing data associated with the model covariates were 7% (online supplemental table S3). Missing data in the outcome measures varied due to the pooling of cohorts and so cohort were controlled for in the models. Mixed model analysis with longitudinal data has been shown to be suitable in handling missing data compared with using multiple imputation which has been found to potentially produce unstable results.22 Time was treated as a discrete categorical variable so that no assumptions about the mathematical function over time was prescribed.

Analyses were performed using Stata V.16.0 (StataCorp). The 95% CIs were evaluated for interpretation and the precision of estimates and/or strengths of associations for all analyses other than the t-tests described above.

Results

Overview of cohort

Across all cohorts and time points 7% of participants did not provide a PedsQL score (online supplemental table S3). The pooled cohort study population (n=2259) was predominantly male (66%) (table 2), from low to moderate SES groups (80%), and under 12 years of age (62%). The mean (SD) age was 8.7 (5.2) years. Most cases were not transport related (74%), 61% of participants were admitted to hospital, and 90% had no reported pre-existing comorbidities at the time of the injury. A TBI was sustained by 48% of participants, with fractures (33%), and contusions or open wounds (19%) the next most common types of injury.

Table 2

Patient characteristics

Outcomes at each time point

The unadjusted distributions of the PedsQL scores at each time point postinjury are presented in table 3. The mean baseline (preinjury) PedsQL physical score of paediatric injury patients was clinically higher than that in the normative population of children (table 3). The mean PedsQL total score and PedsQL psychosocial score postinjury were lower than the norms across all time points postinjury, with a clinical difference at 1 month. The mean PedsQL physical score postinjury were lower than the norms across 1-month and 6-month postinjury, exhibiting a clinical difference at these time points. The PedsQL physical scores were relatively similar at 12 months and 24 months postinjury being consistent with the norms.

Table 3

Comparison of PedsQL scores between injured children and healthy population norm

Predictors of PedsQL scores

The results of the adjusted PedsQL mixed effects models (table 4, figures 1–3) indicated that age group, SES, mechanism of injury, comorbidity status and sustaining a spinal cord injury and/or fracture of the vertebral column were consistent predictors of total, physical and psychosocial health.

Table 4

PedsQL linear mixed effects models (adjusted for VIBES-Junior cohort)

Figure 1

PedsQL Total Mean Difference Coefficients (Coef) for each predictor. ED, emergency department; SES, socioeconomic status; TBI, traumatic brain injury.

Figure 2

PedsQL Physical Mean Difference Coefficients (Coef) for each predictor. ED, emergency department; SES, socioeconomic status; TBI, traumatic brain injury.

Figure 3

PedsQL Psychosocial Mean Difference Coefficients (Coef) for each predictor. ED, emergency department; SES, socioeconomic status; TBI, traumatic brain injury.

The mean PedsQL score following injury was higher with higher SES status. The mean PedsQL score after injury was lower if children had sustained injuries from a transport related incident compared with a non-transport related incident, intentional injuries compared with unintentional injuries, and/or had one or more comorbidities at the time of injury compared with participants without any reported comorbidities. Having either a spinal cord injury or fracture of the vertebral column was associated with lower mean PedsQL scores following injury compared with participants without these types of injuries. Participants who sustained a moderate to severe TBI and/or sustained a fracture of femur, patella, tibia, fibula, ankle were also associated with lower mean PedsQL score following injury compared with participants without these types of injuries.

There were differences between the PedsQL physical and psychosocial score models. The mean physical PedsQL scores across time was higher compared with the mean psychosocial PedsQL scores. Girls had lower mean PedsQL physical score after injury compared with boys. Participants who sustained a fracture of the femur, fracture of the pelvis and/or burns (including lower airways) had lower mean PedsQL physical score following injury compared with participants who had not sustained these types of injuries. Participants who had sustained asphyxiation/non-fatal submersion had lower mean PedsQL psychosocial score after injury compared with those who had not sustained this type of injury. This was also the case for participants who had sustained injuries of muscle/tendon/dislocations.

Discussion

The findings of this longitudinal pooled data study have advanced our understanding of the factors that influence HRQoL, and the variability in the pattern of physical and psychosocial functioning recovery, in children and adolescents following hospital care for injury. All mean HRQoL scores postinjury were clinically lower than the norms at 1 month, and this clinical difference continued up to 6 months for the physical functioning. Physical functioning was more likely than psychosocial functioning to be at a higher level by 24 months after injury. Sex and SES as well as the type, intent and mechanism of injury were associated with HRQoL outcomes.

There were several consistent predictors of total HRQoL, physical and psychosocial functioning following injuries in childhood. Even after the adjusting for the severity and mechanism of injury, low SES was associated with lower mean HRQoL across time compared with mid/high SES. This is consistent with a number of studies across a variety of paediatric problems, and with findings of recent systematic reviews of HRQoL and functional outcomes in children after injury23 24 and with the longitudinal Brain Injury Incidence and Outcomes In the NZ Community study.25 There are several possible explanations for this disparity. Children from lower SES area may have difficulties accessing adequate postinjury treatment, particularly for injuries requiring extensive rehabilitation; the families may be dealing with additional stressors (eg, food insecurity, unemployment, challenging living conditions), that may make it difficult for parents and caregivers to adequately support the injured child; or a combination of these situations impeding recovery. While all the countries from which the data are pooled are high-income countries, the issue of the injury poverty trap is well described elsewhere (eg, Vietnam).26 The increased probability that some families are tipped into poverty due to out of pocket expenses can lead to more adverse outcomes post injury. While this pooled analysis was unable to investigate the impact of changes in SES post injury over time, the collection of this information may be beneficial for future studies. Whatever the reason, the strength and persistence of this finding is concerning and efforts to address it should be a priority for policymakers to overcome structural determinants of health including income-related barriers requiring health service attention alongside public policies focusing on poverty reduction.

Several injury-related factors were associated with poorer HRQoL outcomes. The transport related cases in our study included children and adolescents injured while in a motor vehicle (64%), or as a pedestrian/bicycle rider (36%). Transport injuries have been found to have poorer outcomes than most other types of mechanisms previously.7 27 The impact of a transport related injury involves greater velocity and mass load and, in combination with the potential lack of safety devices (eg, seatbelts, child restraints, helmets), the injuries sustained generally have worse outcomes (eg, moderate to severe TBI, spinal cord lesion and/or fracture of the pelvis). Cost can be a barrier to the use of child restraints and parents with higher SES and education are more likely than others to use car seat restraints.28

Pre-existing conditions impacted on a child’s overall level of functioning, both for the physical and psychosocial components. While commonly identified as a predictor of poorer outcomes in adult trauma patients,29 pre-existing conditions have not frequently been studied in children.19 30 Our finding highlights the importance of appropriate implementation of protocols to identify comorbidities at presentation of the injury for children and appropriate intervention strategies post injury.

Children with a spinal cord lesion or vertebral injury had a lower mean HRQoL compared with those who had not sustained this injury. Traffic incidents have been reported to be the most frequent cause of cervical spine injury in children.31 Spinal cord injury in childhood has been shown to present challenges that are physical (eg, loss of physical control, loss of independence) and psychological (eg, depression and anxiety disorder).32 Children and adolescents who had sustained spinal cord injury engage in fewer physical and social activities than their peers and tended to participate in more sedentary and informal activities,33 which might adversely affect key physical and psychosocial developmental milestones.

Consistent with literature, children who had sustained a moderate to severe TBI experienced relatively poor outcomes. In particular, psychosocial deficits in cognitive and behavioural domains after TBI have been found to negatively impact on HRQoL.34 For example, communication and self-care abilities in children with these types of injuries have been found to not improve by 24 months post injury.35

Of concern, children in our study who had sustained an intentional injury had lower mean HRQoL outcomes compared with an unintentional injury. Child maltreatment and intentional self-harm has been linked to a range of mental health problems, including depression and anxiety disorders, with lower SES compounding some of the risks involved.36 37 Further data and research into the relationship of pre-existing HRQoL with intentional injuries (eg, suicide or child abuse) is needed. In 2002 researchers called for more investigation into the relationship between neighbourhood and community variables with both intentional and unintentional childhood injuries38 and the addition of these variables in future analysis may help to inform prevention strategies.

We found specific predictors related to physical and psychosocial health. Females had poorer physical health outcomes than males following injury. Researchers have found that females tend to have poorer perceptions of their health and physical well-being than males.39 40 More research is needed to address these issues for injured girls and provide interventions to improve this situation. Children who had sustained burns also experienced poorer physical health outcomes. Studies of HRQoL in children after burns are few, but a systematic review by Spronk et al found that children with more severe burns, or with a facial or hand burn had poorer HRQoL.41 Severe burns can result in systemic damage, often require multiple surgical procedures involving readmission for reconstructive surgery and impact on the physical functioning of the area(s) affected.42

Our study found that children who had sustained asphyxiation/non-fatal submersion experienced poorer psychosocial health outcomes. Asphyxiation/non-fatal submersion may have consequences due to hypoxia and subsequent brain damage. A prospective 5-year follow-up cohort study of near-drowning children found 22% of the patients showed behaviour problems, poor communication, executive function and learning difficulties.43 The impact on a child’s psychosocial functioning of this type of injury is complicated as factors such as intent, sex, age and socioeconomic level and parents’ potentially influence this outcome.

A major strength of our analysis was the pooling of injury-specific and primary data of patient-centred outcomes at four time points after injury, resulting in a robust sample size to cover the most commonly used injury classifications defined by GBD 2013.19 However, dealing with multiple and international studies has some limitations. The included data sets differed in terms of follow-up post injury and had an imbalance of sample size across time points. The inclusion criteria differed across data sets which resulted in differing proportions of cases with certain injuries (eg, TBI) and multiple injuries across the datasets and the collection of data encompassed different calendar years, types of healthcare systems and levels of health insurance. However, this diversity allowed our models to account for a variety of injuries and our models were adjusted for data source to ensure estimates were independent of these inherent differences in time and setting. The potential covariate of race/ethnicity was not included in the models as this measure was not consistently collected in each cohort. The linear mixed effects models were applied to asymmetric distribution data of the PedsQL scores, with ceiling effects, rendering some of the model assumptions violated.44 We applied a sensitivity analysis, using mixed effects ordinal models across five categories of each of the PedsQL scores and found generally consistent results with the key predictors reported in this manuscript (online supplemental table S4). This study reports factors influencing the HRQoL outcomes of children and adolescents following injury in high-income countries and the patterns may differ in low to middle income countries due to the impact of SES, nature of injuries sustained and constrained health systems.

Conclusion

We used pooled data from multiple longitudinal studies to explore predictors of HRQoL over time for children and adolescents recovering from injury. Demographic factors, including sex and SES, and injury factors were identified as important predictors of HRQoL. These inequities in HRQoL after injury should be considered in childhood injury prevention prioritisation for strategic policy and public health programmes. The nature of the injury plays an important role in the HRQoL for children and adolescents post injury, with the physical and psychosocial functioning post injury differing according to the injury sustained. This information may guide the prioritisation of prevention efforts, inform health and social service planning to help reduce this burden, and help guide future GBD estimates.

What is already known on the subject

  • Injury is a leading contributor to the global disease burden in children.

  • Injury places children at risk of long-term adverse impacts on their health-related quality of life and development.

  • Previous studies have often been limited to single centres and/or subgroups of the population, and longitudinal analyses into health-related quality of life outcomes limited by sampling and statistical issues, making it difficult to produce predictive models with an overall preventative focus.

What this study adds

  • This study has advanced our understanding of the factors that influence health-related quality of life following injury in children and adolescents.

  • The pooled analysis gave the ability to handle more complex models, produce more precise estimates, and cover more types of injuries than prior studies, making results generalisable.

  • This study found the patterns of recovery of total, physical and psychosocial functioning in children and adolescents following injury varied with socioeconomic status, sex, intent and nature of injury.

  • Findings of this study are important to include in future Global Burden of Disease estimates and have the potential to guide prioritisation of prevention efforts and inform health and social service planning to reduce paediatric injury burden.

Data availability statement

Data are available on reasonable request. The data included in this project are not freely available. Requests for access to data from the participating datasets would need to be directed to the relevant data custodian.

Ethics statements

Patient consent for publication

Ethics approval

The project was approved by the Monash University Human Research Ethics Committee (project #12311) and was conducted in compliance the NHMRC National Statement on Ethical Conduct in Human Research (2007) and the Note for Guidance on Good Clinical Practice (CPMP/ICH-135/95).

Acknowledgments

The authors would like to extend their gratitude and acknowledgements to all study participants and study team members for their time and energy spent on this project. The authors would like to extend their gratitude and acknowledgements to Dr Isabel Canette, Principal Mathematician and Statistician at StataCorp for advice with elements of Stata code used in the analyses.

References

Supplementary materials

Footnotes

  • Twitter @mbrussoni, @DrBenBeck, @EmergTrauma

  • Contributors All authors conceptualised and designed the study. JFD carried out the initial analyses, wrote the original draft of the manuscript and reviewed and edited the final manuscript. BJG, FPR, RAL and MB verified the underlying data. All authors critically reviewed the manuscript for important intellectual content. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work. JFD accepts full responsibility for the work and/or the conduct of the study related to this manuscript, had access to the data, and controlled the decision to publish.

  • Funding VIBES-Junior project: National Health and Medical Research Council of Australia (NHMRC-APP1142325); UKBOI: Policy Research Programme, Department of Health (#0010009); VSTR: Department of Health and Human Services (Victoria), Transport Accident Commission (TAC); CHAI: US Centers for Disease Control and Prevention; BCCH-LIO: Canadian Institutes of Health Research (#TIR-104028), Michael Smith Foundation for Health Research. Mariana Brussoni: BCCH Research Institute and Michael Smith Foundation for Health Research Scholar Award. Belinda Gabbe: Australian Research Council Future Fellowship (FT170100048).

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.