Unintentional injuries pose a serious health threat to children, with young children particularly vulnerable to injuries that occur in and around the home.1^–3 Across the developed world, unintentional injury is the leading cause of hospitalisation and death for children,4^–6 and many children survive their injuries, with a lifetime of physical, emotional and socioeconomic consequences. Further empirical research and prevention programmes must target children because of these implications.

A lapse in, or lack of, parental supervision has been associated with unintentional injuries and death from drownings and near-drownings, pedestrian injuries and falls in children aged 0–14 years.7 Studies measuring parental supervision have drawn conclusions about the implications that the level of supervision has for child injury risk using self-report or mixed methods.8^–10 Other studies have examined the behaviour of injured and uninjured children and the supervisory practices of their parents in a contrived hazard setting.1112 Unfortunately, many studies have made inferences about the importance of parental supervision without actually measuring it.13^–18 Few studies have directly related supervision to child injury risk, despite the individual and societal burdens associated with unintentional injury as a leading childhood health problem.12

Although methodological and conceptual issues related to care giver supervision and its role in childhood injury have been addressed in literature reviews,719 the range of study methods and approaches used to investigate supervision and childhood injury risk have not previously been systematically and comprehensively examined. There is no standardised measure of the relationship between supervision and children’s injuries,20 and there are significant challenges in developing and implementing such a measure.719 It has recently been argued that systematic reviews of the methods used in studies of injury and supervision are required.21

This systematic review addresses this information. It describes the methodologies used in studies of supervision and injuries or injury risk in young children, and highlights the advantages and disadvantages of different study methods. Quality research designs are identified which, should they be adopted in future studies, have the potential to provide a comprehensive understanding of supervision and its role in preventing unintentional childhood injuries. It is beyond the scope of this paper to describe the actual findings from the reviewed studies.

METHODS

Inclusion and exclusion criteria

Studies were included in this systematic review if they met all of the following criteria: (1) explained aspects of supervision and/or evaluated the effectiveness of supervision using an outcome measure (irrespective of the definition of supervision); (2) targeted children aged 0–14 years; (3) reported on any form of supervision provided by parents/care givers, guardians or siblings to prevent unintentional injuries to children; (4) included a primary outcome measure of injury risk and/or injury incidence. Identified studies included investigations of public recreational child play settings and aquatic environments, self-reported or medically attended unintentional injury, and care giver safety practices.

Studies in formal settings (eg, crèche, child care, school) were excluded as parents or guardians were not responsible for providing supervision in these settings. Studies of supervisory neglect were excluded because the point at which absence of adequate supervision is classified as neglect is often inconsistent and unclear.19 Studies of child poisonings and burns were also excluded because such injuries are primarily addressed via non-supervisory strategies such as removal, controlled use and redesign of products.2223

Identification of studies

Academic Search Premier, AUSPORT, AUSPORTMed, CINAHL, Cochrane Database of Systematic Reviews, ERIC, Health Source Consumer, Health Source Nursing, PsycINFO, PubMed, SafetyLit, Scopus, SPORTDiscus, Springer Link and Web of Science databases were searched for potentially relevant studies. Multiple databases were selected because of the growth in journals publishing studies relevant to injury prevention and safety promotion.24 Although English language search terms were used, all databases were searched for all languages, and from the earliest records available (range 1800 (The Cochrane Library) to 1989 (AUSPORT and AUSPORTMed)) until the end of December 2007. No foreign language studies were identified. Electronic searching was supplemented by hand-searching of relevant journals, reference lists of identified studies and bibliographies of systematic and non-systematic reviews.

In order to ensure that all relevant journal articles about injury and supervision were identified, a large number of search keywords and synonyms were adopted,2526 and the article title was broken down into individual concepts and components to ensure that the search was retrieving crucial studies.26 Where possible, the shortened forms of textwords were used so that both singular and plural forms were included in search findings.

The search terms “injury prevention” or “unintentional injury” or “injury risk”, in conjunction with “parent”, “guardian” or “carer” and “supervision”, were used. “Supervision” was further combined with “infant” or “child” or “toddler” or “young children”. “Parent”, “guardian” or “carer” supervision was searched, and then used in juxtaposition with “accident”, “near miss” and “near accident”. The derivatives of applicable terms (eg, child, children) were incorporated.

Supervision in aquatic and active child play environments was of particular interest because of the diverse levels of risk and potential for injury in different settings, and the relative levels of supervision required to prevent injury.27 Therefore, after initial searches, the above search terms were combined separately with “aquatic”, “water”, “pool”, “beach”, “lake”, “river” “drowning” and “near drowning”, “play environment”, “playground” and “pedestrian” terms.

Studies identified with this search strategy were screened by the first author (a PhD student), who examined study titles, abstracts and keywords. Obvious exclusions were removed, and the full texts of potentially relevant studies were obtained and assessed against the inclusion criteria to determine the appropriateness of each study. Uncertainties about appropriateness of studies for inclusion were resolved by consensus with coauthors. Classification of the studies was agreed to by all authors including a biostatistician/epidemiologist (CFF) and exercise scientist (JB), who both had experience in the conduct of injury studies.

Quality assessment

A methodological quality rating scale assessing common study designs in public and clinical health issues was identified.28 To reflect a public health approach, a classification scale for supervision studies was derived from this. Construct validity was established, as the classification was based on the National Health and Medical Research Council levels of evidence28 and a previous classification of injury studies.29 This scale allowed the methodological quality of the studies to be critically analysed and classified according to capacity to eliminate or minimise bias (table 1).

RESULTS

Screening of the study titles, abstracts and keywords identified 112 potentially relevant citations. After screening of the full text, 30 studies met all inclusion criteria. Four studies used a mixed-method research approach,30^–33 and these studies are listed under both types of evidence in table 1. Thirteen studies were conducted in the home,310^–12303134^–40 eight in aquatic environments,41^–48 four in supermarkets,849^–51 two in pedestrian settings,3233 and one in a playground environment.9 Two were general injury studies.2752 Studies that did not meet inclusion criteria were eliminated largely because no level of parental or care giver supervision was discussed or the effectiveness of supervision was not evaluated against an outcome-based measure.

The context, method and duration of studies influenced their sample size, the number of children and supervisors in the included studies ranging from 1911 to 390.8 The heterogeneity in design and context makes it difficult to compare across studies, but the sampling of children and supervisors is often identified as a limiting factor by authors.1027353839 For example, although the findings of most studies may be representative of the population in which they were conducted, they cannot be generalised to other communities because of sample selection biases, such as socioeconomic and parental employment status and parental interest in the research area. Many studies382732333638394549^–51 adjusted or assessed for confounding factors, including child and supervisor characteristics such as gender and age, at the design or analysis phase. Assessment of potential confounders is imperative, as inadequate assessment can damage the strength of findings/results, threatening the validity of causal inferences drawn from the data.53

Owing to the relative infancy of supervision research, the methods reported in the studies varied substantially (table 1). The identified methods, as well as their associated advantages and shortcomings, are presented in the following paragraphs. They are presented in the order that they appear in table 1 according to their quality, from least to most robust.29

Self-report was the most common data-collection method and was implemented through interviews, questionnaires or parent-recorded diaries. A number of studies required participants to self-report on hypothetical scenarios,102735 report their individual supervisory behaviours in specific contexts (also referred to as participant event monitoring),33237^–39 or complete questionnaires about their supervision and attitudes toward supervision.391012202730343845 The results of self-report studies must be considered with care, as they rely on individuals reporting their own behaviours, and response bias is of concern.54Table 1 shows that about half of identified studies were of this type.

A range of survey instruments were used across studies adopting the self-report approach, and some surveys were used in more than one study. The quality of surveys is usually assessed through documented validity and reliability and, where this information was available, it is indicated in table 2. A number of non-injury/supervision-specific surveys (eg, The Family Information Questionnaire, Big-Five Inventory and Parent Health Locus of Control) were also used.3303855 However, as their focus is beyond the direct measurement of supervision and injury risk, they are not included in table 2. Ideally, studies should use validated and reliable surveys so as not to undermine the value of the data collected.

Reconstruction studies of the incidents leading to injury outcomes and their causal factors obtained through case series reviews have often been used to understand parental supervision in aquatic settings.41^–4446^–48 Such studies can suggest when absent or inadequate supervision was a contributing factor in child drowning and near-drowning incidents. As the information is generally collected retrospectively, recall bias and observation of the scene after the incident mean that causal relationships cannot be firmly established. These studies are also limited when information on injuries or key events is missing.58

Observational studies overcome some of the biases inherent in the above methodologies. In laboratory settings,1112 observational study methods are often called contrived hazards because of the controlling of the child’s exposure to risk. Such studies involve creating a naturally appearing situation, with naturally emerging hazards (that have been modified to eliminate the risk of injury) in a controlled environment. Examples include mothers and toddlers being unobtrusively observed and videotaped while they wait in a simulated office12 or playroom setting.11

A number of studies led by Harrell849^–5159 have undertaken “natural observations” of children and their supervisors in the supermarket setting. As table 1 shows, this approach provides higher-quality evidence than contrived hazard methods, because it occurs in an uncontrolled natural setting with minimal disturbance, influence or intrusion on the participants’ behaviours through the observation process.60

Unfortunately, observational studies of interventions relating to supervision are rare to date, with only two studies identified, conducted in the home and supermarket.3149 No prospective non-randomised controls, randomised controlled trials (RCTs) or meta-analysis studies were found.

Many studies3930^–3436^–39 have adopted mixed-method approaches. Such studies can provide a more thorough analysis of injury risk and supervision than would be available from single-method studies.3839 However, the quality of evidence is only as high as the best method implemented, unless triangulation is conducted. Triangulation involves the use and comparison of several different research methods to explore the same phenomenon, and is advantageous as it provides the researcher with a much more rounded picture of what is being studied.6162 Unfortunately, most of the mixed-method studies in this systematic review did not take advantage of the potential to compare or overlap data to validate and confirm data obtained by various methods.

DISCUSSION

This systematic review has highlighted that self-report, observational (in laboratories or natural settings) and case series methods are the most commonly used in child supervision and injury studies. Accordingly, the quality of the information obtained from the studies varied. Very few studies provided robust evidence of the extent to which injury risk is associated with levels of supervision. No RCTs were identified, and, although this design is considered to provide the best evidence,5863 it is likely to be unfeasible in this context because of the ethical difficulty of manipulating supervision levels and children’s exposure to hazards.58 Further, as the primary outcome of interest (ie, injury) is rare in some contexts,58 RCTs may be prohibitively expensive.

Meta-analyses provide a synthesis of studies with overall effect sizes quantified.64 Although stronger conclusions can be drawn, as the findings emerge after synthesis and evaluation of the relationship among results from many individual studies,65 no meta-analyses were identified. Meta-analysis requires empirical quantitative data rather than theoretical contributions or qualitative data; findings need to be configured in a comparable statistical form, and the same or similar constructs examined.65 Investigation of the same construct is problematic in new areas of research with a limited number of published peer-reviewed studies.

Questionnaires are one of the most common self-report research methodologies used,64 with participants asked to report opinions, attitudes, behaviours or beliefs, background and demographic information, and knowledge of intentions and aspirations.66 As a data-collection tool, they are versatile, simple and relatively low cost for gathering a large amount of data of high enough quality to test hypotheses and make real-world policy suggestions.53 However, a low response rate is often identified,65 and can vary according to the method of administration.60 A low response rate has been identified as the major factor leading to response bias. Where possible, a researcher needs to demonstrate an absence of response bias, with the sample of respondents who return the questionnaire representative of the population.65

Few studies indicated whether their injury or supervision questionnaire had good reliability and validity.3920385657 None gave extensive consideration to the problems associated with using measures with unknown reliability or validity. This is an issue because unvalidated instruments may not measure what they claim to measure, and it is not possible to determine the consistency of responses obtained through them.54 Further empirical research is needed to determine the validity and reliability of key measures of supervision, child injury or risk taking before the continued use of such measures in this area.

Questionnaires and interviews both obtain self-report information but with a different method of questioning. Both rely on participants willingly and accurately providing answers to posed questions.53 Validity of interview data can be established through comparison with other methods, and recording bias can be overcome by implementing a mechanical recording method such as audio or video recording.53 There is generally a higher response rate for interviews than written questionnaires.64 However, interviews are generally more expensive and require more time to collect and transcribe data.65

Although not common, information on supervisory behaviour in hypothetical situations is generally collected through interviews35 or questionnaires,1027 where supervisors report their expected behaviours in various hypothetical settings. Morrongiello7 has urged caution in interpreting findings from such studies because the data may not represent actual parental supervision, especially if parents have limited real world experience around which to base their judgements of behaviour in hypothetical situations.37 Consequently, the findings are not transferable to understanding how individuals supervise under natural conditions, because of insufficient ecological validity. Further, whether parents/carers’ self-reported beliefs in hypothetical situations align with their actual supervisory behaviour has not been determined.7

Diary recording of supervisory behaviour and children’s injuries combined with interviews and questionnaires is one example of a mixed-method approach to provide more detailed information as part of a triangulation process.53 Diaries in which participants have reported on supervision have gathered a wide range of information, including reports of thoughts, feelings and behaviours, and accounts of physical and social activity. The time period over which the diary is maintained has ranged from hours to a number of years,53 with entries varying between continuous diary recordings of supervision363955 and periodic recordings each time a child experienced an injury.38 This method is very cost effective and useful for investigating supervision behaviour if data from the same person are required frequently or over a considerable time frame, and yields information that is sequentially ordered. Theoretically, a diary method does not suffer from recall bias unless it is completed well after the event. However, it is not possible to assess whether or not a participant completes the diary honestly and in a timely manner. For these reasons, diaries are best used in conjunction with other methods that can indicate any discrepancies in self-records.5367

An interesting side effect of using diaries is that they can act as an intervention in themselves,53 in that recording specific behaviours can alter the likelihood and degree to which the participant engages in that behaviour. This can be overcome by adding an observational component to the study, to validate the diary recordings. However, continuous diary recording is extremely time consuming and laborious, often resulting in small sample sizes and a large dropout rate.37

Self-reporting of supervisory behaviours (also referred to as participant event monitoring) often involves implementing a mixed-method approach to understand how supervision relates to child injury risk.33237^–39 Self-reporting of actual supervision practices potentially provides more meaningful data than participants reporting on hypothetical situations. However, there is no way of knowing if parents alter their “normal” supervisory behaviour as a result of a study, and any alteration from their typical behaviour would directly influence study findings.38

Case series methods (where reports such as coroners’ findings are investigated) have been used in reconstruction studies of injury outcome to determine mechanisms of fatal injury and patterns of trauma care.58 They have been commonly used in aquatic settings to examine the relationship between drowning or near-drowning incidents and their contributing factors,41^–4446^–48 because they allow data collection after the incident, resulting in understanding and confirmation of factors contributing to the event.

Coronial information, including police reports, post-mortem reports, photographs and coroner’s findings, provides highly detailed data about injury deaths.41 In developed countries, with a well-established coronial system, it is unlikely that cases are missed when coronial records are used.4147 In this review, studies examining coronial information were conducted in the USA,3233444852 Australia41^–4346 and UK,47 and therefore it is unlikely that injury deaths were overlooked. Although providing a wealth of information, a disadvantage of such case series methods is that monitoring of cases takes a long time, as a number of organisations and groups need to be contacted to obtain all relevant information regarding the incident. Furthermore, important details about the injury event and levels of supervision may not be thoroughly documented on death certificates or medical records. For example, in the USA, information about pool fencing was lacking in drowning incident reports.68This was not a concern in this review, as studies had to explain aspects of supervision and/or evaluate the effectiveness of supervision using a primary outcome measure of injury risk and/or incidence to be included.

A contrived hazard method may be advantageous in some areas of research, but because human behaviours are difficult to replicate,69 it is physically impossible to construct all environments in a laboratory setting.7 In early studies using this method, parents potentially supervised in a self-conscious or socially desirable manner, as they were aware that their behaviours were being monitored and understood that the hazards posed no threat of injury to children.11 A later study, incorporating a larger sample of supervisors who believed hazards were real and who were unaware their behaviours were being monitored, provides more ecologically valid results.12 Nonetheless, the contrived hazard method provides some information on parental supervisory behaviours and the interaction between parent and child behaviours without the influence of prior parental knowledge of data collection. A disadvantage of the contrived hazard method is that once participants are informed and familiar with the study, the method is no longer applicable, precluding longitudinal studies,7 which are required to determine if a relationship exists between supervision and child developmental stages.19 The level of parental supervision provided to young children is likely to vary depending on the situation or setting. There is a difficulty associated with constructing all environments and modifying hazards to appear real while posing no risk of injury to participants.12

Naturalistic observation is costly, time consuming and very labour intensive.9 However, it occurs in a natural setting and participants can be unaware that they are being observed,66 and therefore the observed behaviour is more likely to reflect the true supervisory behaviour.50 This method, adopted in a number of supervision studies conducted in supermarkets,85051 has allowed an understanding of the total context in which unintentional injuries occur. Observational studies of care giver supervision eliminate the possibility of bias and distortion associated with measuring individual’s behaviour, which is probable from self-report methods. Unfortunately, however, very few observational studies have used multivariate statistical methods to adjust for confounding variables.851 Natural observations are very accurate when the researcher specifically defines the parent behaviours to be observed, and develops a way of measuring and quantifying the behaviour,69 resulting in maximised ecological validity.7 Observing unobtrusively is not problematic in public venues, although there is the ethical dilemma for the observer of deciding when to intervene in a hazardous situation. This method is suited to active sporting and recreational situations, where unintentional injuries are of concern.50

IMPLICATIONS FOR PREVENTION

This systematic review has described the methods that have been used to measure supervision, as well as their advantages and disadvantages. As effective prevention of unintentional childhood injuries is multifaceted,5670 a thorough understanding of supervision, injury risk and the way that these interact can potentially enhance injury prevention understanding.

Several data-collection methods have contributed to the area to date; however, the existing evidence is generally of low to moderate methodological quality. This systematic review lends credence to the complex nature of measuring supervision and unintentional injuries in children, and acknowledges that further research using higher-quality methods is required to confirm and characterise the risk relationship between supervision and injury. Conducting RCTs is often impractical because of associated ethical issues, but more prospective studies could be undertaken. Nonetheless, there is scope to improve the quality of the research in this area by conducting prospective studies, and the use of mixed-method approaches would allow findings to be triangulated.

The results of this review point to an obvious need for studies to continue to explore the effects of parental supervision on child injury, with the quality of many supervision and child injury risk studies low or moderate. This review paves the way for implementation of higher-quality methods, particularly those using valid and reliable measures to improve studies of the relationship between supervision and child injury risk.