Article Text
PDF
Abstract
Objective Inexperienced, less-skilled driving characterises many newly licensed drivers and contributes to high crash rates. A randomised trial of TeenDrivingPlan (TDP), a new learner driver phase internet-based intervention, demonstrated effectiveness in improving safety relevant, on-road driving behaviour, primarily through greater driving practice diversity. To inform future learner driver interventions, this analysis examined TDP use and its association with practice diversity.
Design Posthoc analysis of data from teen/parent dyads (n=107), enrolled early in learner phase and assigned to treatment arm in randomised trial.
Methods Inserted software beacons captured TDP use data. Electronic surveys completed by parents and teens assessed diversity of practice driving and TDP usability ratings at 24 weeks (end of study period).
Results Most families (84%) used TDP early in the learner period; however, the number of TDP sessions in the first week was three times higher among dyads who achieved greater practice diversity than those with less. By week five many families still engaged with TDP, but differences in TDP use could not be detected between families with high versus low practice diversity. Usability was not a major issue for this sample based on largely positive user ratings.
Conclusions An engaging internet-based intervention, such as TDP, can support families in achieving high practice diversity. Future learner driver interventions should provide important information early in the learner period when engagement is greatest, encourage continued learning as part of logging practice drives, and incorporate monitoring software for further personalisation to meet family needs.
Trial registration NCT01498575.
- Interventions
Statistics from Altmetric.com
Introduction
Road traffic injury is increasingly recognised as a major health concern, particularly for adolescents and young adults.1 Traffic crashes occur disproportionately among newly licensed adolescents2 largely due to their inexperience and developing skill.3 With the goal of increasing experience and skill prior to licensure (and concomitantly reducing crashes), most US states and other countries mandate adult-supervised practice driving (typically a period of 6 months for learner drivers).4 ,5 However, despite good intentions, little evidence to date confirms that adult-supervised practice driving, as measured by number of hours of practice or presence of state-level practice requirements, has had a consistent positive effect on safe driving skills or injury or crash outcomes among newly licensed young drivers.6–10 One challenge is that parents often serve as the primary supervisors, yet many lack knowledge about practice diversity (ie, the wide range of environments (eg, commercial districts, country roads, and highways) and conditions (night-time, suboptimal weather) in which a meaningful amount of practice hours should occur), which is needed to develop safe, skilled drivers, and there is a paucity of proven-effective resources to support parents.9 ,11
TeenDrivingPlan (TDP), a new psychoeducational internet-based intervention, was designed to support the role of parents in facilitating high-quality practice. It includes three principal components: (1) learning, via more than 50 short educational videos, organised as follows: (a) introduction, (b) ‘the basics’ and (c) the six practice environments (parking lot, residential roads, intermediate roads, commercial districts, country roads, and highways) as well as (d) added challenges; (2) planning, through an interactive tool designed to assist parents in identifying specific skills and driving environments to cover in a given practice drive and (3) logging and rating of practice drives, through an interactive tool designed to track accumulated practice hours across a wide range of driving environments, and to prompt parents and teens to identify skills in need of further practice.12 A 24-week randomised trial of 217 learner teen and parent supervisor dyads demonstrated the effectiveness of TDP versus usual practice in improving safety-relevant driving behaviour as measured by performance on a rigorous on-road driving assessment.12 Further analysis demonstrated that the effect of TDP on teens’ driving performance was mediated by practice diversity.13
To optimise TDP's effectiveness for broader dissemination and to inform the development of other parent-focused learner driver interventions, we aimed to examine how the use of TDP varied over time, and the association between use and the behavioural outcome demonstrated to be most related to improved driver performance; that is, practice diversity. This posthoc analysis of data from participants in the intervention arm of the primary randomised trial was conducted in order to: (1) examine use of TDP, as measured by the number of logins to the application, over the 24-week study period; (2) examine usage of its three principal components: learning, planning, and logging, over time; (3) determine the association between TDP use over time and practice driving diversity and (4) describe participant feedback on the usability of TDP and how this varied by practice driving diversity.
Methods
Description of TDP
As described in detail previously, TDP is an internet-based intervention that provides parent supervisors of teens with learner permits with specific guidance for creating a positive learning environment and facilitating supervised practice driving across a range of environments: (1) empty parking lots; (2) residential neighbourhoods; (3) intermediate roads; (4) rural roads; (5) commercial roads and (6) highways.12 ,13 The content and functionality of TDP were informed by extensive literature review, a strong theoretical background, formative research with professionals and families, and usability testing.
Study sample and procedures
Study enrolment has been previously described in detail.12 Of the 512 teen/parent dyads recruited for the trial, the first 217 were assigned to take an on-road driving assessment. Of these, 128 (59%) were randomly assigned to the intervention group who received access to TDP. After excluding dyads who withdrew from the study (n=19) or did not provide practice information (n=2), the final study sample for these analyses included 107 dyads (83.5% of eligible sample).
Upon enrolment, parents and teens were each provided log-in credentials and completed a short, self-report survey that collected relevant sociodemographic data. Welcome calls were made 1 week after enrolment to ensure that participants received their enrolment packet, which included a TDP User's manual. We divided the 24-week study period into 83-week sequential time periods based on the dyad's date of enrolment. For a given 3-week period, the family was contacted by phone or email if the dyad did not log in at least twice (median number of calls per family was one). Technical support was provided if needed (eg, parent forgot log-in) and/or TDP use was encouraged. There were no technical problems with the website at the start of or during the study.
Finally, parents and teens separately completed a survey at 24 (±3) weeks after enrolment that ascertained TDP usability, and the amount of practice driving completed in each of the six driving environments since the teens obtained a learner's permit. For completion of all study activities, teens could receive up to US$100 and parents up to US$80.
Study measurements
Practice in six driving environments was assessed using a 6-point frequency scale: ‘None’; ‘Less than 1 H’; ‘1–2 H’; ‘3–5 H’; ‘6–10 H’; and ‘More than 10 H.’ To capture practice diversity, we summed the number of environments in which dyads reported practicing for at least 1–2 h. Responses were averaged for parents and teens, but for participants with missing data from one reporter, we used the single reporter score (parent only n=10; teen only n=4). The observed range for practice diversity was 1–6 with dyads reporting practicing in most environments: median and IQR=6 (5, 6). We then defined a dichotomous variable, Practice Diversity, with dyads placed in the ‘Higher Practice Diversity’ group if they practiced in five or six environments for at least 1–2 h, and were placed in the ‘Lower Practice Diversity’ group otherwise.
TDP usability items were adapted from the System Usability Scale14 based on a 5-point scale ranging from Strongly Disagree (1) to Strongly Agree (5). Items included: (1) I liked using this system (asked only of parents); (2) I thought the system was easy to use; (3) I would imagine that most people would learn to use this system quickly; (4) I felt very confident using the system; and (5) I needed to learn a lot of things before I could get going with this system. Responses were averaged for parents and teens or taken from one reporter if the other had missing data for that item.
TDP use metrics
Measures of general TDP use were summarised for each 3-week period based on parent and teen use. In a previous TDP process evaluation study, we learned that either the teen or the parent or both could be engaged in the use of TDP on their own or on behalf of the dyad.15 Thus, TDP use metrics will be reported on a dyad level. Two measures of engagement with the intervention were calculated: (1) the number of TDP sessions lasting at least 30 s during each 3-week period (TDP Use Frequency), and (2) the total length of sessions (in minutes) during each 3-week period (TDP Use Duration). Each session was coded as to whether it included learning (ie, at least one video was watched); planning (ie, the completion of at least one practice plan); or logging (ie, at least one practice drive was completely logged). Three component-specific use measures indicating the number of sessions that involved each component were created for each 3-week period. The median frequency of sessions containing each component was examined for each 3-week time period, overall and stratified by practice diversity.
To determine the 10 most popular videos, the proportion of dyads who watched each video at least once during the entire 24-week study period was calculated and ranked. Additionally, recognising the importance of diverse practice, two categories were created: ‘basic’ videos (20 total to address initial driving skills: 1 introduction, 7 ‘the basics’, 4 parking lot, and 8 residential driving) and ‘advanced’ videos (33 total to address additional skills needed for more complex environments: 9 intermediate roads, 7 commercial roads, 9 highways, 2 country roads, and 6 additional challenges). The total number of views of ‘basic’ videos versus ‘advanced’ videos was calculated.
Statistical analyses
Dyad sample characteristics were described using descriptive statistics, including medians and IQRs for continuous variables and proportions for categorical variables. To determine the association between weekly TDP Use Frequency and Practice Diversity, a negative binomial regression model was constructed with time entered as a continuous variable based on week; models included only the first 12 weeks of the study period. A visual inspection of the data indicated that there was no observed difference in TDP use between the high and low practice diversity groups after 12 weeks. Since TDP use within dyads may be correlated, generalised estimating equations were used with an independent correlation working model. Models with polynomial time trends up through quartic (4th degree) were considered; a quadratic time trend appeared to be sufficient, and was used in the analysis.
Results
Distributions of parent and teen sociodemographic characteristics at enrolment for the 107 dyads included in this analysis are provided in table 1. Of the 107 dyads, 86 (80.4%) practiced at least 1–2 h in five or six driving environments, and are considered the higher practice diversity group. Among this group, the environments for which families lacked at least 1–2 practice hours were rural roads (19% reported practicing less than this amount), parking lots (8%), and highways (7%). Among the lower practice diversity group (the remaining 21 dyads, 19.6%), the environments for which families lacked at least 1–2 practice hours were: country roads (76%), highway (81%), parking lot (43%), commercial districts (38%), residential neighbourhoods (14%), and intermediate roads (14%).
Characteristics of teens and parents‡ in dyads assigned to the intervention arm of the TDP randomised trial
Most dyads initially engaged with TDP: 78.5% (n=84) logged in at least once during the first 3 weeks (median number of sessions: 3; IQR: 1, 5) for a median TDP Use Duration over the 3 weeks of 34.7 min (IQR 16.0, 67.7). By the end of the 24-week study period, 32.7% (n=35) dyads used TDP (median number of sessions 0, IQR 0, 1) for a median TDP Use Duration over the final 3 weeks of 17.5 min (IQR 7.8, 40.2). The overall trend in TDP Use Frequency over the 24-week study period is provided in figure 1.
Proportion of dyads with at least one session, overall and with specific activities, by period.
Use of individual TDP components varied over time (figure 1). The most consistently used component was the logging and rating tool, used by approximately 30–60% of dyads throughout the study period. The learning component was also used frequently, but only early in the study. Overall, the proportion of dyads with a learning activity dropped off to <50% in period 2 (46%), continued to drop to 33–35% in periods 3 and 4, and remained stable at 8–16% with no clear downward trend for the remaining 4 periods. However, over time, dyads engaged less in learning activities, though 16% of dyads engaged in learning in the last 3-week period. Planning was the least used component, with fewer than 10% of dyads engaged in planning activities by week 10.
The 10 videos viewed by the highest proportion of dyads are shown in table 2. Each of these videos was viewed by at least 20% of the dyads; eight were on basic topics and two were on advanced topics. Although only 38% of the videos were on basic topics (20 of 53), these videos accounted for 66% (609) of the total 917 videos viewed by dyads.
Ten videos most commonly watched by dyads assigned to the intervention arm of the TDP randomised trial (n=107 dyads)
TDP use over the first 12 weeks of the study period varied between the dyads who achieved high diversity practice by the end of the study and those who did not (figure 2). The estimated relative rate of TDP use was three times higher (relative rate 3.08, 95% CI 1.67 to 5.69) during the first week among dyads who achieved high-practice diversity (mean 1.55 sessions) than those who did not (mean 0.50 sessions). At week three, the relative rate was 1.55 times higher (95% CI 0.99 to 2.43) among those who achieved high practice diversity. By week five, TDP use was similar between the two groups. A visual inspection of the data suggested greater use of learning activities by dyads with high practice diversity in the first two 3-week time periods, a delay in initiating logging activities among those with low practice diversity, and weak use of the planning component in both groups.
Relative rate (RR) of TDP use in high versus low-practice diversity dyads.
We found no evidence that perceptions about website usability differed by our two practice diversity groups on the dyad level, and that both groups rated the site positively. Few dyads reported negative impressions with most dyads (82.5%) neutral about or liked using the TDP intervention (median rating=3); most (88.8%) were neutral about the ease of using TDP or found TDP easy to use (median rating=4); and most (94.4%) were neutral or positive about their confidence in using TDP (median rating=4).
Discussion
In our prior analyses of TDP trial data, we found compelling support that exposure to TDP increased the quality and diversity of practice driving.12 ,13 Our current analyses extend these findings by demonstrating an association between the amount and type of early engagement with TDP and the diversity of driving practice achieved by the end of the 24-week study period. However, only use early in the study period differentiated the two diversity groups. Additionally, the nature of TDP use changed from early use of learning and logging functionalities to a predominance of practice logging by the end of the study period. Throughout the study periods, a minority of dyads used the planning functionality. Of note, we identified that usability was not a major issue for this sample in that families had neutral or positive ratings of TDP and the lack of reported serious technical issues in the use of the website.
While both groups engaged in early practice logging, the higher diversity group initiated logging comparatively earlier and maintained it longer than the lower diversity group. Future studies should examine the directionality of the association of TDP use and higher diversity practice (ie, does higher TDP use lead to high diversity practice or vice versa, or are they interrelated?). We do know from our prior analysis that assignment to the TDP group was effective in increasing practice diversity, thereby lending support to the first hypothesised model.12
Few dyads created plan templates using TDP, but this should not be confused with whether or not families planned. Our prior research indicates that compared with parents in the control group, parents in TDP dyads reported greater engagement as practice supervisors that included planning practice drives (eg, selecting specific practice activities, route planning).13 The relatively weak use of the planning component of TDP compared to the other components could be due to parents feeling able to do this behaviour offline without the support of a web-based tool as well as an underlying preference for opportunistic drives of convenience. Prior research has found that many families reported fitting practice drives in with their current schedule (eg, running errands) rather than intentionally planning practice drives.16 ,17
Although there was a suggestion of greater use of learning activities early in the study by the high diversity group, a minority of both groups used the learning activities after 12 weeks, and only two of the top 10 videos were advanced (related to highways). A minority of dyads viewed videos regarding skills needed for safe driving on intermediate roads, in commercial districts, on country roads or with added challenges (such as due to inclement weather or night). These results are consistent with previous research that showed a lack of awareness by families regarding the higher-order driver training needs of young drivers (eg, scanning and hazard detection).11 This lack of preparation for independent driving may explain, in part, the high prevalence of scanning and other higher-order errors in crashes among teens.3
As with any psychoeducational or behavioural intervention, implementation fidelity and participation of the target population are central to TDP's effectiveness and must be evaluated.18 However, unlike highly structured interventions (eg, e-learning applications, curricula with a prescribed module order, activities, or session length) that dictate a standard a priori minimum ‘dose’ for all participants, TDP users were free to determine how they engaged with the website over the supervised practice period (ie, what they do and when they do it). This flexibility was inherent to the programme, as we recognised that: (1) use of the system would largely be determined by families’ offline availability, needs, and interest in engaging in supervised practice, and (2) at this early stage in TDP development it was premature to institute a system architecture that was overly prescriptive, and no useful expectations were provided beyond at least two sessions per 3 weeks.
Ideally, teens are changing over the learner period, becoming more confident and skilled drivers. These changes will likely influence parents’ and teens’ need for TDP, and this should not necessarily be viewed as a negative outcome, but rather as a normative and appropriately diminishing need for support. TDP is a training tool for parent supervisors. As parents become more adept and confident with structuring practice drives, it is likely that their need for specific components of TDP, especially the learning and planning functions, will decrease. However, the videos in TDP included topics that spanned the range of simple to complex driving, and the logging and rating tool was designed for use over the duration of the learner period. The current analysis did not permit a detailed examination of how dyads used these components, or if users complemented TDP with other offline resources (eg, a paper log). Future studies should examine whether families would benefit from various combinations of extended, or more concentrated, use of TDP. These studies could employ more sophisticated analytical techniques common to digital health applications to ascertain how deviation (eg, attrition) from these prescribed formulations of TDP might affect perceptions about acceptability and outcomes.19–21
These analyses benefited from multiple reporters, detailed measurement of individual-level use of the intervention through automated, individual-level tracking of general and specific component use through a new software platform, and linkage of usage data to individual-level user data, outcomes data and usability ratings which provided detailed information on outcomes and also on how use of the intervention is related to the outcomes of interest. However, some limitations should be noted. Only one parent supervisor participated; future research should explore how multiple supervisors, or how changes in supervisors, might influence TDP use. Additionally, while few families refused to participate; the families in our study may differ from non-participants in unmeasured ways that could limit generalisability of the findings. Families living in certain geographic areas, with more limited access to computers, with lower education or income, or from racially or ethnically diverse backgrounds, may uise TDP differently from our study participants who largely lived in suburban residences with easy access to all of the driving environments and to a computer. Additionally, as this study grouped dyads by practice diversity as measured at the end of the study period, and that this group was self-selected rather than randomly assigned, the clear direction of effect between TDP use and practice diversity could not be determined. For ease of reporting/discussing the results, we binned time (for figure 1, by 3-week periods; for the model, by 1-week periods); therefore, care should be taken to recognise that use was not evenly distributed over any of these assigned time periods. Finally, TDP was delivered in one US state with a requirement of 65 h of practice driving during the learner phase, and it was found to influence the diversity of the practice rather than the absolute number of hours of practice. Additional interventions could focus on increasing the number of hours practiced, and studies could examine how TDP affects practice diversity in geographic regions with varying requirements in practice hours.
Invalid driver licenses in fatal crashes
A study using the US Fatality Analysis Reporting System found that 19% of motor vehicle fatalities involved drivers with invalid licenses, but it is not known why this association exists. Although it is most likely to reflect licenses that had been suspended or revoked because of violations, it could also be due to less ominous explanations, for example, not renewing a license on time or not obtaining one at all.
More needed to reduce social gradients in childhood injuries
The number of injuries among preschool children in the UK has decreased over the past two decades, but significant socioeconomic inequality gradients persist. If the rates in the poorest households were reduced to those in more affluent areas, many fractures, burns and poisonings would be prevented. A report in PLoS ONE used data from The Health Improvement Network, a research database of anonymous patient records from participating GP practices. These data enable the compilation of population-based incidence rates not based on self-reporting surveys or a single source, such as emergency department or hospital admissions. Source: Orton E, Kendrick D, West J, et al. Persistence of Health Inequalities in Childhood Injury in the UK; A Population-Based Cohort Study of Children under 5. PLoS ONE 9: e111631.
In summary, an engaging internet-based intervention, such as TDP, can support families in achieving high diversity practice with the aim of improving safety-relevant driving behaviours. Future versions of TDP and other learner driver interventions should monitor use and measure engagement to ensure that the interventions are being used as intended. Additionally, important information, including content on more advanced driving skills, should be provided early in the learner period when engagement is greatest, encouraging continued learning as part of logging practice drives, and incorporating monitoring software for further personalisation to meet family needs. Additionally, consideration should be given to providing access to the learning component of TDP prior to the learner phase of licensure, a recommendation given by participants in the process evaluation study.15 A mobile version of TDP might be preferred by families who rely on drives of convenience to practice driving. Additional research is necessary to identify barriers to behind-the-wheel practice diversity and the use of an internet-based intervention, like TDP, in supporting that practice.
What is already known on this subject
-
Inexperienced, less-skilled driving characterises many newly licensed drivers and contributes to high crash rates, yet effective learner interventions are lacking.
-
A randomised trial of TeenDrivingPlan (TDP), a new learner driver phase internet-based intervention, demonstrated effectiveness in improving safety-relevant on-road driving behaviour, primarily through greater driving practice diversity.
What this study adds
-
An engaging internet-based intervention, such as TDP, can support families in achieving high practice diversity with the aim of improving safety-relevant driving behaviours.
-
This study demonstrated the value of a new methodology for capturing internet-based intervention-use data, and linking it to individual-level user and outcomes data.
-
The results provided guidance for future internet-based learner driver interventions: important information should be provided early in the learner period when engagement is greatest; continued learning should be encouraged as part of logging practice drives, and monitoring software should be incorporated for further personalisation to meet family needs.
Acknowledgments
We would like to acknowledge Yi-Ching Lee, Lela Jacobsohn, Ken Ginsburg, Mary and Ian Jeffery, and Gerald Murphy for their contributions to the development of the TeenDrivingPlan, and Megan Fisher Thiel for her administrative support and review of the manuscript. Additionally, we want to thank the network of primary care clinicians and their patients and families for contributing to this project and clinical research facilitated through the Paediatric Research Consortium (PeRC) at The Children’s Hospital of Philadelphia. Lastly, we thank the many people that contributed to the design of the TeenDrivingPlan and provided administrative support on this study as well as on earlier studies that informed the development of the intervention.
References
Footnotes
-
Contributors All authors meet the conditions for contributorship.
-
Funding This research was funded by State Farm. The funder provided financial and in-kind support for the development of TDP. The creative and graphical design treatments were developed by State Farm and implemented by CDM SMITH who developed the software. State Farm reviewed this manuscript but did not have a role in the design and conduct of the study, the collection, management, analysis, and interpretation of the data, or in manuscript preparation. State Farm is a registered trademark of State Farm Mutual Automobile Insurance Company. The findings and conclusions are those of the authors and do not necessarily represent the views of State Farm.
-
Competing interests AEC has received an additional grant from State Farm Mutual Automobile Insurance Company (State Farm), the company that funded this study. MRE and DRD have received consulting fees from State Farm for a different project. At the time of the writing of this paper, the funder has posted an internet-based intervention, a modified version of TDP, on its corporate website.
-
Ethics approval The Children's Hospital of Philadelphia IRB.
-
Provenance and peer review Not commissioned; externally peer reviewed.