Article Text

Download PDFPDF
Implementation of a youth violence prevention programme in primary care
  1. Jessica S. Roche1,2,
  2. Meredith L. Philyaw-Kotov1,3,
  3. Eric Sigel4,
  4. Andria B. Eisman5,6,
  5. Golfo Tzilos Wernette7,
  6. Kenneth Resnicow5,
  7. Patrick M. Carter1,2,5,6,
  8. Rebecca M. Cunningham1,2,5,6,
  9. Maureen A. Walton1,3
  1. 1 Injury Prevention Center, University of Michigan, Ann Arbor, Michigan, USA
  2. 2 Department of Emergency Medicine, University of Michigan, Ann Arbor, Michigan, USA
  3. 3 Addiction Center, Department of Psychiatry, University of Michigan, Ann Arbor, Michigan, USA
  4. 4 Department of Pediatrics, University of Colorado Denver School of Medicine, Aurora, Colorado, USA
  5. 5 Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor, Michigan, USA
  6. 6 Youth Violence Prevention Center, University of Michigan School of Public Health, Ann Arbor, Michigan, USA
  7. 7 Family Medicine, University of Michigan, Ann Arbor, Michigan, USA
  1. Correspondence to Jessica S. Roche, Injury Prevention Center, University of Michigan, Ann Arbor, MI 48108, USA; jroche{at}


Background and objectives Youth violence is an alarming public health problem, yet, violence screening and interventions are not systematically offered in primary care (PC). This paper describes data from a pilot effectiveness-implementation trial of an efficacious youth violence prevention programme (SafERteens).

Methods The study was conducted in two PC clinics: a university-affiliated satellite clinic and a community health centre. In phase 1, we obtained stakeholder feedback to customise the SafERteens package and enrolled a comparison group of adolescents (age 14–18) seeking care in two clinics. In phase 2, clinical staff delivered the SafERteens-PC intervention with adolescents, which is a single, behavioural health therapy session delivered one-on-one from clinic providers to youth patients, followed by text message (TM) reminders. In phase 3, we assessed planned maintenance. All participants reported past-year violent behaviour at intake and completed a 3-month follow-up assessment.

Results Based on stakeholder interviews (n=13), we created a web-based SafERteens-PC programme package, including a three-item past-year violence screen, 30 min motivational interviewing-based brief intervention delivery tool, training videos and 2 months of TM boosters. We enrolled a comparison group (n=49) first, then an intervention group (n=61). Intervention delivery characteristics varied by clinic, including completion of intervention (75.9%; 62.5%), modality (100% delivered via telehealth; 60% via telehealth/40% in-person) and enrolment in TMs (81.8%; 55.0%); 91.8% completed the follow-up. Using an intention-to-treat approach, the intervention group showed significantly greater reductions in severe peer aggression (p<0.05), anxiety (p<0.05) and substance use consequences (p<0.05) relative to the comparison group. Participant and staff feedback were positive and identified challenges to long-term implementation, such as lack of availability of reimbursement for youth violence prevention.

Conclusions If these challenges could be addressed, routine provision of behavioural health services for violence prevention in PC could have high impact on health outcomes for adolescents.

  • primary care
  • violence
  • youth
  • interventions
  • implementation / translation

Data availability statement

No data are available. Please contact for more information.

Statistics from


Youth violence is a critical public health problem. Homicides are the leading cause of death among African-American adolescents and the third leading cause of death for all adolescents.1 Additionally, youth violence has serious impacts on physical, social and mental health, and is associated with increased risk for mental health issues, substance use, suicide and future violence.2 Nonetheless, youth violence can be prevented through evidence-based interventions,2 which primarily include educational programmes in school settings, after school mentoring or family interventions, hospital-based programmes and broader environmental-level initiatives.

In particular, primary care (PC) clinics are underused for prevention programmes addressing youth violence.3 4 Time constraints, training inadequacy, stigma and limited resources are barriers complicating prevention service delivery in PC;5 however, the American Academy of Pediatrics recommends addressing youth violence in PC, including screening and intervention.6 A potential solution to fill this gap is translation of promising programmes from other settings to the PC setting.

SafERteens is an evidence-based violence prevention intervention, which consists of a single session behavioural health therapy session delivered in the emergency department (ED). In a prior rigorous randomized control trail (RCT) among youth, compared with a control condition, SafERteens reduced peer violence for up to 1-year post intervention, as well as dating victimisation, alcohol-related consequences and depression at 3–6 months.7–10 There is a tremendous need within public health to accelerate the effective integration of evidenced-based interventions into clinical care.11–13 This manuscript describes the adaption and implementation of this programme into PC settings guided by the Enhanced Replication of Effective Programmes (Enhanced-REP) framework, which includes external facilitation14 and along with the CDC’s research-to-practice framework.15 REP includes an initial phase to identify needs of a setting and population, which for this study is the need for youth violence prevention programmes in PC; subsequent phases include: preimplementation (eg, tailored programme packaging, provider training), programme implementation and maintenance.14

Consequently, we systematically adapted SafERteens for PC guided by Enhanced-REP to specifically address these barriers and enhance intervention-context fit, with outcomes assessed using the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.13 16This study is the first to test the translation of an evidence-based youth violence intervention in PC.


We used an effectiveness-implementation hybrid design,17 conducted in phases at two clinics. In phase 1, we obtained feedback from PC stakeholders (eg, medical providers, social workers, administrators) to identify key barriers and tailor packaging, training, and facilitation, created a SafERteens-PC package, and enrolled a comparison sample. In phase 2, we trained clinic staff and enrolled the intervention sample; and in phase 3, we assessed clinic leadership’s interest in maintenance.18 Thus, we used a quasi-experimental design in which the comparison sample was obtained before the intervention sample. This approach was selected to prevent contamination (providers cannot be untrained) and to maximise collection of implementation data, as efficacy has previously been established.

Study sites

The study (August 2018–July 2019) conducted at PC clinics in Ypsilanti, Michigan (Corner Health Clinic (CHC) and Ypsilanti Health Center (YHC)), serving diverse and socioeconomically disadvantaged youth. CHC is a youth-specific community clinic (ages 12–25; ~1200 teens annually), while YHC is a university-affiliated family medicine clinic (~600 teens annually). The control sample was enrolled first (CHC: June–November 2018; YHC: Aug 2018–January 2019); the intervention sample was recruited second to prevent contamination (as clinical staff delivered the intervention) (CHC: December 2018–July 2019; YHC: January–July 2019).

Phase 1 procedures

To enroll the comparison group, research assistants (RAs) assented/consented youth (ages 14–18; waiver of parent consent for ages 14–17), with youth self-administering the three-item screen for violent behaviour on iPads in the waiting room. Adolescents were excluded if they did not understand English or if they were unable to provide informed assent/consent. Youth screening positive self-administered a 20 min baseline assessment ($30 remuneration) and completed a 3-month follow-up online (US$40 remuneration) using established protocols.19

The study team obtained feedback from clinic staff (eg, medical providers, social workers, administrators) on a previous SafERteens implementation package used in Emergency Department (ED) settings20 to adapt the package for use in PC. Surveys and qualitative interviews (US$100 remuneration) were conducted to understand barriers and benefits to implementation, site priorities and perceived patients’ needs. Interviews were audiotaped, transcribed and coded using NVivo to identify key themes. Based on this feedback, the SafERteens-PC programme package was created.

SafERteens programme

The ED version of the SafERteens implementation package ( includes a patient dashboard, three-item screener for electronic self-administration, training videos and mock patient interventions, computer screens to guide intervention delivery with help screens, optional text messages (TMs) (tailored based on data captures), and resource brochures.

The SafERteens intervention7–10 is a single behavioural health therapy session designed to be delivered one-on-one from clinic provider to youth patient. The intervention uses an adaptive motivational interviewing (MI)21 22 approach and is designed to be culturally relevant for socioeconomically disadvantaged youth (see In the ED version of the SafERteens implementation package,20 optional tailored TM boosters were added to prevent attenuation of effects often found during real-world implementation.23 The programme included automatic ‘push’ TMs (three per day), daily during the first month and every third day during the second month, to assess confidence in avoiding fighting, with tailored feedback, reflect tailored goals, strengths, reasons to avoid fighting and tools, and provide an affirmation and a reminder about ‘pulling’ messages by texting chill (tips for bad days) or plan (tips to avoid fighting). To preserve privacy, TMs did not ‘reveal’ that content is personalised.

Training clinic staff

Based on clinic feedback, 2–4 hours trainings were conducted with clinical staff (ie, clinicians with licensed master of social work (LMSW), limited licensed master of social work (LLMSW) degrees, behavioural health coordinators with master’s degrees) identified to deliver the intervention, focusing on MI skills development and the SafERteens-PC programme, including the online toolkit. After the training, staff completed a mock training session with the project coordinator ($50 remuneration), which was coded for fidelity prior to intervention delivery.

Phase 2 procedures

To obtain the intervention sample, RAs assented/consented participants and administered the violence screener before the clinic appointment. Eligible youth self-administered the baseline assessment and were scheduled by RAs to receive the intervention on a subsequent date (same day ‘on-demand’ delivery was difficult due to clinic time constraints). All interventions were delivered by clinical staff. This process was independent of, and in addition to, the clinic visit. The 3-month follow-up was self-administered online. External facilitation was conducted by the study coordinator.

Phase 3 procedures

During phase 3, feedback was obtained via qualitative interviews and surveys of clinic staff to assess barriers and benefits of the programme, and the research team met with clinics to present study findings and discuss continuation plans.



A three item screen9 24 assessed past year violent behaviours: (1) threw something at someone, pushed, grabbed or shoved, slapped someone; (2) kicked, bit or hit with a fist, hit or tried to hit with something, beat up, choked someone; and/or (3) threatened with a knife or gun, used a knife or fired a gun on someone. Any positive response indicated eligibility for the study.

RE-AIM-guided measures


RAs recorded the number of screens, baselines and interventions conducted.


Primary outcomes were: (1) severe violent behaviour towards non-partners via seven items that assessed severe physical aggression9 24; (2) non-partner victimisation behaviours (physical) via three items9; (3) violence consequences using a seven-item scale (α=0.78).9 Given the small sample size, behaviours were recoded into binary variables (any violence or none). Secondary outcomes were: (1) self-efficacy for non-violence using five items (α=0.85)8; (2) behavioural intention to avoid fights in the next 3 months via a 10-item ruler, coded as yes (10) to no (1–9) given the large number of participants indicating a ‘10’9; (3) participant satisfaction included four items (ie, able to be open and honest with my counsellor, felt my counsellor treated me with respect and understood me, helpfulness in talking with a counsellor about fighting, likelihood of recommending the session). Additional study variables included: (1) demographics25; (2) substance use: alcohol, illicit and prescription drug misuse index with frequency items summed (Alcohol Use Disorders Identification Test (AUDIT)-C, Alcohol, Smoking and Substance Involvement Screening Tool (ASSIST)-Lite)26 27; (3) substance use consequences using the Problem Oriented Screening Instrument for Teenagers (POSIT) (≥2 positive screen); (4) depression via the Patient Health Questionnaire (PHQ)-2 (≥3 positive screen)28; and (5) anxiety via the General Anxiety Disorder (GAD)-2 (≥2 positive screen).29


RAs recorded the number of staff trained, hours of training and use of toolkit.


Intervention fidelity was assessed during phase 1 and at the end of phase 2 by coding an audiotaped standardised patient session from clinic staff for MI competence (eg, mean global ratings from the MITI-4).30

Maintenance was assessed during phase 3 by asking clinical staff to indicate future plans.

Data analysis

Our analytical approach was informed by pilot studies and Consolidated Standards of Reporting Trials extension guidelines for reporting on pilot feasibility studies31–33 to inform subsequent larger trials. Translation is described by reach (% patients screened and receiving interventions; refusal rates), adoption (ie, number of clinic staff trained, hours of participation in training, use of website), implementation (ie, quality of intervention delivery) and maintenance (ie, clinic plans). Regarding effectiveness, given the skewness of the data and the small sample size we used non-parametric two sample paired tests (Wilcoxon and McNemar) to examine changes from baseline to 3 months as a function of receiving the intervention (vs comparison group) using an intention-to-treat approach across both sites combined.


Phase 1


Clinic staff (YHC=8; CHC=5) completed a 10 min survey and a 1.5-hour semistructured audiorecorded interview (46% medical providers/staff, 23% social workers, 31% administrators). Interview transcripts were analysed for a priori themes using NVivo V.11. Specifically, the rapid assessment framework was used to examine the organisational culture and potential facilitators/barriers to implementation of the SafERteens programme. Survey responses indicated that respondents perceived violence prevention programmes to be an important and needed addition to clinical services (mean=8.7 on scale from 1 (not at all) to 10 (very)). Qualitative themes identified included understanding the complexity of youth violence perspectives, enthusiasm to promote health and wellness to improve the community, open to clinical time and resources need to address various patient issues, and acknowledgement of the barriers to health that patients face. Next, themes identified regarding barriers to implementation included logistical and administrative challenges to behavioural health screening (eg, screening via paper/pencil or electronic medical record (EMR) vs stand-alone electronic assessment) and competing demands for programme delivery (ie, time constraints). Also, given patient diversity in terms of gender, clinics recommended updating programme language (eg, remove his/her).

Toolkit Adaptation

Due to feedback from the clinical sites, changes were made to the SafERteens implementation package to adapt it for use in PC. These changes included: (1) removing the requirement for electronic screening in the dashboard flow, as clinics preferred the flexibility to have a paper/pencil option and/or allow for future integration into electronic health record (EHR) systems; (2) refining the intervention guide to include gender neutral language, improve clarity, and reduce length (four scenarios instead of five); (3) creating an option for behavioural health staff to deliver the intervention in-person or via telehealth, with the addition of a warm hand-off procedure for providers (ie, warm hand-off card, provider training video); (4) moving the opt-in screen for TMs to the beginning of the programme to ensure receipt among youth regardless of session completion and (6) reducing the burden of TMs from three to two per day.

Procedural adaptation

Based on feedback, clinic staff were not viewed as appropriate to assist with screening tasks because of the hybrid design of the programme, which included research data collection as opposed to solely implementation; thus, research staff conducted the screening process. Clinic administration did not allow behavioural health staff to deliver the intervention during regular work hours due to inability to obtain reimbursement for services rendered. To adapt, the research project hired clinical staff at YHC to conduct interventions on a day off (ie, Fridays/Saturdays). At CHC, the project hired a clinical staff person to conduct interventions during after school and Saturday hours. Thus, the warm hand-off procedures were not implemented, and research staff scheduled intervention times with participants after screening.

Enrolling the Comparison group

A total of 145 patients were approached, with 84.8% (n=123) consenting to participate in the screen and 51.2% (n=62) screening eligible and n=2 excluded due to non-English speaking. Refusal rates were 16.6% (n=24) at screening, and 16.1% (n=10) at baseline (figure 1), with no significant gender differences in refusals. Follow-up completion exceeded 98.0%. See table 1 for the sample description.

Figure 1

CHC, Corner Health Clinic; RA, research assistant.

Table 1

Baseline sample characteristics

Phase 2

External facilitation

External facilitation consisted of the PI, coinvestigator, and the project manager meeting with clinic leadership at the beginning of the study to understand potential barriers. After enrolment of the comparison sample, the investigators and the project manager provided tailored training to deliver the intervention; and during the intervention enrolment period, the project manager conducted weekly phone calls/emails to engage in one-on-one problem solving of specific barriers to programme delivery with sites.


Between the 2 clinic sites, 177 youth were approached, 125 (79.6%) were screened, 55.2% screened positive (n=69), 61 (88%) were enrolled and n=2 excluded due to non-English speaking (figure 1). Males (35.7%) were significantly more likely to refuse screening than females (14.4%) and other genders (16.7%) (χ2=8.4, p<0.05). At baseline, the comparison group reported significantly more peer victimisation than the intervention group (table 1).

We increased the intervention sample enrolment to maximise feedback data. Overall, 42 youth (68.9% of those eligible) received the intervention (YHC 75.9%; CHC 62.5%) from clinic staff (one staff member delivered the intervention at each site). Among the 19 participants who did not receive the intervention, 1 withdrew from the study and 18 missed the intervention appointment and could not be rescheduled. Intervention delivery method varied by clinic (YHC 100% telehealth (via phone) and CHC 60% telehealth/40% in-person). Among those receiving the intervention, 69.0% agreed to receive the TMs (YHC 81.8%; CHC 55.0%); 75.8% for the full 2 months (24.1% discontinued). The overall follow-up rate was 83.6%.


Overall, 65.8% felt it was somewhat or very helpful to talk to the counsellor about fighting, 72.2% rated their likelihood of recommending the session as ≥7 (out of 10), 86.8% felt somewhat/very much able to be open and honest with the counsellor, and 89.5% felt somewhat/very much that the counsellor treated them with respect. Qualitatively, 83.8% mentioned liking the empathetic counsellor [‘Being able to talk to someone without being judged’ and ‘It was nice to know that people genuinely cared.’] and the helpfulness of the content [‘I thought about my mental health and security differently after that’ and ‘Explaining different types of ways to cope and just going and talking to them was helpful.’]. The tailored TMs were also appreciated [‘The fact that the texts were directed towards me and only me and my goals. It made me feel heard’].

The intervention group showed significantly greater reductions in severe non-partner aggression (p<0.05), anxiety (p<0.05) and substance use consequences (p<0.05) than the comparison group (table 2).

Table 2

Phase 2 effectiveness outcomes: changes from baseline to 3 months


Two 4-hour training sessions were completed at each site. At YHC, two staff were trained, and at CHC five staff were trained. Two of the trainees went on to deliver the intervention at CHC, while one trainee delivered the intervention at YHC. All interventions were delivered using the web package.


Intervention fidelity scores indicated excellent skills among clinic staff trained at both sites during phase 1 and 2. Specifically, standardised patient sessions coded from each clinic indicated high MI competence (eg, MITI-4 mean of global ratings=4.03). Based on data from the website tool, the SafERteens intervention averaged 24.9 min (SD=13.1).


Staff (seven YHC staff and eight CHC) mean ratings of importance of continuing to deliver SafERteens-PC was 7.3 at CHC and 8.6 at YHC, and usefulness of the programme as a clinical tool was 7.0 at CHC and 8.4 YHC (all out of a 10-point scale). The investigators/project coordinator met with clinic leadership to present study findings and discuss plans. Both clinics indicated high perceived need for screening/interventions for dating violence and were surprised at rates of non-partner violence. With structural support for the website, clinics were interested in continuing the programme, however, lack of reimbursement for youth violence prevented programme continuation.


This pilot translation project provides novel data to inform future effective translations of evidenced-based behavioural interventions to address a critical unmet needs for adolescents in PC. Using an Enhanced-REP model,34 we adapted the SafERteens intervention packaging, and provided training and technical support through an engaged, participatory process in collaboration with clinic staff and leadership in order to advance intervention-context fit and the likelihood of success in achieving implementation and behavioural outcomes.

Although video sessions were available, youth preferred delivery via the telephone, likely due to comfort, privacy or logistics (eg, limited Wi-Fi). Recent changes in remote therapy delivery due to COVID-19 could increase future video delivery, although flexibility is recommended given site differences. Provision of telehealth appointments during afterschool and evening hours, and on Saturdays, was essential as session completion was over 80% at the site with this availability.

The SafERteens intervention and TMs were well received by youth. Further, consistent with efficacy trial data,8–10 effectiveness data from this small implementation study were promising. Caution is warranted in generalising data from this pilot study, with replication required. Findings for depression and anxiety may suggest a potential mechanism for the efficacy of our violence intervention, which is consistent with prior data.7 35 Our results are consistent with researchers who suggest that systematically adapting an intervention to meet the context can help retain its effectiveness when translated from highly controlled studies into community settings.36–38

Financial barriers due to lack of public or private payer reimbursement for violence prevention is a key barrier to programme sustainability for this single session behavioural health intervention. Relatedly, although delivery of behavioural health focused TMs is an inexpensive solution to extend intervention dose, continuing TM boosters was a barrier for our clinics, as reimbursement mechanisms for therapeutic TMs are generally lacking despite the minimal costs (eg, <US$10 per participant).

Findings require replication with larger samples in multiple PC clinics given the limited scope of this study at two clinics, and the small sample size for the effectiveness data. Furthermore, the quasi-experimental design cannot rule out confounds; however, given prior efficacy data combined with lack of data published regarding implementation of evidenced-based programmes, these findings make an important contribution to the literature. Future, fully powered implementation trials may benefit from using a stepped-wedge, cluster RCT design across multiple PC clinics (due to the inability to untrain clinic staff once trained). Future studies could also test adaptive implementation strategies, in which sites could be randomised to a variety of implementation strategies (eg, external facilitation) at the provider or clinic level.39


Despite these limitations, findings from this study provide valuable lessons for the implementation of violence prevention interventions in PC. The customised SafERteens-PC programme showed preliminary evidence of potential effectiveness when delivered by clinic staff to adolescents involved with violence, which is consistent with the original efficacy trial,9 likely reflecting careful implementation strategies and flexible delivery based on patient’s needs, including telehealth delivery. Given the increased reach and preference among many youth for telehealth delivery, future multisite implementation trials should consider testing a centralised hub of interventionists for remote delivery, which could alleviate the burden of clinics and enhancing scalability. Future violence implementation research should focus on healthcare reimbursement to sustain delivery of violence prevention in PC and positively impact adolescent health. Finally, to maximise impact on youth violence prevention, the SafERteens programme should be viewed as one piece of the large puzzle of community violence prevention efforts, including those focusing across the spectrum of violence severity and social ecology, including universal prevention programmes in schools (eg, Task Force for Community Preventive Services, SafeDates, YES programme), family/mentoring programmes (eg, Big Brothers Big Sisters of America, Fathers and Sons) to indicated programmes (eg, Cure Violence), as well as programmes focusing on structural/environmental change (eg, Clean and Green, Crime Prevention Through Environmental Design).2 Similar to prior work testing multiple interventions across a community,40 future research should consider simultaneous implementation of multiple programmes across settings to enhance impact.

What is already known on the subject

  • Youth violence is an alarming public health problem, however, violence screening and interventions are not systematically offered in primary care settings.

  • There is a tremendous need within public health to accelerate the transfer of evidenced-based interventions into clinical care.

What this study adds

  • This is the first known study to effectively implement an evidence-based youth violence intervention programme into clinical care at primary care sites

  • This study bridges the gap between research and practice.

Data availability statement

No data are available. Please contact for more information.

Ethics statements

Patient consent for publication

Ethics approval

The study was approved by our IRB and a CDC Certificate of Confidentiality was obtained.


The authors wish to acknowledge project staff, including Linping Duan, and Lisa Orrison for their assistance in data and manuscript preparation. Finally, special thanks are owed to the patients and medical staff at the primary care sites where the study was conducted for their support of this project.



  • Twitter @cartpatr

  • Contributors JR is the guarantor of this work. MW conceptualised and designed the study, conducted the initial analyses and reviewed and revised the manuscript. PMC and RC conceptualised and designed the study, and critically reviewed and critically revised the manuscript for intellectual content. MLP-K collected the data and critically reviewed and critically revised the manuscript for intellectual content. JSR conducted the initial analyses, drafted the initial manuscript, and reviewed and revised the manuscript. ES, ABE, GTW and KR contributed substantially to the study conception and design, and critically reviewed and critically revised the manuscript for intellectual content. All authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

  • Funding This work was funded by the University of Michigan Injury Prevention Centre by the Centres for Disease Control and Prevention (CDC) Award Number R49-CE-002099 and R49-CE-003085.

  • Disclaimer The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the funding agencies. No honoraria, grants or other form of payment were received for producing this manuscript.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.