You are here

Article Text

Article menu
Original research
Using synthetic control methodology to estimate effects of a Cure Violence intervention in Baltimore, Maryland
Free
  1. Shani A Buggs1,
  2. Daniel W Webster2,
  3. Cassandra K Crifasi2
  1. 1 Violence Prevention Research Program, Department of Emergency Medicine, University of California Davis, Sacramento, California, USA
  2. 2 Center for Gun Violence Prevention and Policy, Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland, USA
  1. Correspondence to Dr Shani A Buggs, Violence Prevention Research Program, Department of Emergency Medicine, University of California Davis, Sacramento, CA 95817, USA; sabuggs{at}ucdavis.edu

Abstract

Objective To estimate the long-term impact of Safe Streets Baltimore, which is based on the Cure Violence outreach and violence interruption model, on firearm violence.

Methods We used synthetic control methods to estimate programme effects on homicides and incidents of non-fatal penetrating firearm injury (non-fatal shootings) in neighbourhoods that had Safe Streets’ sites and model-generated counterfactuals. Synthetic control analyses were conducted for each firearm violence outcome in each of the seven areas where Safe Streets was implemented. The study also investigated variation in programme impact over time by generating effect estimates of varying durations for the longest-running programme sites.

Results Synthetic control models reduced prediction error relative to regression analyses. Estimates of Safe Streets’ effects on firearm violence varied across intervention sites: some positive, some negative and no effect. Beneficial programme effects on firearm violence reported in prior research were found to have attenuated over time.

Conclusions For highly targeted interventions, synthetic control methods may provide more valid estimates of programme impact than panel regression with data from all city neighbourhoods. This research offers new understanding about the effectiveness of the Cure Violence intervention over extended periods of time in seven neighbourhoods. Combined with existing Cure Violence evaluation literature, it also raises questions about contextual and implementation factors that might influence programme outcomes.

  • firearm
  • public health
  • safe community
  • case-control study
  • programme evaluation

Data availability statement

Census block group-level data are available in a public, open access repository. Data on homicides, non-fatal shootings and arrests were obtained through both the Baltimore City Open Data portal (https://data.baltimorecity.gov/) and the Baltimore Police Department. Police Department data requests must be submitted to the agency.

https://doi.org/10.1136/injuryprev-2020-044056

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Baltimore, Maryland has long been plagued by high rates of homicides and non-fatal penetrating firearm injuries (non-fatal shootings). The homicide rate in Baltimore, Maryland, has been one of the highest among the US cities with over 500 000 people, with the vast majority of these killings involving firearms.1 From 2003 to 2017, nearly 4000 Baltimore residents lost their lives to homicide and over 8000 were victims of non-fatal shootings.2 City leaders in Baltimore, as in many urban locales, have sought to implement community programmes to prevent violence among individuals at high risk for firearm violence.

    Cure Violence is an intervention that employs community members to interrupt violence through conflict mediation, model and promote non-violence, and link individuals at high risk of violence to needed services.3–5 The Cure Violence model has been implemented and evaluated in multiple cities around the US.5–8 However, studies examining the programme’s impact on homicides and non-fatal shootings, while generally positive, have produced varied programme effect estimates across sites and cities, with some studies reporting no beneficial effects.4–10

    The Cure Violence model was first implemented in Baltimore, Maryland, in 2007. Named ‘Safe Streets’, the programme was started in police posts whose total homicides and non-fatal shootings were in the top 25th percentile (see online supplemental figure 1 for a map with locations and operational periods for all programme sites). The impact of Safe Streets was first assessed in an evaluation of the original four programme sites in Baltimore using 2007–2010 operational data,11 and then again in 2018 using data on the then-seven Safe Streets sites operating between 2007 and 2017.12 Difference-in-differences regression strategies in both evaluations yielded wide-ranging effect sizes across sites in both homicides and non-fatal shootings. The 2018 evaluation found no aggregate protective effects of Safe Streets on homicides or non-fatal shootings. Models that estimated site-specific effects on homicides found that only one site (Cherry Hill) experienced a sizeable and statistically significant reduction, while the direction and effect sizes of homicide estimates for the other six Safe Streets sites ranged from null to a statistically significant increase in homicides in one site. Site-specific effects on non-fatal shootings showed that four of the seven sites had reductions in non-fatal shootings, while three experienced increases; none of the estimates were statistically significant.

    The previous Safe Streets evaluations, like other Cure Violence evaluations, were limited in their ability to identify suitable comparison units for estimating programme effects because they used either traditional regression analyses or interrupted time-series approaches that do not use statistical methods to determine the most appropriate non-intervention controls for estimating the counterfactual for intervention areas. The objectives of the current study were to improve on the 2012 and 2018 evaluations of Safe Streets and extend the literature on the effectiveness of Cure Violence replications by using the synthetic control method (SCM), which has noted advantages over regression approaches for estimating intervention effects of non-randomised interventions.13 We hypothesised that by using the SCM method, we would generate more accurate estimates of intervention effects, allowing us to better assess the effectiveness of Safe Streets.

    Methods

    Design and setting

    This study used a comparative case study design to estimate the impact of the Safe Streets programme on homicides and non-fatal shootings for each programme site operating in Baltimore, Maryland, during the years 2007–2017, using data from 2003 to 2017 to calculate preintervention and postintervention trends.

    Data and measures

    Incident-level data for homicides and non-fatal shootings occurring between 1 January 2003 and 31 December 2017 were obtained directly from the Baltimore Police Department2 or through Baltimore’s Open Data portal.14 Over 80% of all homicides in Baltimore are firearm-related, and Safe Streets is designed to address all violence regardless of weapon type; therefore, we included all homicides in our analysis. Data on the police posts where Safe Streets was implemented, as well as the sites’ dates of operation, were provided by the Baltimore City Health Department.

    Our covariates, used to improve preintervention model fit, were selected based on prior firearm violence studies with similar aims and methodological approaches.15–17 Census block group-level data on the number of households, per cent of households below the poverty level, total population, per cent total male population, per cent males aged 15–34, per cent black, per cent white, median household income and per cent vacant housing were obtained from the 2005–2009 American Community Survey (ACS) 5-year estimates and averaged by police post.18 Arrests for drug possession, drug trafficking and weapon possession occurring between 2003 and 2017 were obtained from the Baltimore Police Department. All data were geocoded and aggregated to police posts by month and year.

    The dependent variables were counts of homicides and non-fatal shootings within police posts by month and year. Given the volatility of monthly homicide and non-fatal shooting counts, we used 5-month moving averages to smooth the data.19 20

    Analytic strategy

    Finding appropriate comparison units can be difficult because areas where interventions such as Cure Violence are implemented often have substantially more violence than non-intervention areas. While strategies such as propensity score matching attempt to minimise the risk for omitted variable bias when treated and untreated units are quite different,21 identifying controls that most closely mirror the treated units, based on observable characteristics, should provide more valid estimates of the counterfactual than statistical approaches that generate estimates using non-treated areas that are much different from intervention sites.22

    SCM is a suitable strategy for detecting the effects of an intervention in comparative case studies, when the treatment is applied in a non-experimental, non-randomised fashion.23–25 The SCM estimates outcome variable values to generate a counterfactual, or ‘synthetic control’, based on a vector of weights for non-intervention comparison units that minimises the root mean square prediction error (RMSPE) between the synthetic covariates and outcomes and those observed in the Safe Streets police posts prior to the intervention.21–25 The SCM captures trends in both observed and unobserved time-varying confounding factors throughout the study period.11 The method is not dependent on assumptions of regression analyses, such as parallel trends or the allocation of constant weights across all control units, that might limit prediction error minimisation.26 Whereas the root mean square error (RMSE) is a common measure of prediction error applied in studies using regression to help evaluate model fit,27 the RMSPE is similarly used in the SCM to assess model goodness of fit.23–25 Additionally, whereas regression analyses rely on a single model to estimate intervention effects across all sites, the SCM produces site-specific models designed to generate the best estimates for each intervention site.

    The donor pool of controls for the SCM analyses was restricted to the 136 Baltimore City police posts without a Safe Streets programme during the study period. Twelve-month lagged averages for the dependent variables and arrest data for each preintervention year from 2003 forward were combined with the ACS 2005–2009 5-year block group estimates to generate preintervention trends for each model.

    Although we were unable to include details on changes in Safe Streets leadership or staff or differences in intervention oversight by site as model covariates, those and other unmeasured neighbourhood-level factors may modify the effects of a local-level intervention such as Safe Streets in a way that could affect the intervention’s impact over time. Thus, 3-year, 5-year and 7-year programme effect estimates were generated for the two longest-running Safe Streets sites: McElderry Park and Cherry Hill.

    During the study period, Baltimore experienced considerable civil unrest in April 2015 following the death of Freddie Gray, Jr, in police custody, and homicides and non-fatal shootings subsequently experienced sustained increases across the city. To assess how programme effect estimates were influenced by the unrest, we conducted a sensitivity analysis with the four Safe Streets sites in operation before and after the unrest by truncating the postintervention period to end in March 2015 and comparing those results with the primary model with data through December 2017.

    Traditional tests of statistical significance are not computed via the SCM. To assess the likelihood that estimated effects were due to the intervention, ‘in-space placebo tests’, which treat each police post in the donor pool as if it received the intervention, were conducted to generate comparisons between the estimated per cent change associated with Safe Streets and the per cent change estimate derived from the placebo tests.23–26 We calculated the proportion of control posts in the placebo tests with an estimated per cent change in homicides and non-fatal shootings that was more favourable than the per cent change estimated in each Safe Streets post. This proportion, similar to a p value, provided an assessment of the strength of the associations found in the SCM.22 28

    Geocoding of point data and aggregation to police post polygons were completed using ESRI Business Analyst 2015 software in ArcGIS Desktop V.10.4.1.29 Data management and analyses were performed in Stata/IC V.15.1 for Mac (64-bit Intel).30

    Results

    Table 1 compares the preintervention average monthly values of outcome variables and predictors in the treated police post, its synthetic control, and the unweighted pool of comparison units which were applied in the previous regression analyses. As illustrated by the larger variance between the unweighted comparison units and the treated sites in relation to the synthetic controls, the weighted approach used to generate the SCM provides closer estimates of the predictors than do the pooled unweighted comparison units. Furthermore, the prediction errors in the SCM, shown in table 2 as the RMSPE, were substantially smaller than the corresponding prediction errors from the regression models (RMSE), demonstrating that the SCM produces a better fit for estimating the counterfactuals and programme effects for each intervention site. This improved model fit can also be visually noted in graphs comparing the SCM preintervention predicted versus observed values for homicides (online supplemental figure 2A) and non-fatal shootings (online supplemental figure 2B) with the regression model preintervention predicted versus observed values for homicides (online supplemental figure 2C) and non-fatal shootings (online supplemental figure 2D). The graphs for the placebo tests showing the observed minus predicted values are shown in online supplemental figure 3A for homicides and online supplemental figure 3B for non-fatal shootings.

    View this table:
    Table 1

    Predictor balance averaged over preintervention periods for each Safe Streets site, their respective synthetic controls and the pooled unweighted comparison units

    View this table:
    Table 2

    Preintervention root mean square prediction error for synthetic control models and preintervention root mean square error for regressions in Safe Streets analyses

    The SCM estimates of programme effects are shown in table 3 as per cent change increases or decreases in homicides and non-fatal shootings post programme implementation. Estimated effects varied widely by both site and outcome. For example, the Cherry Hill and Sandtown-Winchester sites saw reductions in homicides (−21% and −9%, respectively) but not non-fatal shootings (+11% and +15%, respectively). In contrast, the Madison-Eastend and Mondawmin sites experienced increases in homicides (+69% and +76%, respectively) and non-fatal shootings (+153% and +27%, respectively) during the study period. Lower Park Heights experienced non-significant decreases in both homicides and non-fatal shootings, while the homicide increases in Mondawmin and Elwood Park and the non-fatal shootings in Madison-Eastend were significantly different from the placebo test results.

    View this table:
    Table 3

    Estimated Safe Streets programme effects: per cent change, range of per cent change in control posts in placebo tests and proportion of control posts with better outcomes in placebo tests than Safe Streets posts

    We found inconsistent programme effect estimates over time for the two outcomes in McElderry Park and Cherry Hill (table 4). In McElderry Park, the programme was associated with homicide reductions for the 3-year (−62%), 5-year (−48%) and 7-year (−24%) estimates, indicating attenuated protective effects on homicide over time, while non-fatal shootings increased relative to the counterfactual over the same time periods. In contrast, Cherry Hill saw relatively stable programme-related decreases in homicides and greater reductions in non-fatal shootings at each duration tested.

    View this table:
    Table 4

    The 3-year, 5-year and 7-year estimated effects for McElderry Park and Cherry Hill (proportion of control posts with better outcomes)

    The results of the preunrest and postunrest analyses suggest that the civil unrest had varied impacts in the Safe Streets locations (table 5). For example, compared with its preunrest estimate for non-fatal shootings, Lower Park Heights appears to have had a postunrest reduction in non-fatal shootings relative to the estimated counterfactual, while Mondawmin, which was at the centre of the civil unrest, saw dramatic increases in both homicides and non-fatal shootings following the unrest relative to its synthetic control.

    View this table:
    Table 5

    Estimated Safe Streets programme effects for preunrest (through March 2015) and full study periods (through December 2017)

    Discussion

    This study estimated the effects of Safe Streets, a public health programme based on the Cure Violence model, in seven intervention sites in Baltimore, Maryland, using SCM. The findings indicate that Safe Streets has had disparate impacts on violence across implementation sites with more evidence of harm than benefit. Three sites—Madison-Eastend, Elwood Park and Mondawmin—experienced substantial increases in both fatal and non-fatal violence during the study period. These sites lost their funding after experiencing poor outcomes. While other sites did see programme-related reductions in at least one violence outcome, none of the seven Safe Streets sites experienced violence reductions outside the norm when compared with placebo tests.

    The varied effect estimates provide additional context and nuance to the existing literature on Cure Violence interventions. The findings that Safe Streets in Baltimore was associated with significant increases in violence in three locations raise great concern, and when considered along with the null or negative findings in a number of past Cure Violence evaluation studies, the results of this research suggest that closer examination of implementation and contextual factors that influence programme effects is warranted. These findings may also suggest that programme enhancements or modifications are needed to achieve significant reductions in firearm violence in the neighbourhoods with exceptionally high rates of violence.

    This study offered interesting comparisons with the 2018 Safe Streets evaluation. Both analyses showed that the effects of the programme varied widely by location. However, the regression analyses estimated that Cherry Hill experienced a significant reduction in homicide, while the SCM analysis in the current study found that, although Cherry Hill did experience homicide reductions, the effects were not significant. The two analytic approaches also yielded some differences in the magnitude of effect estimates. Given the model fit comparisons highlighted in this study, the findings from the SCM analyses are likely better estimates of Safe Streets’ site-specific effects than were the previous regression analyses.

    Taken in aggregate, the evaluations of Safe Streets suggest that the initial potentially positive effects of the programme attenuated over time. Prior studies found that Safe Streets increased youth’s preferences for non-violent conflict resolution31 and was associated with increases in preferences for non-violent responses to interpersonal conflicts.32 Additionally, the Cure Violence model has been associated with reductions in youth’s willingness to use violence to settle conflict,33 improved confidence in police34 and increased confidence in a community’s ability to reduce firearm violence.9 However, it is important to note that community outreach and violence interruption programmes, regardless of branding, vary from site to site and city to city in their implementation, investment and management strategy. There is limited information on implementation differences across Safe Streets sites; each received the same training, city contracts and technical assistance.11 The differential programme effects found in this and other studies and the attenuation of early protective effects underscore the importance of strong implementation, appropriate oversight and meaningful support of community-level violence interventions like Safe Streets.2 3 Geographical and contextual differences across sites may also influence the model’s probability for success. For example, the Cherry Hill area is geographically isolated, which may offer unique protection that other sites do not, and a cross-sectional study of Baltimore found that Safe Streets-associated reductions in homicides were associated with a higher proportion of gang-related conflict mediations, while neighbourhoods without similar programme-associated homicide reductions saw more mediations for weapons and retaliatory conflicts.35 The underlying drivers of potentially violent interpersonal disputes vary by community, and they must be understood and appropriately matched with interventions that can adequately address those drivers. Future research should also examine how the theoretical concepts of the Cure Violence model apply to various communities in Baltimore and how the strategy might be adapted or enhanced to better achieve the desired outcomes.

    Furthermore, promising findings from studies of Cure Violence programmes in New York suggest that programme success may be associated with increased financial, mental health and training resources for staff and programme participants, as well as the availability of wraparound services for the individuals engaged by outreach workers—things that have been lacking for Safe Streets Baltimore.2 9 33 34 Future research should examine distinct variations in implementation across sites and focus on how discrepancies in programme impact may be explained by factors such as programme oversight, outreach worker training and support, collaboration with community-based organisations, and the type, number, quality and utilisation of support and social services available to programme participants.

    This study is the first known research to assess how civil unrest and increases in community violence, particularly following incidents of police violence, may influence the outcomes of violence interventions. While deeper investigation of unrest-related effects was beyond the scope of this study, the increases in violence in three of the four sites in operation when the unrest occurred suggest that Safe Streets may have been ill-equipped to address the flared tensions and increased violence that frequently occur in communities experiencing unrest following incidents of police violence.36 37 This finding importantly sheds light on similar challenges experienced during the summer of 2020, when dozens of cities across the US saw both civil unrest and coincident increases in violence, even in places where community-based violence prevention strategies existed but were hamstrung by the COVID-19 pandemic.

    There are several study limitations to note. First, due to large differences in operational periods across the programme sites, the amount of data available to estimate preintervention trends or postintervention effects differed greatly and affected model fit. For instance, according to their respective RMSPEs, the models for the Safe Streets sites in Mondawmin and Sandtown-Winchester had better fit than did those for the longest-standing sites in McElderry Park and Cherry Hill. Conversely, the earlier sites benefited from much more postintervention data than did the sites that opened more recently. There was no appealing strategy for addressing this limitation. However, the RMSPEs were small for all sites, and the examination of programme effects over different time periods provided insight into variance across sites over time, irrespective of programme length.

    An additional limitation is that the SCM is unable to account for breaks in the intervention. The intervention’s effects following the April 2015 civil unrest could not be isolated, although we attempted to account for the effects by estimating Safe Streets’ impact on violence preunrest and then for the entire duration of the study period. Similarly, temporary breaks in the programmes in McElderry Park and Mondawmin as a result of site reviews following programme worker arrests could not be modelled with the SCM. However, those programme interruptions were brief and thus unlikely to have had a major impact on the intervention’s effects in those areas.

    Despite these limitations, the current study makes important contributions to the body of literature on community-level violence interventions. To our knowledge, this is the first study to use SCM to examine site-specific and time-varying effects of a Cure Violence programme model on violence outcomes. It is also the longest examination of a Cure Violence programme’s operation in multiple cities within one city, as well as the first to consider differential programme impacts following civil unrest spurred by police violence.

    The current study illuminates challenges associated with achieving and sustaining reductions in violence using a programme like Safe Streets. It also elevates the urgent need to provide greater support for the workers who are tasked with outreach and violence interruption and risk their lives to reduce violence in communities. The model has demonstrated some success in other prior quantitative and qualitative evaluations, but variation in effectiveness highlights the importance of closer examination of various aspects of the programme. The results from this study emphasise careful implementation and continuous evaluation of violence prevention efforts. This research also points to a greater need to more deliberately identify and replicate critical components of effective community outreach and violence interruption work, and to more directly address systemic drivers of community violence, in order to reduce violence in communities nationwide.

    What is already known on the subject

    • Cure Violence is a violence prevention model that has been employed to interrupt violence, facilitate service referrals and promote non-violence among individuals at high risk for violence involvement.

    • While studies of the intervention’s impact on violence have generally been positive, effect estimates across replication sites have varied widely, suggesting the need for further examination.

    • Prior analyses of Cure Violence have used either traditional regression analyses with panel data or interrupted time-series approaches that have limited utility for estimating the counterfactual for intervention areas and generating effect estimates.

    What this study adds

    • The synthetic control method produces more accurate non-intervention comparisons for estimating programme effects than prior research.

    • Programme effects varied by site and outcome, suggesting that intervention effects are dependent on neighbourhood-specific and site-specific context.

    • Drivers of potentially violent interpersonal disputes must be understood and appropriately matched with interventions that can adequately address those drivers.

    Data availability statement

    Census block group-level data are available in a public, open access repository. Data on homicides, non-fatal shootings and arrests were obtained through both the Baltimore City Open Data portal (https://data.baltimorecity.gov/) and the Baltimore Police Department. Police Department data requests must be submitted to the agency.

    Ethics statements

    Patient consent for publication

    Ethics approval

    This study was deemed to be 'not human subjects research' by the Johns Hopkins Bloomberg School of Public Health Institutional Review Board.

    References

    1. ↵
      Federal Bureau of investigation. crime in the US, 2017. Available: https://ucr.fbi.gov/crime-in-the-u.s./2017/crime-in-the-u.s.-2017 [Accessed 2 Apr 2019].
    2. ↵
      1. Baltimore Police Department
      . Homicide, nonfatal shooting, and arrest data 2018.
    3. ↵
      1. Institute of Medicine and National Research Council
      . Contagion of violence: workshop summary. Washington, DC: The National Academies Press, 2013.
    4. ↵
      Cure violence website. Available: http://cureviolence.org/ [Accessed 6 Apr 2020].
    5. ↵
      2. Butts JA ,
      3. Roman CG ,
      4. Bostwick L , et al
      . Cure violence: a public health model to reduce gun violence. Annu Rev Public Health 2015;36:39–53.doi:10.1146/annurev-publhealth-031914-122509 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25581151
      OpenUrlCrossRefPubMed
    6. ↵
      2. Skogan W ,
      3. Hartnett SM ,
      4. Bump N
      . Evaluation of Ceasefire-Chicago. Chicago, IL: Northwestern University, 2008. https://www.ncjrs.gov/pdffiles1/nij/grants/227181.pdf. (Accessed April 6, 2020).
    7. ↵
      2. Fox AM ,
      3. Katz CM ,
      4. Choate DE , et al
      . Evaluation of the Phoenix TRUCE project: a replication of Chicago CeaseFire. Justice Quarterly 2015;32:85–115.doi:10.1080/07418825.2014.902092
      OpenUrl
    8. ↵
      2. McVey E ,
      3. Duchesne JC ,
      4. Sarlati S , et al
      . Operation CeaseFire-New Orleans: an infectious disease model for addressing community recidivism from penetrating trauma. J Trauma Acute Care Surg 2014;77:123–8.doi:10.1097/TA.0000000000000274 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24977766
      OpenUrlPubMed
    9. ↵
      2. Picard-Fritsche S ,
      3. Cerniglia L
      . Testing a public health approach to firearm violence: an evaluation of crown heights save our streets, a replication of the cure violence model, 2013. Available: http://www.courtinnovation.org/sites/default/files/documents/SOS_Evaluation.pdf [Accessed 6 Apr 2020].
    10. ↵
      2. Roman CG ,
      3. Klein HJ ,
      4. Wolff KT
      . Quasi-Experimental designs for community-level public health violence reduction interventions: a case study in the challenges of selecting the counterfactual. J Exp Criminol 2018;14:155–85.doi:10.1007/s11292-017-9308-0
      OpenUrl
    11. ↵
      2. Webster DW ,
      3. Whitehill JM ,
      4. Vernick JS , et al
      . Effects of Baltimore's safe streets program on gun violence: a replication of Chicago's CeaseFire program. J Urban Health 2013;90:27–40.doi:10.1007/s11524-012-9731-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22696175
      OpenUrlCrossRefPubMed
    12. ↵
      2. Webster DW ,
      3. Buggs SAL ,
      4. Crifasi CK
      . Estimating the effects of law enforcement and public health interventions intended to reduce firearm violence in Baltimore. Johns Hopkins center for firearm policy and research, 2018. Available: https://www.jhsph.edu/research/centers-and-institutes/johns-hopkins-center-for-firearm-policy-and-research/publications/JHSPH-Firearm-Violence-in-Baltimore.pdf [Accessed 6 Apr 2020].
    13. ↵
      2. Bouttell J ,
      3. Craig P ,
      4. Lewsey J , et al
      . Synthetic control methodology as a tool for evaluating population-level health interventions. J Epidemiol Community Health 2018;72:673–8.doi:10.1136/jech-2017-210106 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29653993
      OpenUrlAbstract/FREE Full Text
    14. ↵
      City of Baltimore. open data. Available: https://data.baltimorecity.gov/ [Accessed 6 Apr 2020].
    15. ↵
      2. Castillo-Carniglia A ,
      3. Kagawa RMC ,
      4. Webster DW , et al
      . Comprehensive background check policy and firearm background checks in three us states. Inj Prev 2018;24:431–6.doi:10.1136/injuryprev-2017-042475 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28986427
      OpenUrlAbstract/FREE Full Text
    16. ↵
      2. Saunders J ,
      3. Lundberg R ,
      4. Braga AA , et al
      . A synthetic control approach to evaluating Place-Based crime interventions. J Quant Criminol 2015;31:413–34.doi:10.1007/s10940-014-9226-5
      OpenUrl
    17. ↵
      2. Rydberg J ,
      3. McGarrell EF ,
      4. Norris A , et al
      . A quasi-experimental synthetic control evaluation of a place-based police-directed patrol intervention on violent crime. J Exp Criminol 2018;14:83–109.doi:10.1007/s11292-018-9324-8
      OpenUrl
    18. ↵
      United States census bureau. American community survey, 2005-2009 5-year estimates; detailed tables.
    19. ↵
      2. Rudolph KE ,
      3. Stuart EA ,
      4. Vernick JS , et al
      . Association between Connecticut's Permit-to-Purchase handgun law and homicides. Am J Public Health 2015;105:e49–54.doi:10.2105/AJPH.2015.302703 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26066959
      OpenUrlCrossRefPubMed
    20. ↵
      2. Crifasi CK ,
      3. Meyers JS ,
      4. Vernick JS , et al
      . Effects of changes in permit-to-purchase handgun laws in Connecticut and Missouri on suicide rates. Prev Med 2015;79:43–9.doi:10.1016/j.ypmed.2015.07.013 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26212633
      OpenUrlCrossRefPubMed
    21. ↵
      2. Caliendo M ,
      3. Kopeinig S
      . Some practical guidance for the implementation of propensity score matching. J Econ Surv 2008;22:31–72.doi:10.1111/j.1467-6419.2007.00527.x
      OpenUrlCrossRefWeb of Science
    22. ↵
      2. Bonander C
      . Compared with what? estimating the effects of injury prevention policies using the synthetic control method. Inj Prev 2018;24:i60–6.doi:10.1136/injuryprev-2017-042360 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29127114
      OpenUrlAbstract/FREE Full Text
    23. ↵
      2. Abadie A ,
      3. Gardeazabal J
      . The economic costs of conflict: a case study of the Basque country. Am Econ Rev 2003;93:113–32.doi:10.1257/000282803321455188
      OpenUrlCrossRefWeb of Science
    24. ↵
      2. Abadie A ,
      3. Diamond A ,
      4. Hainmueller J
      . Synthetic control methods for comparative case studies: estimating the effect of California’s tobacco control program. J Am Stat Assoc 2010;105:493–505.doi:10.1198/jasa.2009.ap08746
      OpenUrlCrossRefWeb of Science
    25. ↵
      2. Abadie A ,
      3. Diamond A ,
      4. Hainmueller J
      . Comparative politics and the synthetic control method. Am J Pol Sci 2015;59:495–510.doi:10.1111/ajps.12116
      OpenUrlCrossRef
    26. ↵
      2. Doudchenko N ,
      3. Balancing IGW
      . Regression, difference-in-differences and synthetic control methods: a synthesis. No. w22791. National Bureau of Economic Research 2016.
    27. ↵
      2. Desmond C ,
      3. Watt K ,
      4. Saha A , et al
      . Prevalence and number of children living in institutional care: global, regional, and country estimates. Lancet Child Adolesc Health 2020;4:370–7.doi:10.1016/S2352-4642(20)30022-5 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32151317
      OpenUrlPubMed
    28. ↵
      2. McCourt AD ,
      3. Crifasi CK ,
      4. Stuart EA , et al
      . Purchaser licensing, point-of-sale background check laws, and firearm homicide and suicide in 4 us states, 1985-2017. Am J Public Health 2020;110:1546–52.doi:10.2105/AJPH.2020.305822 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32816544
      OpenUrlPubMed
    29. ↵
      1. ESRI
      . ArcGIS desktop: release 10. Redlands, Ca environmental systems research Institute 2015.
    30. ↵
      1. StataCorp
      . Stata statistical software: release 15. College Station, TX: StataCorp LLC, 2017.
    31. ↵
      2. Milam AJ ,
      3. Buggs SA ,
      4. Furr-Holden CDM , et al
      . Changes in attitudes toward guns and shootings following implementation of the Baltimore safe streets intervention. J Urban Health 2016;93:609–26.doi:10.1007/s11524-016-0060-y pmid:http://www.ncbi.nlm.nih.gov/pubmed/27294969
      OpenUrlPubMed
    32. ↵
      2. Milam AJ ,
      3. Furr-Holden CD ,
      4. Leaf P
      . Managing conflicts in urban communities: youth attitudes regarding firearm violence. J Interpers Violence 2016:1–14.
    33. ↵
      2. Delgado S ,
      3. Alsabahi L ,
      4. Butts JA
      . Young men in neighborhoods with Cure Violence programs adopt attitudes less supportive of violence. JohnJayREC - The Research and Evaluation Center at John Jay College of Criminal Justice, City University of New York, 2017. Available: https://johnjayrec.nyc/wp-content/uploads/2017/03/databit201701.pdf [Accessed 6 Apr 2020].
    34. ↵
      2. Butts JA ,
      3. Delgado SA
      . Repairing trust: young men in neighborhoods with Cure Violence programs report growing confidence in police. JohnJayREC - The Research and Evaluation Center at John Jay College of Criminal Justice, City University of New York, 2017. Available: https://johnjayrec.nyc/wp-content/uploads/2017/10/Repairing2017.pdf [Accessed 6 Apr 2020].
    35. ↵
      2. Whitehill JM ,
      3. Webster DW ,
      4. Vernick JS
      . Street conflict mediation to prevent youth violence: conflict characteristics and outcomes. Inj Prev 2013;19:204–9.doi:10.1136/injuryprev-2012-040429 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23002073
      OpenUrlAbstract/FREE Full Text
    36. ↵
      2. Webster DW ,
      3. Crifasi CK ,
      4. Williams RG
      . Reducing violence and building trust: data to guide enforcement of gun laws in Baltimore. Johns Hopkins center for gun policy and research, 2020. Available: https://www.jhsph.edu/research/centers-and-institutes/johns-hopkins-center-for-gun-policy-and-research/_docs/reducing-violence-and-building-trust-gun-center-report-june-4-2020.pdf [Accessed 10 Dec 2020].
    37. ↵
      2. Ali SS
      . Gun violence is surging in cities, and hitting communities of color hardest. NBC news, 2020. Available: https://www.nbcnews.com/news/us-news/gun-violence-surging-cities-hitting-communities-color-hardest-n1233269 [Accessed 6 Jan 2021].

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Contributors SAB and DWW were responsible for the conception and design of the project, as well as interpretation of data. SAB and CKC developed the coding for analysis. SAB was responsible for data collection and management, data analysis and drafting of the article. CKC and DWW contributed to critical revisions of the article. SAB, DWW and CKC provided final approval of the version to be published.

    • Funding This research was supported by the National Institute on Drug Abuse (5T32DA007292-23 and 4T32DA007292-24), the Centers for Disease Control and Prevention (1U01CE001954-01A1), and the Johns Hopkins Center for Gun Violence Prevention and Policy. Funders had no role in the study design, statistical analyses, interpretations of data or drafting of this article.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.