Article Text
PDF
Abstract
Background Comprehensive background check (CBC) policies are hypothesised to reduce firearm-related violence because they extend background checks to private party firearm sales, but no study has determined whether these policies actually increase background checks, an expected intermediary outcome. We evaluate the association between CBC policies and the rates of firearm background checks in three states that recently implemented these policies: Delaware (July 2013), Colorado (July 2013) and Washington (December 2014).
Methods We used the synthetic control group method to estimate the difference from estimated counterfactual postintervention trends in the monthly rate of background checks per 1 00 000 people for handguns, long guns and both types combined, using data for January 1999 through December 2016. Inference was based on results from permutation tests. We conducted multiple sensitivity analyses to assess the robustness of our results.
Results Background check rates increased in Delaware, by 22%–34% depending on the type of firearm, following enactment of its CBC law. No overall changes were observed in Washington and Colorado. Our results were robust to changes in the comparison group and statistical methods.
Conclusions The enactment of CBC policies was associated with an overall increase in firearm background checks only in Delaware. Data external to the study suggest that Washington experienced a modest, but consistent, increase in background checks for private party sales, and Colorado experienced a similar increase in checks for sales not at gun shows. Non-compliance may explain the lack of an overall increase in background checks in Washington and Colorado.
- firearms
- policy
- violence
- time series
Statistics from Altmetric.com
Introduction
Firearm violence is among the leading causes of death and injury in the USA, with firearms involved in >36 000 deaths in 2015.1 About 2 85 000 violent crimes were committed using a firearm that year, injuring >47 000 people.2 The estimated costs attributable to the US firearm injuries in 2012 were >US$229 billion.3 These statistics highlight the need for more evidence on how to prevent the negative health and social consequences of violence at the population level.
Comprehensive background check (CBC) policies are among the potentially most important firearm policies in place or under consideration across the USA.4 Current federal law requires a background check only when a transfer is made by a licensed retailer. CBC policies expand the requirements for background checks to nearly all gun transfers, including those among private parties. Approximately 36% of firearm transfers in recent years were private party transfers5; previously, as many as 40% of transfers were private.6 CBC policies are sometimes augmented by permit-to-purchase (PTP) requirements, whereby prospective purchasers apply for a permit directly with a law enforcement agency charged with processing background checks.
The absence of a background check policy facilitates firearm acquisition by prohibited persons, who can avoid screening measures by purchasing firearms from sources other than licensed retailers. Under federal law, prohibited persons include felons, those convicted of domestic violence crimes or subject to final domestic violence restraining orders, anyone who is ‘an unlawful user of or addicted to any controlled substance’, who ‘has been adjudicated as a mental defective or who has been committed to a mental institution’ and others.7 As many as 80% of firearms acquired for criminal purposes, and 96% of those acquired by prohibited persons, are acquired through private party transfers.8 By expanding background check requirements to private party transfers, CBC policies are expected to make it harder for felons and other prohibited persons to purchase firearms, thereby reducing firearm violence. The potential impact could be substantial. In an Illinois study, 42.6% of homicide offenders were felons; the proportion of all homicide offenders who were prohibited from possessing firearms would necessarily be larger.9
The limited available data from studies of PTP policies suggest that background checks and purchase permits are an effective means of reducing firearm violence. The most relevant evidence comes from a study of the repeal of a PTP policy in Missouri.10 Until repeal, Missouri required a permit and background check for all handgun purchases. Repeal was associated with a 23% increase in firearm homicide. Separately, the implementation of a PTP law in Connecticut was associated with a 40% decrease in firearm homicide.11 Ecological studies have found that improving the rigorousness of background checks has beneficial effects on firearm-related mortality.12 13
Enactment of a CBC policy, whether or not a PTP requirement is included, could reasonably be expected to increase the number of firearm purchase background checks as those from private transactions are added. The absence of such an increase might indicate that the new policy is not being observed or enforced.
The objective of this study is to evaluate the effect of implementing CBC policies on the monthly rates of background checks in three states: Delaware (implementation July 2013), Colorado (July 2013) and Washington (December 2014). These three states adopted quite similar CBC policies, but too recently to assess subsequent changes in violent crimes or suicides. Their new policies did not include a PTP requirement. Oregon adopted a CBC policy in August 2015, but was excluded because of the short observation period following adoption. Our hypothesis was that CBC implementation would be associated with an increased rate of background checks in the intervention states relative to those that had not implemented these policies.
Methods
Data and outcome
We analysed the monthly rate (per 1 00 000 people) of handgun, long gun (rifle and shotgun) and total background checks (including those coded as private sales) in Delaware, Washington and Colorado, using state-level data compiled and published by the Federal Bureau of Investigation (FBI) through its National Instant Criminal Background Check System (NICS).14 Under the Brady Handgun Violence Prevention Act of 1993 (Public Law 103–159), all firearm transactions through a federally licensed firearms retailer became subject to a NICS background check in November 1998. For our study, the first point in the time series was January 1999, and the last was December 2016.
Covariates
We based covariate selection on a comprehensive search of the firearm violence literature, including other studies with similar aims and analytic approaches.11 15–17 The final set of covariates comprised the following: % people between 18 and 24 years of age, % male, % Hispanic, % black, % graduated from high school, % under the poverty line, median household income (5-year estimates from the American Community Survey of 201018), annual rate of violent crimes (murder, rape, robbery, aggravated assault) per 1 00 000 inhabitants collected by the FBI,19 % urban, Gini coefficient (Census Bureau for 2010) and annual firearm suicide as a percentage of total suicide (a proxy for firearm ownership20 21 obtained from the Centers for Disease Control and Prevention’s Web-based Injury Statistics Query and Reporting System).1 Covariates were averaged over the entire preintervention time period following the approach used by Abadie et al.22 We also included the rate of firearm background checks at three time points before the implementation of the CBC policies: January 1999, 2007 and 2012 for Delaware and Washington, and January 2002, 2007 and 2012 for Colorado.
Analytic method
We used the synthetic control group method (SCGM) to estimate the difference in background checks after implementation of CBC policies between a ‘treated’ unit (in our case Delaware, Colorado or Washington) and its synthetic control.22 23 The SCGM seeks to minimise the difference in the preintervention trends for the outcome of interest (in this case, background checks) between the treated unit and a weighted average of the units available for comparison (the donor pool). The indicator used to measure this difference is the root mean square prediction error (RMSPE), where a RMSPE equal to zero would mean perfect overlap between the value of the treated unit and its synthetic control prior to the intervention.
The weighted average of the states in the donor pool is an estimate of the counterfactual trend in the outcome in the postintervention period (referred to as the ‘synthetic control’). States in the donor pool may be assigned a weight of zero, meaning they are not part of the synthetic control. The counterfactual can be interpreted as the trend in background checks that might have occurred if a CBC policy had not been enacted. The difference between the treated unit and its synthetic control in the postintervention period is the estimated effect of the intervention, and can be expressed as 
, where 
represents the value of the outcome in the treated state at time t in the postintervention period, and 
is the weight that is assigned to each state in the donor pool 
such that 
and 
. To estimate the weights for each unit in the donor pool, the SCGM uses information from variables that are predictive of the preintervention outcome in the treated unit, including preintervention measures of the outcome. The algorithm for the synthetic control method seeks to estimate a vector of weight W in order to minimise 
, where 
and 
are the values of the set of predictors in the preintervention period for the treated and donor states, respectively, and V is a positive and semi-definite matrix, which allows for different weights to the variables 
and 
.22 We generated different models for each outcome and for each state of interest.
Our donor pool comprised states that did not have CBC policies (including PTP policies) during the study period. We excluded states with such policies (California, Connecticut, Hawaii, Illinois, Iowa, Maryland, Massachusetts, Michigan, Nebraska, New Jersey, New York, North Carolina, Pennsylvania and Rhode Island), and states that changed their policy status at any time point after January 1999 (Missouri and Oregon). We also excluded Indiana and Tennessee from the donor pool because they repealed CBC policies in 1998, and their background check trend from 1999 onwards was likely affected by that change. The trend and rates of background checks in Colorado from 1999 to 2001 were highly unusual compared with those seen later and in states in the donor pool (see figure 1), leading us to exclude these years from the main analysis. Handgun sales increase after mass shootings, which may partly explain the large rate of background checks in Colorado during the years in question, which followed the Columbine shooting.24
Annual trend in background check rates (per 1 00 000 people) by type of firearm in Delaware, Washington and Colorado, and the 29 states in the donor pool, 1999–2016. Horizontal lines represent the annual firearm background check rate per 1 00 000 people for Delaware (black dot line), Washington (black dash line) and Colorado (black long-dash line). Horizontal grey dot lines represent the trend of each state in the donor pool (n=29). Vertical lines indicate time when CBC was enacted: July 2013 in Delaware and Colorado (dash line); December 2014 in Washington (dash-dot line).
To assess the level of confidence in our results, we performed a permutation test, also called a placebo or falsification test, in which we created synthetic controls for each state in our donor pool. No intervention took place in these states; each was compared with its synthetic control in the ‘postintervention’ period in order to compare the magnitude of the differences that could be expected if no policy were implemented.22 23 Given that not all estimates from the permutation test produce the same fit in the preintervention period, we used the ratio between the post-RMSPE and pre-RMSPE to compare the magnitude of the change across states. That way the difference in the postintervention period for each placebo test is related to the method’s ability to produce a good fit in the preintervention period.
Sensitivity analyses
To assess the robustness of our findings, we performed multiple descriptive analyses, comparing the trend in background checks by number, rate and standardised rate in the treated states with their neighbours and other potential control states. We repeated the analysis using different sets of predictors and preintervention outcome years and the complete preintervention period for Colorado.
We also conducted an interrupted time-series analysis to assess the robustness of our findings to different methods of analysis. We used neighbouring states and other states that had regional proximity to the treated states but did not have or enact CBC or similar policies (resulting in a subset of the donor states included in the SCGM). Using these criteria, we selected four control states for Delaware (Virginia, South Carolina, Georgia and Florida) and Washington (Idaho, Nevada, Minnesota and Wisconsin) and eight for Colorado (Arizona, New Mexico, Utah, Nevada, Idaho, Wyoming, Montana and Texas). As when using the synthetic control method, we excluded from the main analysis the first 3 years for Colorado.
All analyses were conducted using Stata V.14.1 (StataCorp, College Station, Texas, USA).
Results
In all three states, the annual rate of firearm background checks has trended upwards since 2002 (figure 1). Plots of the average monthly rates (see online supplementary figure S1) show a clear seasonal pattern; peaks occur in December and troughs in the summer. Between January and July, the rates for handgun and long gun background checks are quite similar, but from August through December the rate increases more quickly for long guns.
Supplementary file 1
The states contributing to the nine synthetic controls (three outcomes for three states) and their weights are presented in table 1. In general, there was a good fit between the trends in the treated states and their synthetic controls, with RMSPE reductions (in comparison to the average from all donor states) of >65% for Delaware and Washington. The gap between Colorado and synthetic Colorado was greater than for the other two states (figure 2, see online supplementary figure S2). Predictor variables were generally well balanced for each state and its synthetic control, in particular those variables with large weights, suggesting the synthetic controls were well matched to the treated states (see online supplementary tables S1–S3).
Weights and goodness of fit from the synthetic control analyses for Delaware, Washington and Colorado
Trend in monthly handgun background check rates per 1 00 000 people in Delaware, Washington and Colorado and their synthetic controls, 1999–2016.
The largest changes in background checks after CBC policy implementation were observed for Delaware, with increases ranging between 22% and 34% in relation to estimated counterfactual trends (table 2). The permutation test showed that these results were highly unusual compared with states with no CBC law. None of 29 states in the donor pool saw differences in long gun background checks as large as those in Delaware, two had greater changes for handguns and one for both types of firearm combined.
Background check rate (per 100 000 people) after CBC implementation in Delaware, Washington and Colorado
As shown in figure 2 and online supplementary figures S2 and S3, there seems to be a 10-month lag between when the law went into effect and when the number of background checks increased in Delaware; the average difference in the rate of background checks between Delaware and synthetic Delaware after that 10 months was 87.7 for both types of firearm combined, 47.2 for handguns and 60.2 for long guns (increases of 30%, 31% and 47%, respectively).
The differences between the rate of background checks observed in Washington and Colorado and that of their synthetic controls were smaller in magnitude and within the differences witnessed in the permutation test, indicating that there were no detectable changes after CBC implementation.
Results from the synthetic control method and the permutation tests are displayed in online supplementary figures S2–S4. Online supplementary figure S5 presents a comparison of each study state with its neighbours, with rates standardised at 100 in the year before policy implementation. Again, background checks increase in Delaware, but not in Washington and Colorado, after implementation. Results from other sensitivity analyses (eg, using counts instead of rates, changing the criteria defining donor states and using the complete preintervention period for Colorado) were similar (data not shown).
The results of the interrupted time series analysis differ modestly from the synthetic control analysis but support the same conclusions (see online supplementary table S4). At the midpoint postintervention, Delaware had an increase in background checks for handguns (coefficient=38; 95% CI −13 to 63), long guns (coefficient=74; 95% CI 30 to 118) and both types combined (coefficient=112; 95% CI 50 to 175), corresponding to a relative increase ranging from 25% (handguns) to 47% (long guns). Washington had a 16% relative decrease in handgun background checks (coefficient=−23.6; 95% CI −46.7 to –0.5); there were no changes in Colorado.
Discussion
To our knowledge, this is the first study of the association between enactment of CBC policies and changes in background check rates. Enactment was not associated with a change in the rate of background checks in the short term except in Delaware, where we observed an increase (in reference to the counterfactual trend) of 25% for handguns and 34% for long guns. Our hypothesis stated that with the implementation of CBC policies there would be an increase in background checks, since all private sales would be subject to checks. This hypothesis was based on the findings that 36%–40% of firearm transactions were between private parties5 6 and that 50% of these transfers involved a background check (43% in states without CBC).5 Assuming that these proportions were similar in our states of interest, we should expect an increase of around 27% in background checks, which is close to the percentages observed for Delaware.
Our results were contrary to our hypothesis in two of three states and are not easy to reconcile with those of other studies, which show an inverse relation between PTP policy enactment and firearm deaths.10 11 It is possible that the additional permit requirement is essential for background check requirements to increase background checks, but several other explanations are more likely.
One plausible explanation for our findings is low compliance in our study states. In Washington, there was a well-documented public “I will not comply” rally at the state capital, at which firearms were openly transferred between private parties without background checks.25 There were also gun shows where non-compliance was encouraged26 and public calls from profirearm organisations to not comply with the state’s new CBC policy.25 The implication of such events, at least one of which was organised specifically to evade the new Washington CBC law, is that individuals who might have sold a firearm at a different time and place, with a background check, might specifically have chosen to use such events to sell that firearm without a background check. This would produce a lower number of total background checks than might otherwise have occurred. Unfortunately, the impact of those actions cannot be properly quantified.
Another explanation for the null findings may be insufficient enforcement of the law by the states’ authorities. Many county law enforcement officials in Colorado reportedly stated they would not enforce its CBC law, and some retailers were declining to process background checks for private party transfers.27 Washington law enforcement agencies announced there would be no arrests for selling guns at the non-compliance rally and gun show.26 28 Preliminary data from a study of two Eastern states suggest that willingness to prosecute violations of such laws can vary substantially (Crifasi CK, personal communication, 2017).
A low proportion of private sales in the target states could mean that our study was conducted before sufficient time had elapsed after CBC implementation for us to detect a meaningful change. In relation to the potentially too-low proportion of private sales, Colorado statistics show that handgun background checks from private sales have accounted for only 5% on average of all handgun background checks since the implementation of the CBC policy.29 Given the relatively short period of observation and natural variation in the pattern of background checks, our study is not powered to detect an increase of <5% in the overall background check rate. But Colorado’s CBC policy seems to have shifted the composition of private firearms transactions in that state. Colorado had required checks for private party sales at gun shows since 2000. Vernick et al recently showed that after implementation, background checks for private party sales at gun shows decreased, while background checks for other private party sales increased.30
To the best of our knowledge, there are no data available to conduct an evaluation of CBC policies looking at changes in background check for private transactions. The Colorado Bureau of Investigation compiled data on background checks for private transaction, but only beginning 1 year prior to the implementation of their CBC policy (data used in Vernick et al30). The NICS dataset reports background checks for private party transactions, but only after the implementation of CBC policies. Using these data, we saw that Washington experienced a modest, but consistent, increase in background checks for private party sales; unfortunately, we cannot know if this increase is part of a secular trend or an effect of the CBC policy implemented in that state.
We were unable to obtain information on implementation or compliance in Delaware, either from internet searches of local media outlets or by repeated request from Delaware state agencies.
CBC policies may have beneficial effects on firearm-related violence without increasing background checks. The generally accepted mechanism of action of CBC policies is that they extend background checks to firearm transfer between private parties, preventing prohibited persons from acquiring firearms and committing firearm-related violence.9 However, CBC policies might act by other mechanisms. For example, requiring private party sales to go through regulated retailers could interfere with firearm trafficking operations that relied on non-prohibited purchasers, whose purchases would begin generating permanent records and are therefore never made. This would likely reduce the efficiency of such operations in providing firearms for criminal purposes.31 32
Our study has both particular strengths and limitations. We used a state-of-the-art method to evaluate policy changes within a single unit, with results that were compatib‘le w’ith those from other methods and sets of assumptions. We performed multiple sensitivity analyses and in all cases obtained similar results. The main limitation is the lack of information to better explain the different findings across states. For example, we do not know the preintervention proportion of firearm transactions that involved private parties in any of our study states, nor do we have information beyond anecdote about compliance with and enforcement of the law in those states. To the best of our knowledge, no studies have evaluated the validity and accuracy of the NICS dataset.
These results complicate an assessment of the effect of CBC policies on firearm-related violence. Similar studies in Oregon and Nevada, which have enacted CBC policies more recently, and assessments of the impact of CBC policies on violence will be helpful.
What is already known on this subject?
More restrictive firearm policies are associated with less firearm-related violence.
Comprehensive background check policies are among the most widely recommended strategies to reduce firearm violence.
What this study adds?
Implementation of comprehensive background check policies was associated with an increase in background checks in only one of three states evaluated.
These results suggest that mere enactment of comprehensive background checks policies may not reduce firearm violence.
Stronger enforcement and enactment of complementary policies (eg, permit to purchase) may be required to prevent firearm violence.
Acknowledgments
The authors are deeply grateful for the contributions made by Philip J Cook.
References
Footnotes
Contributors AC-C was involved in the design, and led the implementation of the study, including literature review, data analyses and interpretation and manuscript writing; RMCK was involved in developing the analytical strategy, literature review and manuscript writing; DWW, JSV and MC were involved in the design of the study, interpretation of the findings and critical review of the manuscript and GJW was involved in the conceptualisation, design and implementation of the study, including literature review, interpretation of the findings and manuscript writing. All authors read and approved the final version of the manuscript.
Funding This study was funded by the Joyce Foundation (grant ID 15-36377) and the Heising-Simons Foundation (grant ID 2016-219). AC-C and RMCK were supported by the Robertson Fellowship in Violence Prevention Research. AC-C was also supported by Becas Chile as part of the National Commission for Scientific and Technological Research (CONICYT).
Disclaimer The funders did not participate in the design and conduct of the study; in the collection, analysis and interpretation of the data or in the preparation, review or approval of the manuscript.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.