Article Text

PDF

The scientific agreement on firearm issues
  1. David Hemenway,
  2. Elizabeth P Nolan
  1. Harvard Injury Control Research Center, Harvard TH Chan School of Public Health, Boston, Massachusetts, USA
  1. Correspondence to Professor David Hemenway, Harvard Injury Control Research Center, Harvard TH Chan School of Public Health, 677 Huntington Avenue, Boston, MA 02115, USA; hemenway{at}hsph.harvard.edu

Abstract

Introduction No one has systematically collected the views of firearm researchers to determine if and where agreement exists on the scientific evidence about firearms and firearm violence.

Methods We send a short monthly on-line survey to firearm researchers. Each survey asks respondents their level of agreement with a statement about firearms, their rating of the quality of the scientific evidence on the specific issue, their familiarity with that literature and their area of expertise. Survey participants are first-authors of a firearms article published in a peer-reviewed journal since 2011. For the first 15 surveys, on average, surveys were sent to 322 researchers, and 109 researchers responded (34% response rate).

Results Among respondents, approximately 46% were public health researchers and 32% were sociologists/criminologists. Agreement exists among firearm researchers that more guns and weaker gun laws cause serious public health problems, that the costs of gun availability are typically greater than the benefits and that stronger gun laws may improve public safety and health. 84% of researchers agreed, and only 8% disagreed with the statement ‘in the United States, having a gun in the home increases the risk of suicide’. For only three statements did most respondents rate the quality of the scientific evidence as strong or very strong. Overall, there was a higher level of agreement among public health/medicine researchers than among researchers in the other disciplines.

Discussion Surveys of researchers can provide useful information about agreement on specific issues and about the quality of the scientific evidence.

Statistics from Altmetric.com

A decade ago, when discussing climate change, policy-makers and the US media frequently implied that climate science was highly uncertain. Then, a careful study of abstracts in scientific journals showed that the overwhelming majority of researchers accepted the view that climate change was happening and was caused by human action.1 Additional evidence indicated that the scientists reporting on the dangers of climate change were not alarmists, but actually tended to err on the conservative side.2

Today, in the contentious arena of firearms research and policy, there is debate about even the most basic issues such as whether more guns in civilian hands increase or decrease individual and public health, and whether gun control policies improve, reduce or have no effect on public health and safety. Indeed, there have been scientific reviews that concluded that we do not know the answers to most fundamental questions about firearms.3 ,4 We disagree. We believe that many aspects of gun violence research are developed enough to help guide public and private policies and programmes. Unfortunately, even for specific firearm topics (eg, the effect of a gun in the home on suicide), where there is an extensive literature with consistent findings,5–7 the results appear to have had little influence on entrenched positions.

To provide information on the degree of scientific agreement in the firearms arena, we use a slightly different approach to the one used for climate change. In the spring of 2014, we began conducting monthly surveys of firearm researchers. We asked if they agreed or disagreed with specific statements about firearms and to rate the quality of the evidence about that specific topic. Results should help inform journalists, policy-makers and others on the state of the science, and also help firearm researchers themselves identify where their colleagues believe that the evidence is weak and where further research would be most useful.8

Methods

Each month, researchers are asked to participate in a brief anonymous survey. No identifying information is collected. We decided to use very short monthly surveys, rather than one long survey, because we hoped it would lead to high rates of participation, prevent respondent fatigue and allow us to ask topical questions over time.

Each survey follows the same format. There are four questions. The first three have response options on a Likert scale. The first question asks respondents about their level of agreement with a statement about firearms, with six possible responses: (a) strongly disagree, (b) disagree, (c) neither agree nor disagree, (d) agree, (e) strongly agree and (f) ‘I don't know’. The second question asks respondents to rate the quality of the scientific evidence about this statement/issue with six possible responses: (a) very weak, (b) weak, (c) medium, (d) strong, (e) very strong and (f) ‘I don't know’. The third question asks them to rate their level of familiarity with the literature on the specific issue, again with six possible responses: (a) not knowledgeable, (b) slightly knowledgeable, (c) medium, (d) knowledgeable, (e) very knowledgeable and (f) ‘I don't know’. Finally, respondents are asked about their area of expertise (ie, public health/medicine; sociology/criminology; public policy; economics; other).

The surveys are conducted in Qualtrics. This project received institutional review board approval from the Harvard T. H. Chan School of Public Health.

Firearm researchers are defined as individuals who were first-authors of one or more articles published about firearms in peer-reviewed journals in the public health/medicine, public policy, economics and sociology/criminal justice literatures from 2011 to the present. Only first-authors were included because including all authors would overweigh the public health/medicine area of research, since those articles tend to have multiple authors.

Authors are identified through keyword searches using databases including Web of Science, Sociological Abstracts, Criminal Justice Abstracts and Medline. We used the search term ‘firearms’. In March 2014, a total of 1180 citations were reviewed. The following categories of citations were removed: (a) book reviews; (b) case studies; and (c) articles without a clear author. Two types of journals were also excluded: law journals (not peer-reviewed and rarely authored by researchers) and forensic journals. Certain other topics were deemed to be irrelevant based on keywords, including history articles (eg, ‘military history’, ‘civil war’); engineering or manufacturing articles; medical treatment articles in medical journals (eg, ‘treatment’, ‘management’, ‘procedures’); psychiatry or psychology of gun users and victims (eg, ‘resilience’) and guns other than firearms (eg, ‘nail guns’, ‘air guns’, ‘mole guns’, ‘electron guns’).

In March 2014, a total of 468 citations were included. After removing duplicate citations, 358 distinct first-authors remained. Of the first authors, 287 working email addresses were found and included in the initial surveys. The first survey was sent out that month, and at the beginning of every calendar year, new first-authors who published in the previous year are added to the list. As of March 2016, we were sending emails to 389 working addresses. Surveys are sent via email under the signature of the first-author. As of March 2016, there have been 15 surveys (summer months are skipped).

Topics for the first question on the surveys are typically about claims made by advocates and politicians. We also actively elicit suggestions for topics from researchers, reporters, advocates and others. We make the final decision about the question—we try to ask about issues for which there is at least some research evidence. The prior month's results are included in the current month's email invitation to participate in the survey, and detailed results are posted online at: http://www.hsph.harvard.edu/hicrc/firearm-researcher-surveys/.

We disaggregate respondents in three ways, comparing: (a) public health respondents with respondents from other disciplines, (b) respondents who believe that the scientific evidence on the issue is strong or very strong with all other respondents and (c) respondents who report being knowledgeable or very knowledgeable about the issue with everyone else. Responses across surveys are not independent since many of the same people are responding to the various surveys. We use a two-tailed dependent (paired-samples) t-test to determine statistical significance (1% level) across these types of responders (eg, public health vs others).

Results

For the first 15 surveys, on average, 322 emails were sent, 143 were opened and 109 researchers replied (34% response rate). The number of responses ranged from 85 to 150. Unfortunately, the response rate has generally fallen over time.

Among respondents, there was broad agreement about most of the topics selected (see table 1). The average difference between sides was 69% vs 11% (with 20% neither agreeing nor disagreeing or having no opinion). Three statements with the greatest difference between the strongly agree/agree and strongly disagree/disagree were: ‘In the United States, having a gun in the home increases the risk of suicide’ (84% agreed, 8% disagreed); ‘If more citizens are armed with guns, that will reduce the rates of robbery and burglary’ (80% disagreed, 9% agreed) and ‘The carrying of firearms by regular citizens enhances public safety’ (90% disagreed, 5% agreed). There was only one statement where there was no consensus (ie, the majority of respondents neither agreed nor disagreed): ‘The Eddie Eagle gun safety educational program for children effectively reduces gun accidents’ (26% disagreed, 9% agreed).

Table 1

Firearm researcher survey results

On average, about half (50%) of the respondents said they were knowledgeable (or very knowledgeable) about the specific topic; the other half had average knowledge, were slightly knowledgeable, not knowledgeable or responded ‘I don't know’. Among respondents who reported they were knowledgeable about the specific topic, the consensus was usually stronger. Among these respondents, on average, 80% were in agreement, compared with the agreement of 60% among the respondents who were not knowledgeable (p<0.01). On the opposing side, there was no statistical difference (on average, 12% of those knowledgeable disagreed with the consensus, while 9% of those not knowledgeable disagreed) (table 2).

Table 2

Disaggregated survey results

Even though there was consensus, the quality of the scientific evidence for most statements was not considered strong or very strong. For only 3 of the 15 statements/questions did a majority of respondents rate the quality of the evidence as strong or very strong: ‘In the United States, a gun in the home increases the risk of suicide’ (63%); ‘In the United States, the proliferation of guns and permissive gun policies have created a serious public health problem’ (70%); ‘Do you think having a gun in the house makes it a safer or a more dangerous place to be?’ (64%) (see table 1).

At the end of the spectrum with respect to the quality of the evidence, for the following three statements, a quarter or fewer respondents rated the quality of the evidence as strong or very strong: ‘The change in state level concealed gun carry laws in the United States over the past few decades, from more restrictive to more permissive, has reduced crime rates’ (24%); ‘Safe storage (eg, unloaded, locked up) of firearms in the home reduces the likelihood of suicide’ (25%) and ‘The Eddie Eagle gun safety educational program for children effectively reduces gun accidents’ (12%) (see table 1).

For respondents who believed the quality of the evidence was strong, there was greater agreement. On average, among those who considered the evidence strong, an average 87% of respondents were part of the consensus, compared with the agreement of 57% among respondents who did not think the scientific evidence was strong (p<0.01). In all 15 surveys, the percentage of respondents in the consensus was higher for those who believed the evidence was strong. On the opposing side, there was no statistical difference (on average, 10% of those who thought the evidence was strong disagreed with the consensus, while 12% of those who did not report that the evidence was strong disagreed) (see table 2).

On average, 46% of respondents listed their area of expertise as public health/medicine; 32% as criminology; 6% as public policy; 8% as economics and 9% as other. The consensus among public health/medicine experts was typically stronger than that among the other disciplines (76% vs 62%). Public health practitioners had a higher level of agreement than the combined other disciplines in all 15 surveys (76% vs 62%) (p<0.01) Comparing public health/medicine responders with responders from other fields, the absolute difference in consensus, perceived strength of the evidence and knowledge of the respondent was largest for statements concerning education, mental illness and suicide. The two questions where non-public health/medical respondents reported that they were both more knowledgeable and that the evidence was as strong or stronger were about the effect of the changes in concealed carry laws and the effect of having more regular citizens carrying guns (not shown).

Discussion

Broad scientific agreement exists among the responding firearm researchers concerning many aspects of gun violence. In general, agreement exists that in the USA, the proliferation of guns and weak gun laws have created a serious public health problem; guns do not provide the benefits claimed by gun advocates, and stronger gun laws may improve public health and safety.

There is no unanimity about any of the issues, as there is not about even some of the most well-established findings in science (eg, does HIV cause AIDS). But, when there is broad scientific agreement, no one—journalists, bloggers, policy-makers or the public—should engage in ‘he said–she said’ reporting that emphasises the common fact that researchers can be found on both sides of almost any issue.

Our results indicate that where researchers are more likely to believe the scientific evidence is strong, there is also likely to be higher levels of scientific agreement. For example, the evidence that a gun in the home increases the risk for suicide in the USA is very strong and robust. Many case–control studies and ecological studies find that guns substantially increase the suicide risk, and meta-analyses and reviews of the literature reach the conclusion that access to firearms increases the risk of suicide.5 ,6 ,9 ,10 Reviewing this and other evidence, the 2012 National Strategy for Suicide Prevention authored by the US Surgeon General and the National Action Alliance for Suicide Prevention concludes that ‘firearm access is a risk factor for suicide in the United States’.11

Our surveys indicate that for most firearm issues, many researchers do not think the scientific evidence is very strong. This is not surprising given the difficulty firearm researchers have had in obtaining funding.12 This lack of funding for firearms research resulted in a relatively small number of journal articles for such a large public health problem.13

Our results have limitations. First, over 55% of our emails were never opened; so, our overall response rate is only about one-third. We do not know anything about the opinions of the non-responders. We have explored other ways to increase participation, such as trying different subject email lines, with limited success. We had hoped that the very short monthly surveys, rather than one long survey, would lead to high rates of participation. Second, it is possible that authors who were not even asked to respond to the survey might have different opinions from those who were asked. For example, writing a book chapter on firearms or writing in a law journal did not lead to inclusion in the survey, nor did writing an article before 2011, nor being anything other than the first-author of a journal article. Our goal in limiting the group of authors was to make it more likely that the authors were social scientists or epidemiologists who were knowledgeable in the firearms field, while at the same time making our selection process transparent and reproducible. We excluded non-first authors because we felt they were less likely to be knowledgeable about the literature, and because there are usually multiple authors on each public health/medicine article (some of whom are statistical rather than content experts). We did not want to weigh our sample unfairly toward the public health/medical community. We excluded authors who had not written articles since 2011, because they were less likely to be knowledgeable about the recent literature. We focused on social science and epidemiological firearm issues, and excluded medical treatment, forensic and law articles. Third, as in all opinion surveys, the phrasing of the questions can affect the response. Therefore, we tried to phrase the statements in ways as close as possible to the way the claims were being made in the literature or in public discussion. We wrote our survey statements to allow for comparisons to public surveys and to claims in the scientific literature.

We tried to all for comparisons to public surveys and to claims made by politicians, researchers and others. In one survey, we asked: ‘Periodically the Gallup poll asks this exact question of the general public. Please give your own opinion. “Do you think having a gun in the house makes it a safer or a more dangerous place to be?”’ In 2014, Gallup reported that 63% of Americans believed that a gun in the house makes the house safer, up from 47% in 2006.14 By contrast, only 5% of our researcher respondents said guns make the home safer, and 64% said they make the home less safe.

Some gun researchers have been claiming for years that guns are used in self-defence four times more often than they are used in crime.15 ,16 Only 8% of our researcher respondents agreed with that claim, and 73% disagreed. There has also been the claim17 that the change in state-level concealed gun carrying laws from ‘may-issue’ (police discretion) to ‘must issue’ (no police discretion) has reduced crime, a claim that has been highly contested.18 Only 9% of the researchers answering our poll agreed with the claim, while 62% disagreed.

One of the virtues of our ongoing surveys is that we can examine researcher consensus concerning topical issues. For example, in January 2016, an open carry bill in Florida included as a ‘Declaration of Policy’ that ‘The Legislature finds as a matter of public policy and fact that the possession and carrying of weapons and firearms by law-abiding individuals for lawful purposes, including self-defense, enhances public safety’.19 In February 2016, we asked respondents to rate their level of agreement with the following statement: ‘The carrying of firearms by regular citizens enhances public safety’. Only 5% agreed, and 90% disagreed.

Although scientific consensus is not always right, it can be a useful guide to understanding the world. Our approach of surveying researchers is one way to provide information on the level of agreement among researchers concerning what the science says.

What is already known on the subject

  • Firearm policies are among the most controversial in the USA.

  • For a very limited number of issues in the firearms area (eg, the relationship between a gun in the home and suicide), there have been reviews of the scientific literature.

What this study adds

  • We survey and present data on the opinions of firearm researchers on various topics concerning firearms.

  • We also present data on their opinions about the quality of the scientific evidence on each of these topics.

  • We are thus able to know the opinions of these experts. For example, we now know that only a very small percentage agree with the statement that ‘having a gun in the house makes it a safer place’.

References

View Abstract

Footnotes

  • Contributors DH conceived the project and wrote the initial draft. EPN was instrumental in conducting the project and revised and edited the manuscript. Both authors take full responsibility for the overall content as guarantors.

  • Funding The Joyce Foundation provided financial support for this project.

  • Competing interests None declared.

  • Ethics approval Harvard School of Public Health Human Subjects Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement Data are available on the Harvard Injury Control Research Center website.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.