Article Text

Download PDFPDF

Do scientific publications change anything?
  1. I B Pless
  1. Correspondence to:
 Professor I B Pless

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Website hits as a measure of influence

The “simple” answer is it all depends on who you want to influence, what you want to change, and by how much. It may also depend on how optimistic or lucky you are. But the best answer is that it is surprisingly difficult to be certain one way or the other.

Journal papers have several audiences: other scientists, clinicians, policy makers, the media, and the public. In the case of injury prevention, program persons should also be on this list. These are not mutually exclusive: one may influence another. For example, if the press are interested in a publication, this may help persuade program directors or policy makers to follow the authors’ recommendations and make changes. This is especially likely if media reports stimulate widespread interest.

The most direct targets of a scientific paper, however, are other scientists. It is they who read journals the most and who quote one another’s work. Good science usually builds on the findings of others, although in my experience this logical sequence does not happen as often as it should. Nevertheless, as a rule most authors do cite pertinent work of colleagues. Accordingly, these citation counts may be one reasonable indicator that a fellow researcher’s thinking about a problem has changed. And, nowadays it is relatively easy to count the number of such citations. A related measure of influence1—and thus, presumably, “change”—is how often a paper is read. One index of this is how often it has been accessed when it appears on a website such as ours.

Several months ago I gave Grand Rounds to the Department of Surgery at the Hospital for Sick Children and chose to reflect on whether scientific publications helped us achieve our goals. In that presentation, I examined citations in depth. This editorial’s focus is on how often Injury Prevention papers on our website are accessed. What these “hits” mean is controversial but whatever the interpretation they tell something about the popularity of a paper—a likely harbinger of change at some level.

The table that follows lists the 10 papers published in Injury Prevention that garnered the most website accesses during 2004. Also shown is how often these papers were cited based on a Google Scholar search. This is an imperfect method for eliciting citations.2 Much preferable is to use the ISI Scientific Information Science Citation Index on the Web of Science database (WOS)3 as shown in column 3. Unfortunately, as yet, not all papers published in this journal have been indexed on WOS. Although these numbers are low, bear in mind that as a rule there are few citations in the first year or two after publication and those listed only go back to 2002. Later, if a paper is a winner, the number gradually increases.

The main message in this table is the number of web accesses. Nowadays many more people view a paper online than cite it. Even an all-time bestseller such as the paper by Sue Baker et al on the Injury Severity Score4 has “only” been cited 2203 times whereas Jacobsen’s paper5 has been accessed over 5000 times. I maintain that web hits are a reasonable indicator of the amount of interest a paper has generated in general, not exclusively by researchers, because our website like most others is available to the media and the public.

We need to acknowledge, however, that both measures are unsatisfactory. Authors who persuade their friends and relations to log on to see what they have done could inflate hits. By the same token, citations may be a misleading index of genuine importance if a paper is controversial or seriously flawed. Thus, hits or citations are only approximate measures of the popularity, let alone the importance or influence of a paper. But, when all is said and done, I believe they tell something of value.

The influence of a paper on clinicians or other practitioners, or in the case of injury prevention, on public health officers or program persons, may be either reasonably direct or circuitous. The direct route assumes that when rational practitioners read a solid paper published in a reputable journal, most will follow the paper’s recommendations. More often, however, the route is indirect and readers only take the recommended action when other factors come into play. For example, a pharmaceutical representative may bear gifts to encourage a clinician to prescribe the company’s medication featured in a journal report. For better or worse, injury prevention is no part of the pharma world. Sometimes I wistfully wish that it were, or that some safety products would be worth advertising.

For papers with a public health message, there are no middle men akin to pharmaceutical representatives. Instead, there are other sinister factors that need to be considered such as the economic and political ramifications of the author’s message. These are the most likely explanations why, based on the same data, public health officers or their counterparts were able to have bicycle helmet legislation passed in some places and not in others.6,7

That the public can be influenced by scientific publications heavily depends on both the media and the message. If reporters write about a paper we publish the public will be influenced in one way or another. A good reporter explaining the evidence supporting the benefits of safer playgrounds, for example, is quite likely to change the thinking of some parents and this, in turn, may influence others, including politicians.8,9 Obviously some messages are not intended for the general public because they are too esoteric. But others, such as those with promising clinical or public health results will reach the public if the media finds the message newsworthy.

The bottom line? Of course editors, authors, and publishers believe that any paper in any journal can change things. Much depends on luck to be sure, but there are other strategies to increase the chances of success. My list of ways to do so, by no means complete, includes writing clearly and as briefly as is feasible, involving the media whenever possible, and having direct contact with the person or group whose behaviors or ideas you want to change. If your goal is to influence other researchers, think about what it is that made the papers shown in the table of such widespread interest. One factor that is heavily related to “citation impact” is the study design, with meta-analyses being the hands-down winner.10 But the most enduring element in deciding whether a paper will be published and in what journal remains the quality of the science.

Table 1

 Top 10 most frequently accessed Injury Prevention papers in 2004 and their citation frequency

Website hits as a measure of influence