Journal of Safety Research

Journal of Safety Research

Volume 32, Issue 3, 1 November 2001, Pages 267-276
Journal of Safety Research

Identifying controlled evaluation studies of road safety interventions: Searching for needles in a haystack

https://doi.org/10.1016/S0022-4375(01)00055-XGet rights and content

Abstract

Problem: Worldwide, over one million people die and about 10 million people sustain permanent disabilities each year in road traffic crashes. It is a matter of urgency that effective strategies, especially from existing controlled trials (CTs), are identified for crash prevention programs. Methods: We used word frequency analysis to develop a search strategy of known sensitivity and positive predictive value (PPV) to identify reports of controlled evaluation studies of road safety interventions in the TRANSPORT database. Results: 23,554 records were searched and 319 (1.4%) records of controlled evaluation studies were identified by handsearching. We were unable to devise search strategies that combined acceptable sensitivity and PPV. Impact on industry: Efforts to improve the identification of CTs of road safety interventions are urgently required. This would involve the documentation of the study design in the title, abstract, and methods section of the research report. The study methodology should be carefully and consistently indexed in road safety databases, and editors can encourage appropriate indexing by insisting on the use of structured abstracts that give details of study methodology.

Introduction

It is estimated that, worldwide, over one million people die and some 10 million people sustain permanent disabilities each year in road traffic crashes (Murray & Lopez, 1996). For people under 35 years of age, traffic crashes are the leading cause of death and disability. Moreover, the global epidemic of traffic injuries is only beginning. It is estimated that by 2020, road traffic crashes will be the third leading cause of healthy life years lost worldwide and the second leading cause in developing countries (Murray & Lopez, 1997).

An epidemic of this scale demands a timely public health response. It is a matter of urgency that effective strategies are identified for the prevention of road traffic crashes. A logical first step is to find out what is already known about the effectiveness of road crash prevention strategies by conducting systematic reviews of controlled evaluation studies. A systematic review is a concise summary of the best available evidence that addresses a sharply defined question. Conducting a systematic review is a structured process involving several steps:

  • (a) a well-formulated question,

  • (b) a comprehensive data search,

  • (c) an unbiased selection and abstraction process,

  • (d) a critical appraisal of data, and

  • (e) synthesis of data.

The Cochrane Injuries Group is an international network of individuals who prepare, maintain, and promote the accessibility of systematic reviews in the prevention, treatment, and rehabilitation of injury (see Cochrane Injuries Group's website). The Injuries Group maintains a register of reports of controlled trials (CTs) in order to facilitate the preparation and updating of systematic reviews.

We used word frequency analysis (Boynton, Glanville, McDaid, & Lefebvre, 1998) to develop electronic search strategies of known sensitivity and positive predictive value (PPV) to identify reports of controlled evaluation studies of road safety interventions in the TRANSPORT database. The TRANSPORT database was chosen because it is the largest and most widely used road-related database. The TRANSPORT database contains approximately 600,000 records from the Transportation Research Information Services (TRIS) database, the International Road Research Documentation (IRRD) database, and the TRANSDOC database from the European Conference of Ministers of Transport (ECMT). In addition to records from journals, books, and conference proceedings, it contains a high proportion of references to grey literature (i.e., publications not controlled by commercial publishers).

Section snippets

Inclusion criteria for relevant reports

The aim of this study was to develop a sensitive search strategy to identify reports of controlled evaluation studies of road safety interventions. Although randomized CTs (RCTs) provide the most valid evidence about effectiveness, these are sometimes not possible in the evaluation of road safety interventions. Consequently, we sought to identify a broader range of study methodologies, including RCT, CT, controlled before and after studies (CBA), and interrupted time series (ITS). Definitions

Results

Of the 23,554 records corresponding to the handsearched reports, 319 records (1.4%) met the inclusion criteria. There were 50 (16%) reports of CT, 112 (35%) CBA, and 157 (49%) ITS. There were 175 eligible records in the set used to develop search strategies and 144 in the set used for retesting the selected search strategies.

Discussion

Identification of effective interventions for the prevention of road traffic crashes is of global health importance. Controlled evaluation studies provide the most valid information on effectiveness. We used word frequency analysis in an attempt to devise electronic search strategies with high sensitivity and PPV for the retrieval of controlled evaluation studies of road safety interventions in the TRANSPORT database. No strategies could be identified. As would be expected, there was an inverse

Acknowledgements

The Medical Research Council in the UK funded this study as part of the Health of the Public Initiative.

Reinhard Wentz is a senior information scientist at the Medical Library at Imperial College of Science, Technology and Medicine in London. He spent 2 years on secondment with the Cochrane Injuries Group as Trials Search Coordinator. His professional interests include evidence-based medicine, critical appraisal, and systematic literature reviews.

References (9)

There are more references available in the full text version of this article.

Cited by (0)

Reinhard Wentz is a senior information scientist at the Medical Library at Imperial College of Science, Technology and Medicine in London. He spent 2 years on secondment with the Cochrane Injuries Group as Trials Search Coordinator. His professional interests include evidence-based medicine, critical appraisal, and systematic literature reviews.

Ian Roberts is Professor of Epidemiology and Public Health at the London School of Hygiene and Tropical Medicine. He is coordinating editor of the Cochrane Injuries Group, an international network of individuals who prepare and maintain systematic reviews of the effectiveness of interventions in the prevention, treatment, and rehabilitation of injury.

Frances Bunn is the Review Group Coordinator of the Cochrane Injuries Group based at the London School of Hygiene and Tropical Medicine.

Phil Edwards is a Research Fellow at the Department of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine.

Irene Kwan is a Research Fellow of the Cochrane Injuries Group based at the London School of Hygiene and Tropical Medicine.

Carol Lefebvre is an information specialist at the Cochrane Centre, Oxford, UK.

View full text