Statistics from Altmetric.com
The inclusion of unpublished and grey literature is essential for minimizing the potential effects of publication bias. It is well known that published studies can not be assumed to be an accurate representation of the whole evidence base, as studies that show statistically significant, “positive” results are more likely to be published than those that do not.1 Consequently, if systematic reviews are limited to published studies, they risk excluding vital evidence and yielding inaccurate results, which are likely to be biased to positive results.2 Previous research has indicated that meta-analyses that exclude grey literature can lead to exaggerated estimates of intervention effects.34
It is therefore considered essential that active and extensive searching for unpublished and grey literature is undertaken as part of the review process for all CIG reviews, although we feel this to be particularly important for reviews of injury prevention interventions.
In comparison to reviews covering other aspects of the CIG’s scope, we have found that the CIG’s injury prevention reviews have a higher proportion of included studies sourced from the grey and unpublished literature. In issue 3, 2007, of the Cochrane Database of Systematic Reviews, there are 24 reviews of interventions for injury prevention published by the CIG. These reviews examine a total of 481 included studies, with 137 (28.5%) of these having originated from the grey or unpublished literature. The percentage of such included studies ranges from zero to 85%, with six5–10 having more than 50% of their included studies sourced from grey or unpublished literature: these six are all reviews of interventions for preventing road traffic crashes.
The above figures highlight that there is a substantial amount of injury prevention research located in the grey and unpublished literature, which is never formally published. Therefore, it is vital that review authors are proactive in their efforts to identify and obtain such research reports. However, identifying and obtaining reports on studies that have been completed, but never published, is extremely challenging. There are a number of research data sources that we advise our authors to explore, in addition to the standard bibliographic database searching, in an attempt to identify unpublished reports.
Cochrane Central Register of Controlled Trials (CENTRAL). CENTRAL is the most comprehensive source of records related to controlled trials, and is searched for each Cochrane review. It incorporates not only records for controlled trials that have been downloaded from bibliographic databases, but it also includes records of unpublished studies identified through other means, such as handsearching.
Handsearching. This is the manual page-by-page examination of the entire contents of a publication to identify all eligible reports of studies. It is generally limited to journal issues and conference proceedings collections, which include multiple individual reports or abstracts of studies that may not be adequately indexed elsewhere.
Databases of unpublished and ongoing studies. These are databases specifically containing grey literature, such as Zetoc (http://zetoc.mimas.ac.uk/), which indexes conference proceedings. Additionally, there are a number of registers of ongoing studies, including TrialsCentral™ (www.trialscentral.org) and Current Controlled Trials (www.controlled-trials.com).
Internet. With the increasing availability of reports and official documents on the Internet, searching for these documents is now easier, although full texts are not always available. Using a well thought-out search string on search engines such as Google (http://www.google.co.uk/) and Google Scholar (http://scholar.google.com), authors can identify both published and unpublished research reports. Authors can also target the websites of relevant organizations, which often have their own online, searchable publications library.
Reference lists. Further studies can be identified by checking the reference lists of other eligible studies. The reference lists of any previous literature reviews can be particularly helpful for any further potentially relevant studies.
Contact with experts. It can sometimes be worthwhile to contact experts in the field who are familiar with the literature and who might be able to advise review authors of any unpublished studies of which they are aware.
Despite a review team’s greatest efforts to identify all relevant unpublished and published studies, the possibility of the presence of publication bias can never be dismissed. We would, therefore, suggest that authors make an effort to measure the extent to which publication bias may be present in their review, such as by examining funnel plots.
The full text of all of the CIG’s injury prevention reviews are published in the Cochrane Database of Systematic Reviews (www.thecochranelibrary.com). For further information about the work of the CIG, visit www.cochrane-injuires.lshtm.ac.uk.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.