Article Text

Download PDFPDF
Assessing inter-rater agreement of environmental audit data in a matched case-control study on bicycling injuries
  1. Nicole T R Romanow1,
  2. Amy B Couperthwaite2,
  3. Gavin R McCormack3,
  4. Alberto Nettel-Aguirre1,
  5. Brian H Rowe2,
  6. Brent E Hagel1
  1. 1Departments of Paediatrics and Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
  2. 2Department of Emergency Medicine, School of Public Health, University of Alberta, Edmonton, Alberta, Canada
  3. 3Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
  1. Correspondence to Dr Brent E Hagel, Departments of Paediatrics and Community Health Sciences, University of Calgary, Alberta Children's Hospital, 2888 Shaganappi Trail NW, Calgary, AB, Canada T3B 6A8; brent.hagel{at}


Background Environmental audit tools must be reliable in order to accurately estimate the association between built environmental characteristics and bicycling injury risk.

Objective To examine the inter-rater agreement of a built environment audit tool within a case-control study on the environmental determinants of bicycling injuries.

Methods Auditor pairs visited locations where bicycling injuries occurred and independently recorded location characteristics using the Systematic Pedestrian and Cyclist Environmental Scan (SPACES). Two case groups were defined: (1) where a bicyclist was struck by a motor-vehicle (MV) and (2) where the bicyclist's injuries required hospitalisation. The two corresponding control groups were (1) where non-MV bicycle-related injuries occurred and (2) where minor bicycle-related injuries occurred. Inter-rater reliability of each item on the tool was assessed using observed agreement and κ with 95% CI.

Results Ninety-seven locations were audited. Inter-observer agreement was generally high (≥95%); most items had a 1–2% difference in responses. Items with ≥5% differences between raters included path condition, slope and obstructions. For land use, path and roadway characteristics, κ ranged from 0.3 for presence of offices and cleanliness to 0.9 for schools and number of lanes; overall, 78% of items had at least substantial agreement (κ≥0.61). For bicyclists struck by a MV the proportion of items with substantial agreement was 60%, compared with 73% for non-MV related injuries. For hospitalisations and minor bicycle-related injuries, 76% of items had substantial agreement.

Conclusions Agreement was substantial for most, but not all SPACES items. The SPACES provides reliable quantitative descriptions of built environmental characteristics at bicycling injury locations.

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.