Purpose: The purpose of the study is to compare different approaches to the identification of confounders needed for analyzing observational data. Whereas standard analysis usually is conducted as if the confounders were known a priori, selection uncertainty also must be taken into account.
Methods: Confounders were selected by using backward elimination (BE), change in estimate (CIE) method, Akaike information criterion, Bayesian information criterion (BIC), and an empirical approach using a priori information. A modified ridge regression estimator, which shrinks effects of confounders toward zero, also was considered. For each criterion, uncertainty in the estimated exposure effect was assessed by using bootstrap simulations for which confounders were selected in each sample. These methods were illustrated by using data for mercury neurotoxicity in Faroe Islands children. Point estimates and standard errors of mercury effects on confounder-sensitive neurobehavioral outcomes were calculated for each selection procedure.
Results: The full model and the empirical a priori model showed approximately the same precision, and these methods were (slightly) inferior to only modified ridge regression. Lower precisions were obtained by using BE with a low cutoff level, BIC, and CIE.
Conclusions: Standard analysis ignores model selection uncertainty and is likely to yield overoptimistic inferences. Thus, the traditional BE procedure with p = 5% should be avoided. If data-dependent procedures are required for confounder identification, we recommend that inferences be based on bootstrap statistics to describe the selection process.