Invited Commentary: Dealing With the Inevitable Deficiencies of Bias Analysis-and All Analyses

Lash et al. (Am J Epidemiol. 2021;190(8):1604-1612) have presented detailed critiques of 3 bias analyses that they identify as suboptimal. This identification raises the question of what optimal means for bias analysis, because it is practically impossible to do statistically optimal analyses of typical population studies-with or without bias analysis. At best the analysis can only attempt to satisfy practice guidelines and account for available information both within and outside the study. One should not expect a full accounting for all sources of uncertainty; hence, interval estimates and distributions for causal effects should never be treated as valid uncertainty assessments-they are instead only example analyses that follow from collections of often questionable assumptions. These observations reinforce those of Lash et al. and point to the need for more development of methods for judging bias-parameter distributions and utilization of available information.

Automated summary

The study discusses the optimal practices for bias analysis and highlights the impossibility of achieving statistically optimal analysis in typical population studies. Therefore, analysis can only attempt to satisfy practice guidelines and consider available information both within and outside the study.