Bland and Altman agreement method: to plot differences against means or differences against standard? An endless tale?

Objectives: In the Bland and Altman analysis of agreement studies, there is some controversy whether to plot the differences between the Standard/actual measurement method and the test/new measurement method against their mean or to plot the differences against the standard method. Of course, this is not just a graphic dispute as a regression model is inherent in the graphical choice to test the proportional and systematic biases.Methods: We revised two relevant papers claiming to plot the differences against the standard and outlined their pitfalls taking into account the underlying statistical methodology. Furthermore, we have considered the conditions (correlation between the two measurement methods and ratio of their variances) leading correlation coefficient and regression slope between differences and means or differences and standard different from zero.Results: We have shown the situations in which the regression slope and the correlation coefficient calculated from the differences and means according to Bland and Altman approach or calculated from the differences and standard are closer to zero giving the minimum possible value of spurious proportional error between the two methods.Conclusions: We highlighted how the calculation of the expected values of the correlation coefficients and, above all, of the regression slope can be very useful for choosing the statistical model in the context of an agreement study between two measurement methods. Finally, we outlined some recommendations for understanding the real possibility of carrying out agreement or calibration studies.

Automated summary

There is controversy in agreement studies about whether to plot the differences between measurement methods against their mean or against the standard method. We revised two relevant papers and outlined their pitfalls, taking into account the underlying statistical methodology. We also considered the conditions that lead to different correlation coefficients and regression slopes between differences and means or differences and standard.