Power, selection bias and predictive performance of the population pharmacokinetic covariate model

Identification and quantification of covariate relations is often an important part of population pharmacokinetic/pharmacodynamic (PK/PD) modelling. The covariate model is regularly built in a stepwise manner. With such methods, selection bias may be a problem if only statistically significant covariates are accepted into the model. Competition between multiple covariates may further increase selection bias, especially when there is a moderate to high correlation between the covariates. This can also result in a loss of power to find the true covariates. The aim of this simulation study was to investigate the effect on power, selection bias and predictive performance of the covariate model, when altering study design and system-related quantities. Data sets with 20-1000 subjects were investigated. Five covariates were created by sampling from a multivariate standard normal distribution. The true covariate was set up to have no, low, moderate and high correlation to the other four covariates, respectively. Data sets, in which each individual had two or three PK observations, were simulated using a one-compartment i.v. bolus model. The true covariate influenced clearance according to one of several magnitudes. Different magnitudes of residual error and inter-individual variability in the structural model parameters were also introduced to the simulation model. A total of 7400 replicate data sets were simulated independently for each combination of the above conditions. Models with one of the five simulated covariates influencing clearance and the model without any covariate were fitted to the data. The probability of selecting (according to a pre-specified P-value) the different covariates, along with the estimated covariate coefficient, was recorded. The results show that selection bias is very high for small data sets (less than or equal to50 subjects) simulated with a weak covariate effect. If selected under these circumstances, the covariate coefficient is on average estimated to be more than twice its true value, making the covariate model useless for predictive purposes. Surprisingly, even though competition from false covariates caused substantial loss in the power of selecting the true covariate, the already high selection bias increased only marginally. This means that the bias due to competition is negligible if statistical significance is also required for covariate selection. Bias and predictive performance are direct functions of power, only indirectly affected by study design and system-related quantities. Mainly because of selection bias, low-powered covariates can be expected to harm the predictive performance when selected. For the same reason these low-powered covariates may falsely appear to be clinically relevant when selected. If the aim of an analysis is predictive modelling, we do not recommend stepwise selection or significance testing of covariates to be performed on small or moderately sized data sets (<50-100 subjects).