Bayesian Logistic Regression: A New Method to Calibrate Pretest Items in Multistage Adaptive Testing

An operational multistage adaptive test (MST) requires the development of a large item bank and the effort to continuously replenish the item bank due to concerns about test security and validity over the long term. New items should be pretested and linked to the item bank before being used operationally. The linking item volume fluctuations in MST, however, bring into question the quality of the link to the reference scale. In this study, various calibration/linking methods along with a newly proposed Bayesian logistic regression (BLR) method were evaluated by comparison with the test characteristic curve method through simulated MST response data in terms of item parameter recovery. Results generated by the BLR method were promising due to its estimation stability and robustness across studied conditions. The findings of the present study should help inform practitioners of the utilities of implementing the pretest item calibration method in MST.

Automated summary

An operational multistage adaptive test (MST) requires the development of a large item bank and the effort to continuously replenish the item bank due to concerns about test security and validity over the long term. In this study, various calibration/linking methods along with a newly proposed Bayesian logistic regression (BLR) method were evaluated by comparison with the test characteristic curve method through simulated MST response data in terms of item parameter recovery. The findings suggest that the BLR method is promising in terms of estimation stability and robustness across different conditions.