Recognizing Uncertainty in the Q-Matrix via a Bayesian Extension of the DINA Model

In the typical application of a cognitive diagnosis model, the Q-matrix, which reflects the theory with respect to the skills indicated by the items, is assumed to be known. However, the Q-matrix is usually determined by expert judgment, and so there can be uncertainty about some of its elements. Here it is shown that this uncertainty can be recognized and explored via a Bayesian extension of the DINA (deterministic input noisy and) model. The approach used is to specify some elements of the Q-matrix as being random rather than as fixed; posterior distributions can then be used to obtain information about elements whose inclusion in the Q-matrix is questionable. Simulations show that this approach helps to recover the true Q-matrix when there is uncertainty about some elements. An application to the fraction-subtraction data of K. K. Tatsuoka suggests a modified Q-matrix that gives improved relative fit.