Examining the influence of using same versus different questions on the reliability of the medical school preadmission interview

Background: Researchers generally recommend a structured format for the medical school preadmission interview (MSPI). However, the relative benefits of various elements of structure remain unexamined. Purpose: In this study, we compared the performance of a highly structured interview format with a semistructured format. Specifically, we examined how the reliability of interview ratings is likely to change when using the same versus different questions for each applicant being interviewed. Method: Variance components from a generalizability (G) study of a structured interview are used in decision studies to compare the relative efficiency of using the same versus different questions for each applicant. Results: Using different questions for each interviewee is practically as reliable as using the same questions for all applicants (G=.55 vs. .57, respectively). Conclusions: Because there are a number of drawbacks to using the same questions for all applicants (i.e., security and validity) and little advantage in terms of increased reliability, the semistructured question format should be considered when conducting the MSPI. A suggested method of implementing a semistructured interview is by presenting each applicant a set of questions randomly drawn from a pool of interview questions.