How important is distractor efficiency for grading Best Answer Questions?

BackgroundDistractor efficiency and the optimum number of functional distractors per item in One Best Answer Questions have been debated. The prevalence of non-functional distractors has led to a reduction in the number of distractors per item with the advantage of adding more items in the test. The existing literature eludes a definite answer to the question of what distractor efficiency best matches excellent psychometric indices. We examined the relationship between distractor efficiency and the psychometric indices of One Best Answer Questions in search of an answer.MethodsWe analysed 350 items used in 7 professional examinations and determined their distractor efficiency and the number of functional distractors per item. The items were sorted into five groups - excellent, good, fair, remediable and discarded based on their discrimination index. We studied how the distractor efficiency and functional distractors per item correlated with these five groups.ResultsCorrelation of distractor efficiency with psychometric indices was significant but far from perfect. The excellent group topped in distractor efficiency in 3 tests, the good group in one test, the remediable group equalled excellent group in one test, and the discarded group topped in 2 tests.ConclusionsThe distractor efficiency did not correlate in a consistent pattern with the discrimination index. Fifty per cent or higher distractor efficiency, not hundred percent, was found to be the optimum.

Automated summary

The study examined the relationship between distractor efficiency and psychometric indices in One Best Answer Questions by analysing 350 items used in 7 professional examinations. The results showed that while there was a significant correlation between distractor efficiency and psychometric indices, it was not perfect, and the correlation between distractor efficiency and the number of functional distractors per item was also inconsistent.