Signal detection models as contextual bandits

Signal detection theory (SDT) has been widely applied to identify the optimal discriminative decisions of receivers under uncertainty. However, the approach assumes that decision-makers immediately adopt the appropriate acceptance threshold, even though the optimal response must often be learned. Here we recast the classical normal-normal (and power-law) signal detection model as a contextual multi-armed bandit (CMAB). Thus, rather than starting with complete information, decision-makers must infer how the magnitude of a continuous cue is related to the probability that a signaller is desirable, while simultaneously seeking to exploit the information they acquire. We explain how various CMAB heuristics resolve the trade-off between better estimating the underlying relationship and exploiting it. Next, we determined how naive human volunteers resolve signal detection problems with a continuous cue. As anticipated, a model of choice (accept/reject) that assumed volunteers immediately adopted the SDT-predicted acceptance threshold did not predict volunteer behaviour well. The Softmax rule for solving CMABs, with choices based on a logistic function of the expected payoffs, best explained the decisions of our volunteers but a simple midpoint algorithm also predicted decisions well under some conditions. CMABs offer principled parametric solutions to solving many classical SDT problems when decision-makers start with incomplete information.

Automated summary

Signal detection theory (SDT) is widely used for optimal decision-making under uncertainty, but it assumes decision-makers immediately adopt the appropriate acceptance threshold, which may not be the case in real-world situations that require learning. This study recasts the traditional SDT model into a contextual multi-armed bandit (CMAB), where decision-makers must infer the relationship between a continuous cue and the desirability of a signal while seeking to exploit the acquired information. Various CMAB heuristics are discussed to address the trade-off between estimating the underlying relationship and exploiting it. The results suggest that CMABs provide principled parametric solutions to SDT problems when decision-makers have incomplete information.