The Paradoxical Effect of Low Reward Probabilities in Suboptimal Choice

When offered a choice between 2 alternatives, animals sometimes prefer the option yielding less food. For instance, pigeons and starlings prefer an option that on 20% of the trials presents a stimulus always followed by food, and on the remaining 80% of the trials presents a stimulus never followed by food (the Informative Option), over an option that provides food on 50% of the trials regardless of the stimulus presented (the Noninformative Option). To explain this suboptimal behavior, it has been hypothesized that animals ignore (or do not engage with) the stimulus that is never followed by food in the Informative Option. To assess when pigeons attend to the stimulus usually not followed by food, we increased the probability of reinforcement, p, in the presence of that stimulus. Across 2 experiments, we found that the value of the Informative Option decreased with p. To account for the results, we added to the Reinforcement Rate Model (and also to the Hyperbolic Discounting Model) an engagement function, f(p), that specified the likelihood the animal attends to a stimulus followed by reward with probability p, and then derived the model predictions for 2 forms of f(p), a linear function, and an all-or-none threshold function. Both models predicted the observed findings with a linear engagement function: The higher the probability of reinforcement after a stimulus, the higher the probability of engaging the stimulus, and, surprisingly, the less the value of the option comprising the stimulus.