Statistical approaches to identifying lapses in psychometric response data

Psychometric curve fits relate physical stimuli to an observer's performance. In experiments an observer may lapse and respond with a random guess, which may negatively impact (e.g., bias) the psychometric fit parameters. A lapse-rate model has been popularized by Wichmann and Hill, which reduces the impact of lapses on other estimated parameters by adding a parameter to model the lapse rate. Since lapses are discrete events, we developed a discrete lapse theory and tested a lapse identification algorithm to identify individual outlier trials (i.e., potential lapses) based upon an approximate statistical criterion and discard these trials. Specifically, we focused on stimuli sampled using an adaptive staircase for a one-interval, direction-recognition task (i.e., psychometric function ranging from 0 to 1 and the spread of the curve corresponds to the threshold, which is often a parameter of interest for many fitted psychometric functions). Through simulations, we found that as the lapse rate increased the threshold became substantially overestimated, consistent with earlier analyses. While the lapse-rate model reduced the overestimation of threshold with many lapses, with lower lapse rates it yielded substantial threshold underestimation, though less so when fitting many (e.g., 1,000) trials. In comparison, the lapse-identification algorithm yielded accurate threshold estimates across a wide range of lapse rates (from 0 to 5%), which is critical since the lapse rate is seldom known. We further demonstrate the performance of the lapse-identification algorithm to be suitable for a variety of experimental conditions and conclude with some considerations of its use. In particular, we suggest using the lapse-identification algorithm unless the experiment has many trials (e.g., >500) or if somehow the lapse rate is known to be high (e.g., >= 5%), for which the lapse-rate model approaches remain preferred.

Automated summary

Psychometric curve fits relate physical stimuli to an observer's performance. In experiments, the impact of lapses on psychometric fit parameters can be reduced by using a lapse-rate model. A discrete lapse theory and lapse identification algorithm were developed to identify and discard outlier trials due to lapses, resulting in accurate threshold estimates across various lapse rates. The lapse-identification algorithm is recommended for experiments with fewer trials or unknown lapse rates, while lapse-rate model approaches are preferred for experiments with many trials or high known lapse rates.