Modeling Touch-based Menu Selection Performance of Blind Users via Reinforcement Learning

Although menu selection has been extensively studied in HCI, most existing studies have focused on sighted users, leaving blind users' menu selection under-studied. In this paper, we propose a computational model that can simulate blind users' menu selection performance and strategies, including the way they use techniques like swiping, gliding, and direct touch. We assume that selection behavior emerges as an adaptation to the user's memory of item positions based on experience and feedback from the screen reader. A key aspect of our model is a model of long-term memory, predicting how a user recalls and forgets item position based on previous menu selections. We compare simulation results predicted by our model against data obtained in an empirical study with ten blind users. The model correctly simulated the efect of the menu length and menu arrangement on selection time, the action composition, and the menu selection strategy of the users.

Automated summary

This paper proposes a computational model that simulates blind users' menu selection behavior. The model takes into account the impact of long-term memory on users' selection behavior and is validated against empirical study data.