AI in the hands of imperfect users

As the use of artificial intelligence and machine learning (AI/ML) continues to expand in healthcare, much attention has been given to mitigating bias in algorithms to ensure they are employed fairly and transparently. Less attention has fallen to addressing potential bias among AI/ML's human users or factors that influence user reliance. We argue for a systematic approach to identifying the existence and impacts of user biases while using AI/ML tools and call for the development of embedded interface design features, drawing on insights from decision science and behavioral economics, to nudge users towards more critical and reflective decision making using AI/ML.

Automated summary

While bias in algorithms has received much attention, there is a need to address potential biases among human users of AI/ML and factors that influence user reliance. This article argues for a systematic approach to identifying user biases and calls for the development of interface design features informed by decision science and behavioral economics to promote critical decision-making using AI/ML.