Augmenting the algorithm: Emerging human-in-the-loop work configurations

How do configurations of humans and algorithms evolve as firms adopt artificial intelligence (AI) capabilities, and what are the implications for work and organization? We explored these questions through a two-year long case study of an organization in the international maritime trade that introduced automated algorithmic support for data analysis and prediction work. Drawing on a human-machine configuration perspective, we found that humans and the algorithm were configured and reconfigured in multiple ways over time as the organization dealt with the introduction of algorithmic analysis. In contrast to replacing human work, the emergent configurations required new roles and redistribution of extant expertise to augment and improve the accuracy of the algorithm. Our analysis suggests that the new configuration resembled a human-in-the-loop pattern, comprised of both the augmentation work of auditing (i.e. the generation of a ground truth and assessment of the algorithmic output against this) as well as the work of altering the algorithm and the data acquisition architecture. Our research points to the strategic importance of a human-in-the-loop pattern for organizational reflexivity to ensure that the performance of the algorithm meets the organization's requirements and changes in the environment.