AVOIDING AN OPPRESSIVE FUTURE OF MACHINE LEARNING: A DESIGN THEORY FOR EMANCIPATORY ASSISTANTS

Widespread use of machine learning (ML) systems could result in an oppressive future of ubiquitous monitoring and behavior control that, for dialogic purposes, we call Informania. This dystopian future results from ML systems' inherent design based on training data rather than built with code. To avoid this oppressive future, we develop the concept of an emancipatory assistant (EA), an ML system that engages with human users to help them understand and enact emancipatory outcomes amidst the oppressive environment of Informania. Using emancipatory pedagogy as a kernel theory, we develop two sets of design principles: one for the near future and the other for the far-term future. Designers optimize EA on emancipatory outcomes for an individual user, which protects the user from Informania's oppression by engaging in an adversarial relationship with its oppressive ML platforms when necessary. The principles should encourage IS researchers to enlarge the range of possibilities for responding to the influx of ML systems. Given the fusion of social and technical expertise that IS research embodies, we encourage other IS researchers to theorize boldly about the long-term consequences of emerging technologies on society and potentially change their trajectory.

Automated summary

The article discusses the potential oppressive future of widespread use of machine learning systems, proposing the concept of an emancipatory assistant (EA) to help users achieve emancipatory outcomes in the oppressive environment of Informania. With emancipatory pedagogy as the core theory, design principles are developed for both near-term and long-term future, aiming to protect users from oppression by engaging in adversarial relationship with oppressive ML platforms when necessary. The principles encourage IS researchers to expand the possibilities for responding to the influx of ML systems, and theorize about the long-term consequences of emerging technologies on society.