Journal
TRANSACTIONS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS
Volume 11, Issue -, Pages 488-510Publisher
MIT PRESS
DOI: 10.1162/tacl_a_00558
Keywords
-
Ask authors/readers for more resources
We propose the AMSAL algorithm, which removes implicit information from neural representations. The algorithm alternates between finding an assignment of input representations to be erased and creating projections of both input representations and information to be erased into a joint latent space. Our results demonstrate the effectiveness of our algorithm on multiple datasets, including those with guarded and protected attributes.
We present the Assignment-Maximization Spectral Attribute removaL (AMSAL) algorithm, which erases information from neural representations when the information to be erased is implicit rather than directly being aligned to each input example. Our algorithm works by alternating between two steps. In one, it finds an assignment of the input representations to the information to be erased, and in the other, it creates projections of both the input representations and the information to be erased into a joint latent space. We test our algorithm on an extensive array of datasets, including a Twitter dataset with multiple guarded attributes, the BiasBios dataset, and the BiasBench benchmark. The latter benchmark includes four datasets with various types of protected attributes. Our results demonstrate that bias can often be removed in our setup. We also discuss the limitations of our approach when there is a strong entanglement between the main task and the information to be erased.(1)
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available