Where Does Bias in Common Sense Knowledge Models Come From?

Common Sense knowledge bases and models have been shown to embed bias. We investigate the source of such bias in a knowledge model called common sense transformer (COMET) by training it on various combinations of language models and knowledge bases. We experiment with three language models of different sizes and architectures, and two knowledge bases with different modeling principles. We use sentiment and regard as proxy measures of bias and analyze bias using three methods: overgeneralization and disparity, keyword outliers, and relational dimensions. Our results show that larger models tend to be more nuanced in their biases but are more biased than smaller models in certain categories (e.g., utility of religions), which can be attributed to the larger knowledge accumulated during pretraining. We also observe that training on a larger set of common sense knowledge typically leads to more bias, and that models generally have stronger negative regard than positive.

Automated summary

It has been found that bias exists in common sense knowledge bases and models. The study investigates the source of bias in a knowledge model called COMET by training it on different combinations of language models and knowledge bases. Bias is measured using sentiment and regard as proxies, and analyzed through three methods: overgeneralization and disparity, keyword outliers, and relational dimensions. The results show that larger models are more nuanced in their biases but can be more biased than smaller models in certain categories (e.g. utility of religions), which is attributed to the larger knowledge accumulated during pretraining. It is also observed that training on a larger set of common sense knowledge often leads to more bias, and that models generally have stronger negative regard than positive.