Comparing incomparable survey responses: Evaluating and selecting anchoring vignettes

When respondents use the ordinal response categories of standard survey questions in different ways, the validity of analyses based on the resulting data can be biased. Anchoring vignettes is a survey design technique, introduced by King et al. (2004, Enhancing the validity and cross-cultural comparability of measurement in survey research. American Political Science Review 94 [February]: 191-205), intended to correct for some of these problems. We develop new methods both for evaluating and choosing anchoring vignettes and for analyzing the resulting data. With surveys on a diverse range of topics in a range of countries, we illustrate how our proposed methods can improve the ability of anchoring vignettes to extract information from survey data, as well as saving in survey administration costs.