Recognizability bias in citizen science photographs

Citizen science and automated collection methods increasingly depend on image recognition to provide the amounts of observational data research and management needs. Recognition models, meanwhile, also require large amounts of data from these sources, creating a feedback loop between the methods and tools. Species that are harder to recognize, both for humans and machine learning algorithms, are likely to be under-reported, and thus be less prevalent in the training data. As a result, the feedback loop may hamper training mostly for species that already pose the greatest challenge. In this study, we trained recognition models for various taxa, and found evidence for a 'recognizability bias', where species that are more readily identified by humans and recognition models alike are more prevalent in the available image data. This pattern is present across multiple taxa, and does not appear to relate to differences in picture quality, biological traits or data collection metrics other than recognizability. This has implications for the expected performance of future models trained with more data, including such challenging species.

Automated summary

Citizen science and automated collection methods rely on image recognition for observational data, but recognition models also require large amounts of data, creating a feedback loop. Harder-to-recognize species tend to be under-reported and less prevalent in training data, hampering training for challenging species. This study found a 'recognizability bias' across multiple taxa, where species easily identified by humans and models are more prevalent in available image data, regardless of picture quality or biological traits. This has implications for training future models with more data.