We need to talk about nonprobability samples

In most circumstances, probability sampling is the only way to ensure unbiased inference about population quantities where a complete census is not possible. As we enter the era of 'big data', however, nonprobability samples, whose sampling mechanisms are unknown, are undergoing a renaissance. We explain why the use of nonprobability samples can lead to spurious conclusions, and why seemingly large nonprobability samples can be (effectively) very small. We also review some recent controversies surrounding the use of nonprobability samples in biodiversity monitoring. These points notwithstanding, we argue that nonprobability samples can be useful, provided that their limitations are assessed, mitigated where possible and clearly communicated. Ecologists can learn much from other disciplines on each of these fronts.

Automated summary

In most cases, probability sampling is necessary for unbiased inference about population quantities. However, nonprobability samples are becoming popular in the era of big data, although they can lead to misleading conclusions and may have smaller effective sizes than expected. Despite recent controversies, nonprobability samples can still be useful if their limitations are acknowledged, mitigated, and communicated clearly, and ecologists can learn from other disciplines in this regard.