Protecting against researcher bias in secondary data analysis: challenges and potential solutions

Analysis of secondary data sources (such as cohort studies, survey data, and administrative records) has the potential to provide answers to science and society's most pressing questions. However, researcher biases can lead to questionable research practices in secondary data analysis, which can distort the evidence base. While pre-registration can help to protect against researcher biases, it presents challenges for secondary data analysis. In this article, we describe these challenges and propose novel solutions and alternative approaches. Proposed solutions include approaches to (1) address bias linked to prior knowledge of the data, (2) enable pre-registration of non-hypothesis-driven research, (3) help ensure that pre-registered analyses will be appropriate for the data, and (4) address difficulties arising from reduced analytic flexibility in pre-registration. For each solution, we provide guidance on implementation for researchers and data guardians. The adoption of these practices can help to protect against researcher bias in secondary data analysis, to improve the robustness of research based on existing data.

Automated summary

This article discusses the problems that researcher biases can cause in secondary data analysis and proposes solutions and alternative approaches. These solutions include addressing biases related to prior knowledge of the data, enabling pre-registration of non-hypothesis-driven research, ensuring that pre-registered analyses are appropriate for the data, and addressing difficulties arising from reduced analytical flexibility. Implementing these practices can protect against researcher biases in secondary data analysis and improve the robustness of research based on existing data.