Effect of Source Type and Protective Message on the Critical Evaluation of News Messages on Facebook: Randomized Controlled Trial in the Netherlands

Background: Disinformation has become an increasing societal concern, especially due to the speed that news is shared in the digital era. In particular, disinformation in the health care sector can lead to serious casualties, as the current COVID-19 crisis clearly shows. Objective: The main aim of this study was to experimentally examine the effects of information about the source and a protective warning message on users' critical evaluation of news items, as well as the perception of accuracy of the news item. Methods: A 3 (unreliable vs reliable vs no identified source) x 2 (with protective message vs without) between-subject design was conducted among 307 participants (mean age 29 (SD 10.9] years). Results: The results showed a significant effect of source information on critical evaluation. In addition, including a protective message did not significantly affect critical evaluation. The results showed no interaction between type of source and protective message on critical evaluation. Conclusions: Based on these results, it is questionable whether including protective messages to improve critical evaluation is a way to move forward and improve critical evaluation of health-related news items, although effective methodologies to tackle the spread of disinformation are highly needed.

Automated summary

This study experimentally examined the effects of information about the source and a protective warning message on users' critical evaluation of news items and the perception of accuracy. The results showed that source information had a significant effect on critical evaluation, while including a protective message did not significantly affect critical evaluation. Furthermore, there was no interaction between type of source and protective message on critical evaluation.