Why do we think we know what we know? A metaknowledge analysis of the salt controversy

Background: Although several public health organizations have recommended population-wide reduction in salt intake, the evidence on the population benefits remains unclear. We conducted a metaknowledge analysis of the literature on salt intake and health outcomes. Methods: We identified reports-primary studies, systematic reviews, guidelines and comments, letters or reviews-addressing the effect of sodium intake on cerebro-cardiovascular disease or mortality. We classified reports as supportive or contradictory of the hypothesis that salt reduction leads to population benefits, and constructed a network of citations connecting these reports. We tested for citation bias using an exponential random graph model. We also assessed the inclusion of primary studies in systematic reviews on the topic. Results: We identified 269 reports (25% primary studies, 5% systematic reviews, 4% guidelines and 66% comments, letters, or reviews) from between 1978 and 2014. Of these, 54% were supportive of the hypothesis, 33% were contradictory and 13% were inconclusive. Reports were 1.51 [95% confidence interval (CI) 1.38 to 1.65] times more likely to cite reports that drew a similar conclusion, than to cite reports drawing a different conclusion. In all, 48 primary studies were selected for inclusion across 10 systematic reviews. If any given primary study was selected by a review, the probability that a further review would also have selected it was 27.0% (95% CI 20.3% to 33.7%). Conclusions: We documented a strong polarization of scientific reports on the link between sodium intake and health outcomes, and a pattern of uncertainty in systematic reviews about what should count as evidence.