Unfounded conclusions of equivalence in diagnostic accuracy studies: a pervasive fallacy of inference in pediatric radiology scientific abstracts

BackgroundIn studies of diagnostic performance that fail to detect a statistically significant difference between compared techniques, investigators often declare evidence of equivalence or similarity without having actually tested that assertion due to incorrect methodology or insufficient statistical power.ObjectiveThe purpose of our investigation is to measure the prevalence of unfounded assertions of equivalence or similarity in comparison studies presented at the International Pediatric Radiology (IPR) meeting of 2016 and promote awareness of this fallacy of inference to the pediatric radiology community.Materials and methodsTwo pediatric radiologists independently reviewed the methodology and reporting quality of the 194 scientific paper abstracts from the 2016 IPR meeting. All comparison studies were identified and those failing to detect a statistically significance difference and making a claim of equivalence or similarity in the results or conclusion were assessed for a description of the study design type, statistical power and sample size estimator calculation.ResultsOf 194 scientific paper abstracts, 112 (58%) were comparison studies. Of these, 36/112 (32%) made unfounded inferences of equivalence or similarity in diagnostic imaging performance. No study had an equivalence or non-inferiority design. No abstract specified the statistical power of the study, and only one abstract acknowledged a small sample size as a limitation in detecting a statistically significant difference.ConclusionInadequate reporting and unfounded inferences of equivalence or similarity were common in diagnostic performance comparison studies presented at IPR 2016. Failure to recognize these limitations could have adverse consequences by leading to the adoption of unvalidated imaging techniques.