How to read and review papers on machine learning and artificial intelligence in radiology: a survival guide to key methodological concepts

In recent years, there has been a dramatic increase in research papers about machine learning (ML) and artificial intelligence in radiology. With so many papers around, it is of paramount importance to make a proper scientific quality assessment as to their validity, reliability, effectiveness, and clinical applicability. Due to methodological complexity, the papers on ML in radiology are often hard to evaluate, requiring a good understanding of key methodological issues. In this review, we aimed to guide the radiology community about key methodological aspects of ML to improve their academic reading and peer-review experience. Key aspects of ML pipeline were presented within four broad categories: study design, data handling, modelling, and reporting. Sixteen key methodological items and related common pitfalls were reviewed with a fresh perspective: database size, robustness of reference standard, information leakage, feature scaling, reliability of features, high dimensionality, perturbations in feature selection, class balance, bias-variance trade-off, hyperparameter tuning, performance metrics, generalisability, clinical utility, comparison with traditional tools, data sharing, and transparent reporting. Key Points center dot Machine learning is new and rather complex for the radiology community. center dot Validity, reliability, effectiveness, and clinical applicability of studies on machine learning can be evaluated with a proper understanding of key methodological concepts about study design, data handling, modelling, and reporting. center dot Understanding key methodological concepts will provide a better academic reading and peer-review experience for the radiology community.

Automated summary

In recent years, there has been a significant increase in research papers on machine learning and artificial intelligence in radiology. Understanding key methodological concepts about study design, data handling, modelling, and reporting is essential for evaluating the validity, reliability, effectiveness, and clinical applicability of these studies. Mastery of these key methodological concepts can enhance the academic reading and peer-review experience within the radiology community.