Deep learning object detection in materials science: Current state and future directions

Deep learning-based object detection models have recently found widespread use in materials science, with rapid progress made in just the past two years. Scanning and tunneling electron microscopy methods are among the most important and widely used characterization techniques for understanding fundamental materials structure-property-performance linkages from the micron to atomic scale. Dramatic increases in dataset size and complexity from modern electron microscopy instruments have necessitated the development and use of automated methods of extracting pertinent features of images. Here, the use of object detection in materials science, with a focus on the analysis of features in electron microscopy images, is reviewed. Key findings and limitations of recent seminal studies using object detection to characterize and quantify defects in irradiated metal alloys, segment and analyze micro and nanoparticles, find individual atoms at the nanoscale, and detect and track objects from in situ video are reviewed. Opportunities and challenges presently facing the materials community are highlighted, where discussion of best practices for model assessment and applicability are presented, along with the potential of improved model training with synthetic data. This review concludes with offering more speculative, forward-looking thoughts on the potential of the broader materials community to construct a living ecosystem integrating community-consensus curated data and validated models as tools to best inform application of object detection and segmentation models to specific materials domains.

Automated summary

Deep learning-based object detection models have been widely used in materials science, especially in the analysis of features in electron microscopy images. This review highlights the key findings and limitations of recent studies using object detection in characterizing defects in metal alloys, segmenting and analyzing micro and nanoparticles, detecting individual atoms, and tracking objects in in situ videos. The opportunities and challenges faced by the materials community are discussed, along with best practices for model assessment and potential improvements in model training.