TagLab: AI-assisted annotation for the fast and accurate semantic segmentation of coral reef orthoimages

Semantic segmentation is a widespread image analysis task; in some applications, it requires such high accuracy that it still has to be done manually, taking a long time. Deep learning-based approaches can significantly reduce such times, but current automated solutions may produce results below expert standards. We propose agLab, an interactive tool for the rapid labelling and analysis of orthoimages that speeds up semantic segmentation. TagLab follows a human-centered artificial intelligence approach that, by integrating multiple degrees of automation, empowers human capabilities. We evaluated TagLab's efficiency in annotation time and accuracy through a user study based on a highly challenging task: the semantic segmentation of coral communities in marine ecology. In the assisted labelling of corals, TagLab increased the annotation speed by approximately 90% for nonexpert annotators while preserving the labelling accuracy. Furthermore, human-machine interaction has improved the accuracy of fully automatic predictions by about 7% on average and by 14% when the model generalizes poorly. Considering the experience done through the user study, TagLab has been improved, and preliminary investigations suggest a further significant reduction in annotation times.

Automated summary

Semantic segmentation is a widely-used image analysis task, and deep learning-based approaches have the potential to significantly reduce manual annotation time. However, current automated solutions may not meet expert standards. TagLab is introduced as an interactive tool that speeds up semantic segmentation by integrating multiple degrees of automation to empower human capabilities. Through a user study on coral community segmentation, TagLab demonstrated a 90% increase in annotation speed for nonexpert annotators without compromising labeling accuracy, and improved fully automatic predictions by 7% on average and by 14% in challenging cases. Preliminary investigations also suggest further significant reductions in annotation times.