Automated detection and classification of southern African Roman seabream using mask R-CNN

The availability of relatively cheap, high-resolution digital cameras has led to an exponential increase in the capture of natural environments and their inhabitants. Video-based surveys are particularly useful in the un-derwater domain where observation by humans can be expensive, dangerous, inaccessible, or destructive to the natural environment. However, a large majority of marine data has never gone through analysis by human experts - a process that is slow, expensive, and not scalable. We test a Mask R-CNN object detection framework for the automated localisation, classification, counting and tracking of fish in unconstrained underwater envi-ronments. We present a novel, labelled image dataset of roman seabream (Chrysoblephus laticeps), a fish species endemic to Southern Africa, to train and validate the accuracy of our model. The Mask R-CNN model accurately detected and classified roman seabream on the training dataset (mAP50 = 80.29%), validation dataset (mAP50 = 80.35%), as well as on previously unseen footage (test dataset) (mAP50 = 81.45%). The fact that the model performs well on previously unseen data suggests that it is capable of generalising to new streams of data not included in this research.

Automated summary

The availability of affordable, high-resolution digital cameras has led to a significant increase in capturing natural environments and their inhabitants. Video-based surveys, especially in underwater areas, are valuable as human observation can be expensive, dangerous, inaccessible, or damaging to the environment. This study tests the use of a Mask R-CNN object detection framework for automated fish localization, classification, counting, and tracking. The model performs accurately on both training and validation datasets and even on previously unseen footage.