Advanced image recognition: a fully automated, high-accuracy photo-identification matching system for humpback whales

We describe the development and application of a new convolutional neural network-based photo-identification algorithm for individual humpback whales (Megaptera novaeangliae). The method uses a Densely Connected Convolutional Network (DenseNet) to extract special keypoints of an image of the ventral surface of the fluke and then a separate DenseNet trained to look for features within these keypoints. The extracted features are then compared against those of the reference set of previously known humpback whales for similarity. This offers the potential to successfully automate recognition of individuals in large photographic datasets such as in ocean basin-wide marine mammal studies. The algorithm requires minimal image pre-processing and is capable of accurate, rapid matching of fair to high-quality humpback fluke photographs. In real world testing compared to manual image matching, the algorithm reduces image management time by at least 98% and reduces error rates of missing potential matches from approximately 6-9% to 1-3%. The success of this new system permits automated comparisons to be made for the first time across photo-identification datasets with tens to hundreds of thousands of individually identified encounters, with profound implications for long-term and large population studies of the species.

Automated summary

This new convolutional neural network-based photo-identification algorithm is able to extract features and automatically recognize individuals in large humpback whale photo datasets, significantly reducing time and error rates compared to manual matching.