Machine learning to detect marine animals in UAV imagery: effect of morphology, spacing, behaviour and habitat

Machine learning algorithms are being increasingly used to process large volumes of wildlife imagery data from unmanned aerial vehicles (UAVs); however, suitable algorithms to monitor multiple species are required to enhance efficiency. Here, we developed a machine learning algorithm using a low-cost computer. We trained a convolutional neural network and tested its performance in: (1) distinguishing focal organisms of three marine taxa (Australian fur seals, loggerhead sea turtles and Australasian gannets; body size ranges: 0.8-2.5 m, 0.6-1.0 m, and 0.8-0.9 m, respectively); and (2) simultaneously delineating the fine-scale movement trajectories of multiple sea turtles at a fish cleaning station. For all species, the algorithm performed best at detecting individuals of similar body length, displaying consistent behaviour or occupying uniform habitat (proportion of individuals detected, or recall of 0.94, 0.79 and 0.75 for gannets, seals and turtles, respectively). For gannets, performance was impacted by spacing (huddling pairs with offspring) and behaviour (resting vs. flying shapes, overall precision: 0.74). For seals, accuracy was impacted by morphology (sexual dimorphism and pups), spacing (huddling and creches) and habitat complexity (seal sized boulders) (overall precision: 0.27). For sea turtles, performance was impacted by habitat complexity, position in water column, spacing, behaviour (interacting individuals) and turbidity (overall precision: 0.24); body size variation had no impact. For sea turtle trajectories, locations were estimated with a relative positioning error of <50 cm. In conclusion, we demonstrate that, while the same machine learning algorithm can be used to survey multiple species, no single algorithm captures all components optimally within a given site. We recommend that, rather than attempting to fully automate detection of UAV imagery data, semi-automation is implemented (i.e. part automated and part manual, as commonly practised for photo-identification). Approaches to enhance the efficiency of manual detection are required in parallel to the development of effective implementation of machine learning algorithms.

Automated summary

Machine learning algorithms are increasingly used to process wildlife imagery data from UAVs, but the efficiency can be enhanced by developing suitable algorithms for monitoring multiple species. A low-cost computer was used here to train a convolutional neural network for distinguishing marine organisms and delineating sea turtle trajectories. The algorithm performed best when detecting individuals of similar body length, displaying consistent behavior, or occupying uniform habitat. Accuracy was impacted by factors such as morphology, behavior, spacing, and habitat complexity for different species.