Do experts make mistakes? A comparison of human and machine identification of dinoflagellates

The authors present evidence of the difficulties facing human taxonomists/ecologists in identifying marine dinoflagellates. This is especially important for work on harmful algal blooms in marine aquaculture. It is shown that it is difficult for people to categorise specimens from species with significant morphological variation, perhaps with morphologies overlapping with those of other species. Trained personnel can be expected to achieve 67 to 83% self-consistency and 43% consensus between people in an expert taxonomic labelling task. Experts who are routinely engaged in particular discriminations can return accuracies in the range of 84 to 95%. In general, neither human nor machine can be expected to give highly accurate or repeatable labelling of specimens. It is also shown that automation methods can perform as well as humans on these complex categorisations.