4.2 Article

Comparison of image annotation data generated by multiple investigators for benthic ecology

Journal

MARINE ECOLOGY PROGRESS SERIES
Volume 552, Issue -, Pages 61-70

Publisher

INTER-RESEARCH
DOI: 10.3354/meps11775

Keywords

Expert knowledge; Scoring; Visual imaging; Multiple investigators; Data quality; Quality wassurance/quality control

Funding

  1. UK Natural Environment Research Council (NERC) [NERC NE/H021787/1]
  2. NERC Marine Environmental Mapping Programme (MAREMAP)
  3. Porcupine Abyssal Plain - Sustained Observatory (NERC National Capability programme)
  4. Bundesministerium fur Wirtschaft und Energie (BMWI) as part of the IS2U project [FKZ 03SX344A]
  5. NERC [NE/H021787/1, noc010009] Funding Source: UKRI
  6. Natural Environment Research Council [noc010009, NE/H021787/1] Funding Source: researchfish

Ask authors/readers for more resources

Multiple investigators often generate data from seabed images within a single image set to reduce the time burden, particularly with the large photographic surveys now available to ecological studies. These data (annotations) are known to vary as a result of differences in investigator opinion on specimen classification and of human factors such as fatigue and cognition. These variations are rarely recorded or quantified, nor are their impacts on derived ecological metrics (density, diversity, composition). We compared the annotations of 3 investigators of 73 megafaunal morphotypes in similar to 28 000 images, including 650 common images. Successful annotation was defined as both detecting and correctly classifying a specimen. Estimated specimen detection success was 77%, and classification success was 95%, giving an annotation success rate of 73%. Specimen detection success varied substantially by morphotype (12-100%). Variation in the detection of common taxa resulted in significant differences in apparent faunal density and community composition among investigators. Such bias has the potential to produce spurious ecological interpretations if not appropriately controlled or accounted for. We recommend that photographic studies document the use of multiple annotators and quantify potential inter-investigator bias. Randomisation of the sampling unit (photograph or video clip) is clearly critical to the effective removal of human annotation bias in multiple annotator studies (and indeed single annotator works).

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.2
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available