The classification conundrum: species fidelity as leading criterion in search of a rigorous method to classify a complex forest data set

We present a test involving a large number of data-analytical techniques to identify a rigorous numerical classification method optimising on statistically identified faithful species. The test follows a stepwise filtering process involving various numerical-classification tools. Five steps were involved in the testing: (1) evaluation of 322 classification tools using OptimClass 1; (2) comparison of 20 best performing methods by standardising the various performances across a range of fidelity values using OptimClass 1 and OptimClass 2, to assess the effectiveness of the agglomerative clustering and one divisive technique; (3) calculation and comparison of Uniqueness values and ISAMIC (Indicator Species Analysis Minimising Intermediate Constancies) scores of the resulting classifications; (4) comparison of different classifications by analysing the similarities of the resulting synoptic tables using faithful species, assuming that clusters with similar faithful species represent corresponding vegetation types, and (5) final selection of the single best method based on an expert review of non-geometric internal evaluators, NMDS ordinations and mapped classification solutions. A complex data set, representing many forest vegetation types and consisting of 506 releves of 20 m x 20 m sampled in the indigenous forests of Mpumalanga Province (South Africa), was tested. Analysis of Uniqueness provided insight into which methods produced classifications that did not share faithful species. The analysis of synoptic table similarity showed that the classification results were at most 88% similar, while in the most divergent case similarity of only 50% was achieved. OptimClass eliminated poorly performing numerical-classification combinations and highlighted the best performing methods. Yet it was unable to reveal the single best performing method unequivocally across the range of fidelity values used. In such cases, we suggest the solution can be sought in relying on involving external data through expert opinion. Ordinal Clustering and TWINSPAN produced the most outlying classification results. Flexible beta clustering (beta = -0.25) in combination with Bray-Curtis coefficient, standardised by sample unit totals, produced the most informative result for our data set when using informal expert-defined ecological and biogeographical judgement criteria. We recommend that the performance of a set of methods be tested prior to selecting the final classification approach.