Evaluation of Nuance Forensics 9.2 and 11.1 under conditions reflecting those of a real forensic voice comparison case (forensic_eval_01)

Two automatic speaker recognition systems, Nuance Forensics 9.2 and 11.1, were tested within the setting of the Speech Communication virtual special issue Multi-laboratory evaluation of forensic voice comparison systems under conditions reflecting those of a real forensic case (forensic_eval_01). Nuance Forensics 9.2 is an i-vector PLDA system and Nuance Forensics 11.1 combines i-vector PLDA technology with some Deep Neural Networks functionalities. Both systems were tested in three variants. The difference between the first and second variant lies in the size of the Reference Population(42 vs. 105 speakers) and the difference between the first two and the third variant lies in the use of the Background Model, either working with a system default (first two variants) or a dedicated model drawn from the forensic_eval_01 training data (third variant). The Reference Population is used for the purpose of calibration (arriving at calibrated likelihood ratios from voice comparison scores); the Background Model is used for normalising the scores (Adaptive S-norm). Comparing the three variants, it was shown across the two systems that the inclusion of a Background Model that is dedicated to the conditions of the case leads to improved performance over the use of a system default. The difference in the size of the Reference Population however did not matter. Comparing the two systems, it was found that the system that includes Deep Neural Network technology leads to improved results over the use of a pure i-vector PLDA system.