PPCU Sam: Open-source face recognition framework

In recent years by the popularization of AI, an increasing number of enterprises deployed machine learning algorithms in real life settings. This trend shed light on leaking spots of the Deep Learning bubble, namely the catastrophic decrease in quality when the distribution of the test data shifts from the training data. It is of utmost importance that we treat the promises of novel algorithms with caution and discourage reporting near perfect experimental results by fine-tuning on fixed test sets and finding metrics that hide weak points of the proposed methods. To support the wider acceptance of computer vision solutions we share our findings through a case-study in which we built a face-recognition system from scratch using consumer grade devices only, collected a database of 100k images from 150 subjects and carried out extensive validation of the most prominent approaches in single-frame face recognition literature. We show that the reported worst-case score, 74.3% true-positive ratio drops below 46.8% on real data. To overcome this barrier, after careful error analysis of the single-frame baselines we propose a low complexity solution to cover the failure cases of the single-frame recognition methods which yields an increased stability in multi-frame recognition during test time. We validate the effectiveness of the proposal by an extensive survey among our users which evaluates to 89.5% true-positive ratio. (C) 2019 The Authors. Published by Elsevier B.V.