SoK: Machine vs. machine - A systematic classification of automated machine learning-based CAPTCHA solvers

Internet services heavily rely on CAPTCHAs for determining whether or not a user is a human being. The recent advances in ML and AI make the efficacy of CAPTCHAs in strengthening Internet services against bots questionable. In this paper, we conduct a systematic analysis and classification of the state-of-theart ML-based techniques for the automated text-based CAPTCHA breaking problem. The current state and robustness of text-based CAPTCHAs as are utilized by modern Internet applications, against ML-based automated breaking tools, is examined and reported. Our study suggests that ML can be very effective in increasing: (a) accuracy, (b) speed, and (c) abstraction in CAPTCHA solving. Especially, as far as (c) is concerned, ML-based techniques are easier to be applied in different classes of text-based CAPTCHA schemes. To assess the importance of ML in breaking CAPTCHAs, we build our own ML-only classifiers. Surprisingly, an ML-only approach for solving CAPTCHAs is not sufficient. Overall, our study suggests that fundamentally different ways of conducting reverse Turing test, that will be painless for legitimate users (i.e., humans) but at the same time challenging for automated systems (i.e., software), should be considered for ensuring the healthy operation of current Internet services. (c) 2020 The Authors. Published by Elsevier Ltd.