期刊
JOURNAL OF SYSTEMS AND SOFTWARE
卷 181, 期 -, 页码 -出版社
ELSEVIER SCIENCE INC
DOI: 10.1016/j.jss.2021.111040
关键词
Learning techniques; Text localization; Stability; Metamorphic testing
资金
- Key Research Program of Frontier Sciences, CAS, China [QYZDJ-SSW-JSC036]
- CAS-INRIA major project, China [171311KYS B20170027]
- Guangdong Science and Technology Department, China [2018B010107004]
The paper proposes a methodology to automatically evaluate the stability of text localization systems using metamorphic relations, with six metamorphic relations defined for stable systems and corresponding metrics for evaluation. By applying metamorphic testing techniques based on the defined relations, system stability can be evaluated and causes of inconsistency diagnosed effectively. Experimental results on academic and commercial text localization systems demonstrate the effectiveness of the proposed method in stability evaluation.
The success of learning techniques in solving a variety of hard AI problems promotes the flourish of recognition-based applications. Many state-of-the-art text localization systems, which can detect and report the positions of text segments in an image, are mainly implemented with learning-based techniques. Data-driven learning raises a series of questions on how to verify, validate and evaluate such learning-based systems. In this paper, we propose a methodology to automatically evaluate the stability of text localization systems via metamorphic relations, where a stable system should output consistent results for similar inputs with the same text segments. We introduce six metamorphic relations that should be preserved in a stable text localization system and define the corresponding metrics for stability evaluation. With the defined metamorphic relations, we apply metamorphic testing techniques to compare the inputs and outputs to evaluate system stability, and further diagnose the causes of inconsistency. The extensive experimentation on both academic and commercial text localization systems demonstrates the effectiveness of our method on stability evaluation for such systems. (C) 2021 Elsevier Inc. All rights reserved.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据