A Survey of Field-based Testing Techniques

Field testing refers to testing techniques that operate in the field to reveal those faults that escape in-house testing. Field testing techniques are becoming increasingly popular with the growing complexity of contemporary software systems. In this article, we present the first systematic survey of field testing approaches over a body of 80 collected studies, and propose their categorization based on the environment and the system on which field testing is performed. We discuss four research questions addressing how software is tested in the field, what is tested in the field, which are the requirements, and how field tests are managed, and identify many challenging research directions.

Automated summary

Field testing refers to testing techniques that uncover faults missed during in-house testing by operating in the field. It is becoming more popular as software systems become more complex. By categorizing field testing approaches based on environment and system, and addressing research questions related to field testing requirements and management, this article highlights challenging research directions in field testing.