4.6 Article

Machine Learning Testing: Survey, Landscapes and Horizons

Related references

Note: Only part of the references are listed.
Article Computer Science, Software Engineering

Metamorphic Relations for Enhancing System Understanding and Use

Zhi Quan Zhou et al.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2020)

Article Computer Science, Artificial Intelligence

Explanation in artificial intelligence: Insights from the social sciences

Tim Miller

ARTIFICIAL INTELLIGENCE (2019)

Article Computer Science, Hardware & Architecture

Metamorphic Testing of Driverless Cars

Zhi Quan Zhou et al.

COMMUNICATIONS OF THE ACM (2019)

Article Multidisciplinary Sciences

Preventing undesirable behavior of intelligent machines

Philip S. Thomas et al.

SCIENCE (2019)

Proceedings Paper Computer Science, Theory & Methods

Fairness-Aware Programming

Aws Albarghouthi et al.

FAT*'19: PROCEEDINGS OF THE 2019 CONFERENCE ON FAIRNESS, ACCOUNTABILITY, AND TRANSPARENCY (2019)

Proceedings Paper Computer Science, Software Engineering

Testing Untestable Neural Machine Translation: An Industrial Case

Wujie Zheng et al.

2019 IEEE/ACM 41ST INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: COMPANION PROCEEDINGS (ICSE-COMPANION 2019) (2019)

Proceedings Paper Computer Science, Artificial Intelligence

Input Prioritization for Testing Neural Networks

Taejoon Byun et al.

2019 IEEE INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE TESTING (AITEST) (2019)

Proceedings Paper Computer Science, Artificial Intelligence

Adversarial Sample Detection for Deep Neural Network through Model Mutation Testing

Jingyi Wang et al.

2019 IEEE/ACM 41ST INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING (ICSE 2019) (2019)

Proceedings Paper Computer Science, Software Engineering

Towards Improved Testing For Deep Learning

Jasmine Sekhon et al.

2019 IEEE/ACM 41ST INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: NEW IDEAS AND EMERGING RESULTS (ICSE-NIER 2019) (2019)

Proceedings Paper Computer Science, Artificial Intelligence

Detecting Failures of Neural Machine Translation in the Absence of Reference Translations

Wenyu Wang et al.

49TH ANNUAL IEEE/IFIP INTERNATIONAL CONFERENCE ON DEPENDABLE SYSTEMS AND NETWORKS (DSN 2019): INDUSTRY TRACK (2019)

Proceedings Paper Computer Science, Software Engineering

DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems

Lei Ma et al.

2019 IEEE 26TH INTERNATIONAL CONFERENCE ON SOFTWARE ANALYSIS, EVOLUTION AND REENGINEERING (SANER) (2019)

Proceedings Paper Computer Science, Software Engineering

Boosting Operational DNN Testing Efficiency through Conditioning

Zenan Li et al.

ESEC/FSE'2019: PROCEEDINGS OF THE 2019 27TH ACM JOINT MEETING ON EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING (2019)

Proceedings Paper Computer Science, Software Engineering

Storm: Program Reduction for Testing and Debugging Probabilistic Programming Systems

Saikat Dutta et al.

ESEC/FSE'2019: PROCEEDINGS OF THE 2019 27TH ACM JOINT MEETING ON EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING (2019)

Proceedings Paper Computer Science, Software Engineering

Software Engineering for Machine Learning: A Case Study

Saleema Amershi et al.

2019 IEEE/ACM 41ST INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: SOFTWARE ENGINEERING IN PRACTICE (ICSE-SEIP 2019) (2019)

Proceedings Paper Computer Science, Information Systems

DeepBase: Deep Inspection of Neural Networks

Thibault Sellam et al.

SIGMOD '19: PROCEEDINGS OF THE 2019 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA (2019)

Proceedings Paper Engineering, Electrical & Electronic

Do Pseudo Test Suites Lead to Inflated Correlation in Measuring Test Effectiveness?

Jie M. Zhang et al.

2019 IEEE 12TH CONFERENCE ON SOFTWARE TESTING, VALIDATION AND VERIFICATION (ICST 2019) (2019)

Proceedings Paper Engineering, Electrical & Electronic

Testing Machine Learning Algorithms for Balanced Data Usage

Arnab Sharma et al.

2019 IEEE 12TH CONFERENCE ON SOFTWARE TESTING, VALIDATION AND VERIFICATION (ICST 2019) (2019)

Article Computer Science, Software Engineering

code2vec: Learning Distributed Representations of Code

Uri Alon et al.

PROCEEDINGS OF THE ACM ON PROGRAMMING LANGUAGES-PACMPL (2019)

Article Health Care Sciences & Services

Calibration of medical diagnostic classifier scores to the probability of disease

Weijie Chen et al.

STATISTICAL METHODS IN MEDICAL RESEARCH (2018)

Article Computer Science, Information Systems

Enabling Adaptability in Web Forms Based on User Characteristics Detection Through A/B Testing and Machine Learning

Juan Cruz-Benito et al.

IEEE ACCESS (2018)

Article Computer Science, Software Engineering

Adaptation of General Concepts of Software Testing to Neural Networks

Yu L. Karpov et al.

PROGRAMMING AND COMPUTER SOFTWARE (2018)

Proceedings Paper Computer Science, Software Engineering

MuNN: Mutation Analysis of Neural Networks

Weijun Shen et al.

2018 IEEE 18TH INTERNATIONAL CONFERENCE ON SOFTWARE QUALITY, RELIABILITY AND SECURITY COMPANION (QRS-C) (2018)

Proceedings Paper Computer Science, Interdisciplinary Applications

A Monte Carlo Method for Metamorphic Testing of Machine Translation Services

Daniel Pesu et al.

2018 IEEE/ACM 3RD INTERNATIONAL WORKSHOP ON METAMORPHIC TESTING (MET 2018) (2018)

Proceedings Paper Computer Science, Software Engineering

A Survey of Software Quality for Machine Learning Applications

Satoshi Masuda et al.

2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW) (2018)

Proceedings Paper Computer Science, Software Engineering

A Test Architecture for Machine Learning Product

Yasuharu Nishi et al.

2018 IEEE 11TH INTERNATIONAL CONFERENCE ON SOFTWARE TESTING, VERIFICATION AND VALIDATION WORKSHOPS (ICSTW) (2018)

Proceedings Paper Computer Science, Theory & Methods

Concepts in Quality Assessment for Machine Learning - From Test Data to Arguments

Fuyuki Ishikawa

CONCEPTUAL MODELING, ER 2018 (2018)

Article Robotics

Failing to Learn: Autonomously Identifying Perception Failures for Self-Driving Cars

Manikandasriram Srinivasan Ramanagopal et al.

IEEE ROBOTICS AND AUTOMATION LETTERS (2018)

Proceedings Paper Computer Science, Software Engineering

DeepMutation: Mutation Testing of Deep Learning Systems

Lei Ma et al.

2018 29TH IEEE INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING (ISSRE) (2018)

Proceedings Paper Computer Science, Software Engineering

DLFuzz: Differential Fuzzing Testing of Deep Learning Systems

Jianmin Guo et al.

ESEC/FSE'18: PROCEEDINGS OF THE 2018 26TH ACM JOINT MEETING ON EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING (2018)

Proceedings Paper Computer Science, Information Systems

MISTIQUE: A System to Store and Query Model Intermediates for Model Diagnosis

Manasi Vartak et al.

SIGMOD'18: PROCEEDINGS OF THE 2018 INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA (2018)

Proceedings Paper Computer Science, Software Engineering

Metamorphic Testing for Machine Translations: MT4MT

Liqun Sun et al.

2018 25TH AUSTRALASIAN SOFTWARE ENGINEERING CONFERENCE (ASWEC) (2018)

Article Computer Science, Theory & Methods

Fruit recognition from images using deep learning

Horea Muresan et al.

ACTA UNIVERSITATIS SAPIENTIAE INFORMATICA (2018)

Proceedings Paper Optics

Test data reuse for evaluation of adaptive machine learning algorithms: Overfitting to a fixed test dataset and a potential solution

Alexej Gossmann et al.

MEDICAL IMAGING 2018: IMAGE PERCEPTION, OBSERVER PERFORMANCE, AND TECHNOLOGY ASSESSMENT (2018)

Article Computer Science, Information Systems

Automating Large-Scale Data Quality Verification

Sebastian Schelter et al.

PROCEEDINGS OF THE VLDB ENDOWMENT (2018)

Article Computer Science, Artificial Intelligence

A survey on deep learning in medical image analysis

Geert Litjens et al.

MEDICAL IMAGE ANALYSIS (2017)

Proceedings Paper Computer Science, Software Engineering

Fairness Testing: Testing Software for Discrimination

Sainyam Galhotra et al.

ESEC/FSE 2017: PROCEEDINGS OF THE 2017 11TH JOINT MEETING ON FOUNDATIONS OF SOFTWARE ENGINEERING (2017)

Proceedings Paper Computer Science, Theory & Methods

Repairing Decision-Making Programs Under Uncertainty

Aws Albarghouthi et al.

COMPUTER AIDED VERIFICATION, CAV 2017, PT I (2017)

Proceedings Paper Computer Science, Software Engineering

An Empirical Study on Real Bugs for Machine Learning Programs

Xiaobing Sun et al.

2017 24TH ASIA-PACIFIC SOFTWARE ENGINEERING CONFERENCE (APSEC 2017) (2017)

Proceedings Paper Computer Science, Information Systems

FairTest: Discovering Unwarranted Associations in Data-Driven Applications

Florian Tramer et al.

2017 IEEE EUROPEAN SYMPOSIUM ON SECURITY AND PRIVACY (EUROS&P) (2017)

Proceedings Paper Computer Science, Information Systems

Data Management Challenges in Production Machine Learning

Neoklis Polyzotis et al.

SIGMOD'17: PROCEEDINGS OF THE 2017 ACM INTERNATIONAL CONFERENCE ON MANAGEMENT OF DATA (2017)

Proceedings Paper Computer Science, Software Engineering

Easy over Hard: A Case Study on Deep Learning

Wei Fu et al.

ESEC/FSE 2017: PROCEEDINGS OF THE 2017 11TH JOINT MEETING ON FOUNDATIONS OF SOFTWARE ENGINEERING (2017)

Proceedings Paper Computer Science, Theory & Methods

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

Guy Katz et al.

COMPUTER AIDED VERIFICATION, CAV 2017, PT I (2017)

Proceedings Paper Computer Science, Information Systems

Towards Evaluating the Robustness of Neural Networks

Nicholas Carlini et al.

2017 IEEE SYMPOSIUM ON SECURITY AND PRIVACY (SP) (2017)

Proceedings Paper Computer Science, Artificial Intelligence

TFX: A TensorFlow-Based Production-Scale Machine Learning Platform

Denis Baylor et al.

KDD'17: PROCEEDINGS OF THE 23RD ACM SIGKDD INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND DATA MINING (2017)

Article Computer Science, Artificial Intelligence

The MovieLens Datasets: History and Context

F. Maxwell Harper et al.

ACM TRANSACTIONS ON INTERACTIVE INTELLIGENT SYSTEMS (2016)

Article Social Sciences, Interdisciplinary

How the machine 'thinks': Understanding opacity in machine learning algorithms

Jenna Burrell

BIG DATA & SOCIETY (2016)

Proceedings Paper Computer Science, Interdisciplinary Applications

A Framework for Ensuring the Quality of a Big Data Service

Junhua Ding et al.

PROCEEDINGS 2016 IEEE INTERNATIONAL CONFERENCE ON SERVICES COMPUTING (SCC 2016) (2016)

Article Computer Science, Software Engineering

The Oracle Problem in Software Testing: A Survey

Earl T. Barr et al.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2015)

Review Multidisciplinary Sciences

Deep learning

Yann LeCun et al.

NATURE (2015)

Review Multidisciplinary Sciences

Machine learning: Trends, perspectives, and prospects

M. I. Jordan et al.

SCIENCE (2015)

Proceedings Paper Computer Science, Artificial Intelligence

Calibrating Probability with Undersampling for Unbalanced Classification

Andrea Dal Pozzolo et al.

2015 IEEE SYMPOSIUM SERIES ON COMPUTATIONAL INTELLIGENCE (IEEE SSCI) (2015)

Proceedings Paper Computer Science, Artificial Intelligence

DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving

Chenyi Chen et al.

2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV) (2015)

Article Computer Science, Software Engineering

Compiler Validation via Equivalence Modulo Inputs

Vu Le et al.

ACM SIGPLAN NOTICES (2014)

Article Computer Science, Artificial Intelligence

A data-driven approach to predict the success of bank telemarketing

Sergio Moro et al.

DECISION SUPPORT SYSTEMS (2014)

Proceedings Paper Computer Science, Information Systems

Drebin: Effective and Explainable Detection of Android Malware in Your Pocket

Daniel Arp et al.

21ST ANNUAL NETWORK AND DISTRIBUTED SYSTEM SECURITY SYMPOSIUM (NDSS 2014) (2014)

Proceedings Paper Computer Science, Software Engineering

An Analysis of the Relationship between Conditional Entropy and Failed Error Propagation in Software Testing

Kelly Androutsopoulos et al.

36TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING (ICSE 2014) (2014)

Article Health Care Sciences & Services

On the assessment of the added value of new predictive biomarkers

Weijie Chen et al.

BMC MEDICAL RESEARCH METHODOLOGY (2013)

Article Computer Science, Theory & Methods

State of the art: Dynamic symbolic execution for automated test generation

Ting Chen et al.

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE (2013)

Review Computer Science, Information Systems

A systematic review of software robustness

Ali Shahrokni et al.

INFORMATION AND SOFTWARE TECHNOLOGY (2013)

Article Robotics

Vision meets robotics: The KITTI dataset

A. Geiger et al.

INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH (2013)

Article Computer Science, Information Systems

Squeeziness: An information theoretic measure for avoiding fault masking

David Clark et al.

INFORMATION PROCESSING LETTERS (2012)

Article Computer Science, Artificial Intelligence

Classifier variability: Accounting for training and testing

Weijie Chen et al.

PATTERN RECOGNITION (2012)

Article Computer Science, Software Engineering

An Analysis and Survey of the Development of Mutation Testing

Yue Jia et al.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2011)

Article Computer Science, Software Engineering

Testing and validating machine learning classifiers by metamorphic testing

Xiaoyuan Xie et al.

JOURNAL OF SYSTEMS AND SOFTWARE (2011)

Article Computer Science, Software Engineering

A Theoretical and Empirical Study of Search-Based Testing: Local, Global, and Hybrid Search

Mark Harman et al.

IEEE TRANSACTIONS ON SOFTWARE ENGINEERING (2010)

Article Computer Science, Information Systems

A systematic review of search-based testing for non-functional system properties

Wasif Afzal et al.

INFORMATION AND SOFTWARE TECHNOLOGY (2009)

Article Computer Science, Information Systems

A search based approach to fairness analysis in requirement assignments to aid negotiation, mediation and decision making

Anthony Finkelstein et al.

REQUIREMENTS ENGINEERING (2009)

Article Computer Science, Software Engineering

MuJava: an automated class mutation system

YS Ma et al.

SOFTWARE TESTING VERIFICATION & RELIABILITY (2005)

Article Chemistry, Multidisciplinary

The problem of overfitting

DM Hawkins

JOURNAL OF CHEMICAL INFORMATION AND COMPUTER SCIENCES (2004)

Article Computer Science, Software Engineering

Search-based software test data generation: a survey

P McMinn

SOFTWARE TESTING VERIFICATION & RELIABILITY (2004)

Article Computer Science, Information Systems

Search-based software engineering

M Harman et al.

INFORMATION AND SOFTWARE TECHNOLOGY (2001)

Article Radiology, Nuclear Medicine & Medical Imaging

Feature selection and classifier performance in computer-aided diagnosis: The effect of finite sample size

B Sahiner et al.

MEDICAL PHYSICS (2000)