Journal
IEEE CONTROL SYSTEMS LETTERS
Volume 6, Issue -, Pages 121-126Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/LCSYS.2021.3050444
Keywords
Artificial neural networks; Training; Robustness; Estimation; Upper bound; Perturbation methods; Neurons; Linear matrix inequalities; neural networks; robustness
Categories
Funding
- Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) [2075 -390740016]
- International Max Planck Research School for Intelligent Systems (IMPRS-IS)
Ask authors/readers for more resources
In this study, a framework for training multi-layer neural networks with increased robustness is proposed. By minimizing the Lipschitz constant and using a semidefinite programming based training procedure, the framework successfully enhances the robustness of neural networks. Two examples are provided to demonstrate the effectiveness of the proposed framework.
Due to their susceptibility to adversarial perturbations, neural networks (NNs) are hardly used in safety-critical applications. One measure of robustness to such perturbations in the input is the Lipschitz constant of the input-output map defined by an NN. In this letter, we propose a framework to train multi-layer NNs while at the same time encouraging robustness by keeping their Lipschitz constant small, thus addressing the robustness issue. More specifically, we design an optimization scheme based on the Alternating Direction Method of Multipliers that minimizes not only the training loss of an NN but also its Lipschitz constant resulting in a semidefinite programming based training procedure that promotes robustness. We design two versions of this training procedure. The first one includes a regularizer that penalizes an accurate upper bound on the Lipschitz constant. The second one allows to enforce a desired Lipschitz bound on the NN at all times during training. Finally, we provide two examples to show that the proposed framework successfully increases the robustness of NNs.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available