4.7 Article

Differentially Private Deep Learning With Dynamic Privacy Budget Allocation and Adaptive Optimization

Journal

Publisher

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TIFS.2023.3293961

Keywords

Differential privacy; deep learning; layer-wise relevance; adaptive optimization

Ask authors/readers for more resources

In this paper, we propose a method that combines layer-wise relevance propagation with gradient descent to address limitations in deep learning related to data and user privacy. The method injects proper noise into gradients to improve model utility, and uses the NoisyMin algorithm to select the best step size for each gradient perturbation. Experimental evaluations validate the effectiveness of the proposed algorithm and its ability to protect privacy.
Deep learning (DL) has been adopted in a broad range of Internet-of-Things (IoT) applications such as auto-driving, intelligent healthcare and smart grids, but limitations such as those relating to data and user privacy can complicate its broader implementation. Seeking to jointly address both privacy and utility, in this paper we connect the layer-wise relevance propagation with gradient descent for injecting proper noise into gradients. We also improve the conventional gradient clipping method by dividing the gradients into several groups; thus, minimizing the gradient distortion. Since the noisy gradient causes the undetermined descent direction and might adversely affect the loss minimization, we use the NoisyMin algorithm to select the best step size for each gradient perturbation. Finally, we integrate the adaptive optimizer into the gradient descent. In addition to improving the model utility, we also leverage the leading Sinh-Normal noise addition mechanism to achieve truncated concentrated differential privacy (tCDP) - as demonstrated by our rigorous analysis. Our experimental evaluations also validate the effectiveness of the proposed algorithm.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.7
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available