.png){kind=icon}
返回
An improved stochastic gradient descent algorithm based on Renyi differential privacy
.png){kind=icon}
OA
AI
DOI：10.1002/int.22944.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep learning techniques based on the neural network have made significant achievements in various fields of artificial intelligence. However, model training requires large-scale data sets, these data sets are crowd-sourced and model parameters will contain the encoding of private information, resulting in the risk of privacy leakage. With the trend toward sharing pretrained models, the risk of stealing training data sets through member inference attacks and model inversion attacks is further heightened. To tackle the privacy-preserving problems in deep learning tasks, we propose an improved Differential Privacy Stochastic Gradient Descent algorithm, using Simulated Annealing algorithm and Laplace Smooth denoising mechanism to optimize the allocation method of privacy loss, replacing the constant clipping method with adaptive gradient clipping method to improve model accuracy. we also analyze privacy cost under random shuffle data batch processing method in detail within the framework of Subsampled Renyi Differential Privacy. Compared with the existing privacy protection training methods with fixed parameters and dynamic privacy parameters in classification tasks, our implementation and experiments show that we can use less privacy budget train deep neural networks with the nonconvex objective function, obtain a higher model evaluation, and have almost zero additional cost in terms of model complexity, training efficiency, and model quality.
Keyword:
deep learning
differential privacy
nonconvex optimization
simulated annealing
stochastic gradient descent
期刊
IF:
3.7
论文数:
3.1K
被引数:
8.1K
机构
引用论文
Modeling and Simulation of Main Control Valve Mechanism of LNG Ship Steam TurbineLNG船舶汽轮机主控阀机构建模与仿真OAAI
OAAI分享收藏Glycan OH Exchange Rate Determination in Aqueous Solution: Seeking Evidence for Transient Hydrogen Bonds水溶液中聚糖OH交换速率的测定: 寻找瞬态氢键的证据
PREAI
分享收藏Gradient-based learning applied to document recognition基于梯度的学习在文档识别中的应用.png)
PROCEEDINGS OF THE IEEE
PROCEEDINGS OF THE IEEEIF25.9
PREAI
分享收藏The effect of high intensity interval training on flexibility and anaerobic power<p>Yüksek yoğunluklu interval antrenmanın esneklik ve anaerobik kapasite üzerine etkisi高强度间歇训练对柔韧性和无氧功率的影响 y ü ksekyo ğ unluklu间隔时间和厌氧菌kapasite ü zerine etkisiOAAI
OAAI分享收藏Catalytic activity of dihydride ruthenium complexes in the hydrogenation of nitrogen containing heterocycles二氢化物钌配合物在含氮杂环氢化中的催化活性
PREAI
分享收藏
查看更多内容