.png){kind=icon}
返回
AG-SGD: Angle-Based Stochastic Gradient Descent
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2021.3055993.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In the field of neural network, stochastic gradient descent is often employed as an effective method of accelerating the result's convergence. Generating the new gradient from the past gradient is a common method adopted by many existing optimization algorithms. Since the past gradient is not computed based on the most updated stochastic gradient descent state, it can introduce a deviation to the new gradient computation, negatively impacting its rate of convergence. To resolve this problem, we propose an algorithm that quantifies this deviation based on the angle between the past and the current gradients, which is then applied to calibrate these two gradients, generating a more accurate new gradient. To demonstrate the broad applicability of the algorithm, the proposed method is implemented into a neural network and a logistic regression classifier which are evaluated on the datasets MNIST and NSL-KDD, respectively. An in-depth analysis is performed to compare our algorithm with nine optimization algorithms in two experiments, demonstrating the advantages in the cost and the error rate reductions from adopting the proposed method.
Keyword:
Neural networks
Optimization
Classification algorithms
Trajectory
Rendering (computer graphics)
Reliability
Object recognition
Stochastic gradient descent
optimization algorithm
gradient deviation
neural network
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
A Benchmark Dataset and Learning High-Level Semantic Embeddings of Multimedia for Cross-media Retrieval.png)
IEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏分享收藏 没有更多内容
没有更多内容