.png){kind=icon}
返回
Riemannian Adaptive Optimization Algorithm and Its Application to Natural Language Processing
.png){kind=icon}
OA
AI
DOI：10.1109/TCYB.2021.3049845.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Y This article proposes a Riemannian adaptive optimization algorithm to optimize the parameters of deep neural networks. The algorithm is an extension of both AMSGrad in Euclidean space and RAMSGrad on a Riemannian manifold. The algorithm helps to resolve two issues affecting RAMSGrad. The first is that it can solve the Riemannian stochastic optimization problem directly, in contrast to RAMSGrad which only achieves a low regret. The other is that it can use constant learning rates, which makes it implementable in practice. Additionally, we apply the proposed algorithm to Poincare embeddings that embed the transitive closure of the WordNet nouns into the Poincare ball model of hyperbolic space. Numerical experiments show that regardless of the initial value of the learning rate, our algorithm stably converges to the optimal solution and converges faster than the existing algorithms.
Keyword:
Hyperbolic space
natural language processing
Poincare embeddings
RAdaGrad
RAdam
RAMSGrad
Riemannian adaptive optimization algorithm
Riemannian optimization
Riemannian stochastic gradient descent (RSGD)
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
Consensus AP-1 and CRE motifs upstream from the human cytotoxic serine protease B (CSP-B/CGL-1) gene synergize to activate transcription.png)
BloodOAAI
BloodIF0
OAAI分享收藏分享收藏
分享收藏分享收藏分享收藏 没有更多内容
没有更多内容