.png){kind=icon}
返回
EFFICIENT NATURAL GRADIENT DESCENT METHODS FOR LARGE-SCALE PDE-BASED OPTIMIZATION PROBLEMS
.png){kind=icon}
OA
AI
DOI：10.1137/22M1477805.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
We propose efficient numerical schemes for implementing the natural gradient descent (NGD) for a broad range of metric spaces with applications to PDE-based optimization problems. Our technique represents the natural gradient direction as a solution to a standard least-squares problem. Hence, instead of calculating, storing, or inverting the information matrix directly, we apply efficient methods from numerical linear algebra. We treat both scenarios where the Jacobian, i.e., the derivative of the state variable with respect to the parameter, is either explicitly known or implicitly given through constraints. We can thus reliably compute several natural NGDs for a large-scale parameter space. In particular, we are able to compute Wasserstein NGD in thousands of dimensions, which was believed to be out of reach. Finally, our numerical results shed light on the qualitative differences between the standard gradient descent and various NGD methods based on different metric spaces in nonconvex optimization problems.
Keyword:
natural gradient
constrained optimization
least-squares method
gradient flow
inverse problem
期刊
IF:
2.6
论文数:
5.1K
被引数:
1.8W
机构
引用论文
.png){kind=icon}
分享收藏Design And Practical Considerations In The Construction Of Magnetic Induction Stimulators磁感应刺激器构建的设计与实践考量
PREAI
分享收藏Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations物理信息神经网络: 一种用于解决涉及非线性偏微分方程的正反问题的深度学习框架
OAAI
OAAI分享收藏Induced electric fields by magnetic stimulation in non-homogeneous conducting media磁刺激在非均匀导电介质中诱导的电场
PREAI
分享收藏
查看更多内容