.png){kind=icon}
返回
Speeding up and reducing memory usage for scientific machine learning via mixed precision
.png){kind=icon}
OA
AI
DOI：10.1016/j.cma.2024.117093.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Scientific machine learning (SciML) has emerged as a versatile approach to address complex computational science and engineering problems. Within this field, physics -informed neural networks (PINNs) and deep operator networks (DeepONets) stand out as the leading techniques for solving partial differential equations by incorporating both physical equations and experimental data. However, training PINNs and DeepONets require significant computational resources, including long computational times and large amounts of memory. In search of computational efficiency, training neural networks using half precision (float16) rather than the conventional single (float32) or double (float64) precision has gained substantial interest, given the inherent benefits of reduced computational time and memory consumed. However, we find that float16 cannot be applied to SciML methods, because of gradient divergence at the start of training, weight updates going to zero, and the inability to converge to a local minima. To overcome these limitations, we explore mixed precision, which is an approach that combines the float16 and float32 numerical formats to reduce memory usage and increase computational speed. Our experiments showcase that mixed precision training not only substantially decreases training times and memory demands but also maintains model accuracy. We also reinforce our empirical observations with a theoretical analysis. The research has broad implications for SciML in various computational applications.
Keyword:
Scientific machine learning
Partial differential equations
Physics-informed neural networks
Deep operator networks
Mixed precision
Computational efficiency
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.3
论文数:
1.3W
被引数:
5.6W
机构
引用论文
Gradient-enhanced physics-informed neural networks for forward and inverse PDE用于正向和反向PDE的梯度增强物理通知神经网络OAAI
OAAI分享收藏Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations物理信息神经网络: 一种用于解决涉及非线性偏微分方程的正反问题的深度学习框架OAAI
OAAI分享收藏Data-driven Modeling of Hemodynamics and its Role on Thrombus Size and Shape in Aortic Dissections.png)
SCIENTIFIC REPORTSOAAI
SCIENTIFIC REPORTSIF3.9
OAAI分享收藏A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks物理信息神经网络的非自适应和基于残差的自适应采样的综合研究OAAI
OAAI分享收藏Physics-informed neural networks for inverse problems in nano-optics and metamaterials用于纳米光学和超材料中反问题的物理通知神经网络OPTICS EXPRESS
OAAI
OPTICS EXPRESSIF3.3
OAAI分享收藏Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data用于高维代理建模和不确定性量化的物理约束深度学习，无需标记数据OAAI
OAAI分享收藏
查看更多内容