.png){kind=icon}
.png){kind=icon}
Return
Statistical mechanics of physics-informed neural networks: Free energy landscapes, phase transitions, and generalization in thermodynamic learning
.png){kind=icon}
PRE
AI
M
F
DOI：10.1016/j.physa.2026.131805.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Physics-informed neural networks (PINNs) have emerged as a powerful paradigm for solving partial differential equations (PDEs) and modeling complex physical systems by embedding governing laws directly into the loss function. However, the theoretical foundations of PINN training dynamics remain poorly understood, particularly regarding why and how these networks generalize from limited data while respecting physical constraints. In this paper, we propose a principled statistical mechanics framework for understanding and improving PINN training, which we call Thermodynamic Physics-Informed Learning (TPIL). Our framework establishes a rigorous mapping—derived from the Fokker-Planck equation governing SGD dynamics—between the PINN loss landscape and a statistical mechanical free energy (Eq. 9) landscape, where the network parameters play the role of thermodynamic degrees of freedom and the loss function serves as the energy functional. We introduce four key innovations derived from this mapping: (1) a Boltzmann-weighted physics-informed loss function that naturally balances data fidelity and PDE residual terms through temperature-dependent weighting; (2) a Tsallis entropy-based (Eq. 29) phase transition detector that identifies the critical transition from memorization to generalization during training; (3) an adaptive temperature annealing schedule inspired by simulated annealing that systematically explores and exploits the loss landscape; and (4) entropy-based adaptive weighting for PDE residual terms that dynamically adjusts the contribution of different physical constraints. Extensive experiments on three real-world public datasets—the UCI Superconductivity dataset (21,263 samples, 81 features), the NIST ThermoML thermodynamic properties database (14,500 measurements), and the MD17 molecular dynamics benchmark (seven molecules)—demonstrate that TPIL consistently outperforms standard PINNs and baseline methods. On the UCI Superconductivity dataset, TPIL achieves an RMSE of 4.23 K compared to 7.81 K for standard MLP and 5.67 K for standard PINN, representing improvements of 45.8% and 25.4%, respectively. Phase transition analysis reveals that TPIL undergoes a well-defined thermodynamic transition at approximately epoch 200, providing an interpretable criterion for early stopping. Our ablation study confirms that each component of the TPIL framework contributes meaningfully to the overall performance, with the Boltzmann weighting (Eqs. 16–18) and entropy adaptation components providing the largest marginal gains. A comprehensive gap assessment demonstrates that TPIL is the novel integrated framework combining thermodynamic training, Tsallis entropy regularization, phase-transition detection, and PINN-specific adaptive weighting, addressing significant limitations in existing adaptive PINN methods and statistical mechanics approaches to learning.
Journal
P
IF:
3.1
Papers:
1.3K
Citations:
3.6W
Organization
No organization information available
Cited Papers