MTformer: A physics-guided spatio-temporal transformer for complex dynamic system modeling

Hao Tian

doi:10.1016/j.knosys.2026.115969

MTformer: A physics-guided spatio-temporal transformer for complex dynamic system modeling

2026-05-08

0

PRE

AI

DOI：10.1016/j.knosys.2026.115969

Original

Original request for help

Save

Abstract

En 中文

Accurately modeling complex spatiotemporal dynamics in systems such as fluid dynamics and weather forecasting remains a grand challenge due to the difficulty in balancing long-range dependency with numerical stability. Conventional CNN-based and RNN-based frameworks exhibit limited capability in capturing global dependencies and suffer from substantial error accumulation. While recent Transformer-based models excel at capturing global correlations, their lack of explicit physical inductive bias causes them to overfit statistical noise instead of underlying dynamical laws, leading to the generation of non-physical high-frequency oscillations,termed ”physical artifacts”,which undermines long-horizon forecasting stability.To address these issues, we propose MTformer, a physics-guided model built upon a systematic interleaving evolution framework. Our primary contribution is the strategic integration of TD-Blocks and Fourier Neural Operators directly into the Gated Transformer backbone, enabling a self-correcting feature evolution that alternates between neural-driven attention and operator-driven physical constraints. In this architecture, the Gated Transformer captures complex spatiotemporal nonlinearities, while the TD-Blocks embed a Laplacian-type smoothness prior to mitigate non-physical drifts. Simultaneously, the FNO components refine global spectral representations to maintain multi-scale consistency.Extensive experiments demonstrate the effectiveness of MTformer. On the TaxiBJ urban traffic prediction benchmark, the proposed model achieves an MSE of 0.271 and an MAE of 14.31, outperforming all compared state-of-the-art baselines. On the Navier–Stokes benchmark for strongly nonlinear flow dynamics, MTformer also exhibits superior long-term forecasting accuracy and stability. These results indicate that the proposed framework provides a robust solution for spatiotemporal prediction by jointly leveraging global attention, spectral modeling, and Laplacian-inspired temporal regularization.Our code is available at https://github.com/Long-th99/MTformer .

Keywords:

spatiotemporal dynamics

physics-guided modeling

Transformer

Fourier Neural Operator

long-range dependency

MTformer: A physics-guided spatio-temporal transformer for complex dynamic system modeling

Abstract

Journal

Organization