.png){kind=icon}
.png){kind=icon}
Return
NuwaDynamics+: A Causality-Aware Generative Framework for Spatio-Temporal Representation Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/TPAMI.2026.3652303.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Spatio-temporal (ST) prediction is crucial in earth sciences, including meteorological forecasting and urban computing, to name just a few. Access to ample high-quality data, combined with deep models adept at inference, is essential for attaining significant outcomes. Yet, data scarcity and the substantial costs of sensor deployment result in notable data imbalances. Overly specialized models that lack causal linkages further undermine the generalizability of inference techniques. To address these challenges, we <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">first</b> introduce a causal framework for ST predictions, named <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><inline-formula><tex-math notation="LaTeX">$\mathtt{NuwaDynamics}$</tex-math><alternatives><mml:math><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi></mml:math><inline-graphic xlink:href="wang-ieq1-3652303.gif"/></alternatives><alternatives><mml:math><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi></mml:math><inline-graphic xlink:href="wang-ieq1-3652303.gif"/></alternatives></inline-formula></b>, aimed at pinpointing causal regions in data and providing models with the capability for causal reasoning in a dual-phase process. Initially, we employ upstream self-supervision to identify causally significant patches, equipping the model with generalizable insights and performing targeted interventions on non-essential patches to approximate potential testing distributions. This stage is known as the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">discovery</b> phase. Progressing from discovery, we apply the insights to downstream tasks tailored to specific ST goals, enhancing the model’s recognition of a wider potential data distribution and augmenting its causal perceptual abilities (referred to as the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Update</b> phase). Additionally, we address environmental controllability and high computational complexity by implementing channel multiplication and conditional generation methods. This process, termed <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><inline-formula><tex-math notation="LaTeX">$\mathtt{NuwaDynamics+}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq2-3652303.gif"/></alternatives><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq2-3652303.gif"/></alternatives></inline-formula></b>, can further be interpreted as the front-door adjustment technique in the causality domain. Through comprehensive experiments across ten real-world or simulated ST benchmarks, we demonstrate that integrating the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><inline-formula><tex-math notation="LaTeX">$\mathtt{NuwaDynamics+}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq3-3652303.gif"/></alternatives><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq3-3652303.gif"/></alternatives></inline-formula></b> concept substantially improves various model performance. <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><inline-formula><tex-math notation="LaTeX">$\mathtt{NuwaDynamics+}$</tex-math><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq4-3652303.gif"/></alternatives><alternatives><mml:math><mml:mrow><mml:mi mathvariant="monospace">NuwaDynamics</mml:mi><mml:mo>+</mml:mo></mml:mrow></mml:math><inline-graphic xlink:href="wang-ieq4-3652303.gif"/></alternatives></inline-formula></b> concept also significantly enhances the versatility across various dynamic ST tasks, such as extreme weather forecasting and long-temporal-step super-resolution predictions.
Keywords:
Spatio-temporal predictive learning
computer vision
causal inference
Journal
IF:
18.6
Papers:
831
Citations:
9.8W
View more organizations