.png){kind=icon}
Return
A masked autoencoder network for spatiotemporal predictive learning
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-024-06214-2.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper is about predictive learning, which is generating future frames given previous images. Suffering from the vanishing gradient problem, existing methods based on RNN and CNN can't capture the long-term dependencies effectively. To overcome the above dilemma, we present MastNet a spatiotemporal framework for long-term predictive learning. In this paper, we design a Transformer-based encoder-decoder with hierarchical structure. As for the transformer block, we adopt the spatiotemporal window based self-attention to reduce computational complexity and the spatiotemporal shifted window partitioning approach. More importantly, we build a spatiotemporal autoencoder by the random clip mask strategy, which leads to better feature mining for temporal dependencies and spatial correlations. Furthermore, we insert an auxiliary prediction head, which can help our model generate higher-quality frames. Experimental results show that the proposed MastNet achieves the best results in accuracy and long-term prediction on two spatiotemporal datasets compared with the state-of-the-art models.
Keywords:
Spatiotemporal prediction
Predictive learning
Hierarchical architecture
Spatiotemporal transformer blocks
Random clip masked autoencoder
Auxiliary head
Journal
IF:
3.5
Papers:
7.6K
Citations:
1.7W
Organization
Cited Papers
ShareSaveCAST: A convolutional attention spatiotemporal network for predictive learning.png)
APPLIED INTELLIGENCE
APPLIED INTELLIGENCEIF3.5
PREAI
ShareSave
Synthesis of n-type semiconducting diamond film using diphosphorus pentaoxide as the doping source
PREAI
ShareSaveShareSaveShareSaveShareSave
View more