Kernelized Similarity Learning and Embedding for Dynamic Texture Synthesis

doi:10.1109/TSMC.2022.3188500

Kernelized Similarity Learning and Embedding for Dynamic Texture Synthesis

2023-02-01

0

OA

AI

DOI：10.1109/TSMC.2022.3188500

原文链接

原文求助

摘要

En 中文

Dynamic texture (DT) exhibits statistical stationarity in the spatial domain and stochastic repetitiveness in the temporal dimension, indicating that different frames of DT possess a high similarity correlation that is critical prior knowledge. However, existing methods cannot effectively learn a synthesis model for high-dimensional DT from a small number of training samples. In this article, we propose a novel DT synthesis method, which makes full use of similarity as prior knowledge to address this issue. Our method is based on the proposed kernel similarity embedding, which can not only mitigate the high dimensionality and small sample issues, but also has the advantage of modeling nonlinear feature relationships. Specifically, we first put forward two hypotheses that are essential for the DT model to generate new frames using similarity correlations. Then, we integrate kernel learning and the extreme learning machine into a unified synthesis model to learn kernel similarity embeddings for representing DTs. Extensive experiments on DT videos collected from the Internet and two benchmark datasets, i.e., Gatech Graphcut Textures and Dyntex, demonstrate that the learned kernel similarity embeddings can provide discriminative representations for DTs. Further, our method can preserve the long-term temporal continuity of the synthesized DT sequences with excellent sustainability and generalization. Meanwhile, it effectively generates realistic DT videos with higher speed and lower computation than the current state-of-the-art methods. The code and more synthesis videos are available at our project page https://shiming-chen.github.io/Similarity-page/Similarit.html.

Keyword:

Kernel

Correlation

Computational modeling

Stochastic processes

Training

Vehicle dynamics

Streaming media

Dynamic texture (DT)

extreme learning machine (ELM)

kernel similarity embedding

similarity prior knowledge

Kernelized Similarity Learning and Embedding for Dynamic Texture Synthesis

摘要

期刊

机构

引用论文