A Deep Learning Framework for Start-End Frame Pair-Driven Motion Synthesis

Guiyu Xia

doi:10.1109/TNNLS.2022.3213596

2024-05-01

PRE

DOI：10.1109/TNNLS.2022.3213596

原文链接

原文求助

摘要

En 中文

A start-end frame pair and a motion pattern-based motion synthesis scheme can provide more control to the synthesis process and produce content-various motion sequences. However, the data preparation for the motion training is intractable, and concatenating feature spaces of the start-end frame pair and the motion pattern lacks theoretical rationality in previous works. In this article, we propose a deep learning framework that completes automatic data preparation and learns the nonlinear mapping from start-end frame pairs to motion patterns. The proposed model consists of three modules: action detection, motion extraction, and motion synthesis networks. The action detection network extends the deep subspace learning framework to a supervised version, i.e., uses the local self-expression (LSE) of the motion data to supervise feature learning and complement the classification error. A long short-term memory (LSTM)-based network is used to efficiently extract the motion patterns to address the speed deficiency reflected in the previous optimization-based method. A motion synthesis network consists of a group of LSTM-based blocks, where each of them is to learn the nonlinear relation between the start-end frame pairs and the motion patterns of a certain joint. The superior performances in action detection accuracy, motion pattern extraction efficiency, and motion synthesis quality show the effectiveness of each module in the proposed framework.

Keyword:

Feature extraction

Training

Data models

Representation learning

Motion segmentation

Deep learning

Task analysis

Action detection

local self-expression (LSE) layer

long short-term memory (LSTM)

motion pattern extraction