Enhancing Contrastive Learning in Sequence Classification via Data Augmentation

Hanjuan Huang

doi:10.1007/978-981-96-6291-3_20

Return

Enhancing Contrastive Learning in Sequence Classification via Data Augmentation

2026-01-01

PRE

Hanjuan Huang

C. J. Tang

Hsing-Kuo Pao *

DOI：10.1007/978-981-96-6291-3_20

Original

Original request for help

Save

Abstract

En 中文

We rely on massive labeled data for effective modeling. The statement remains true for sequential data. Any help to ease the acquirement of large-scale labeled data, such as self-supervised learning may become an unavoidable component in model learning. In this work, we propose a data augmentation technique that is suitable for sequential data with temporal dependency which can boost the performance of contrastive learning, as one of the major self-supervised learning approaches. When dealing with sequential data that own temporal dependency, various difficulties arise. In general, we have to pay attention to the temporal structures in the data so that any form of augmentation, on any part of the data may not hurt the temporal dependency or the dependency remains similar in the data. The proposed data augmentation falls into the strong augmentation category where cutting and shuffling various parts of the data are performed but the significant events remained unchanged in the temporal dependency. A dynamic time warping technique is adopted to isolate the significant events through an alignment procedure. A self-supervised learning technique is utilized to realize the mechanism. Compared to other data augmentation of the same kind, we emphasize an effective learning on the data representation. A fair evaluation is conducted to justify the effectiveness of the proposed method if compared to other state-of-the-art methods on various down-stream learning tasks. After all, experiment results show that the proposed method can effectively utilize information from both labeled and unlabeled data to build an internal representation to produce high-performance classifiers in the end.

Keywords:

contrastive learning

data augmentation

self-supervised learning

sequential data