Syntax-Oriented Shortcut: A Syntax Level Perturbing Algorithm for Preventing Text Data From Being Learned

Bo Li

doi:10.1109/TNNLS.2025.3609842

Return

Syntax-Oriented Shortcut: A Syntax Level Perturbing Algorithm for Preventing Text Data From Being Learned

2025-10-06

PRE

DOI：10.1109/TNNLS.2025.3609842

Original

Original request for help

Save

Abstract

En 中文

The vast availability of free data has been critical to the success of large language models (LLMs). With the widespread use of LLMs, more and more concerns have been raised about the unauthorized use of publicly available data. To protect data from unauthorized use for training models, researchers have proposed adding imperceptible perturbations into image data so that models would be misled by the generated shortcut features and cannot mine information from these images. However, due to the inherent discrete property and semantic complexity of texts, directly applying these methods to text will cause semantic changes, resulting in meaningless shortcut features being constructed. To tackle this problem, in this article, we design a novel Unlearnable text examples generation algorithm via syntax-oriented shortcut (UTE-SS) by incorporating the syntactic structure of texts. Specifically, we propose a syntax template generator (STG) to generate the optimal perturbing syntax for a given category, which will realize imperceptible perturbations. Then, a perturbing text generator (PTG) is designed to perturb the in-class texts with the selected syntax template to stably deviate from the original texts. Along this line, models will be misled to learn the shortcut between the syntax template and the category, so as to keep text examples unlearnable. Extensive experiments over eight advanced Transformer-based pretrained language models (PLMs) on four different natural language processing (NLP) tasks demonstrate the effectiveness and flexibility of our proposed algorithm. Our method is easy to implement, and the code is publicly available at https://github.com/libolb/UTE-SS.

Keywords:

Data security

privacy protection

shortcut learning

unlearnable examples