.png){kind=icon}
返回
PE-Attack: On the Universal Positional Embedding Vulnerability in Transformer-Based Models
.png){kind=icon}
PRE
AI
DOI：10.1109/TIFS.2024.3442617.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The Transformer model has gained significant recognition for its remarkable computational capabilities and versatility, positioning itself as a fundamental component in numerous practical applications. However, the robustness of the Transformer model, specifically its stability and reliability under various types of adversarial attacks, is of utmost importance for its practical applicability. Furthermore, it offers valuable insights for the design of more efficient and secure models. In contrast with conventional investigations into adversarial robustness, our study focuses on the analysis of Positional Embeddings (PEs), a crucial component that sets the Transformer model apart from previous model architectures. Theoretical analysis of PEs has been limited due to previous predominantly empirical design, which includes features such as sinusoidal or linear patterns, learned or fixed characteristics, and absolute or relative measurements. Our investigation delves deep into potential vulnerabilities within PEs. Initially, we develop a set of input infection techniques that can be universally applied to exploit vulnerabilities present in the Transformer architecture and its variants. In addition, we propose a novel adversarial attack that manipulates the model by providing it with incorrect positional information, enabling an evasion attack. Significantly, in contrast to previous attacks that were limited to a single task, our conducted experiments involving time-series analysis, natural language processing, and computer vision indicate that the susceptibility of PEs could be universal and transferable. This finding serves as a significant warning for future Transformer-based model design, urging researchers to consider potential security risks inherent in the model's structure.
Keyword:
Transformers
Computational modeling
Task analysis
Robustness
Analytical models
Predictive models
Noise
Neural networks
robustness
transformer
adversarial attack
期刊
IF:
8
论文数:
5.3K
被引数:
2.3W
机构
引用论文
.png){kind=icon}
分享收藏分享收藏Economic benefit evaluation method for the micro-grid renewable energy system operation微网可再生能源系统运行经济效益评价方法
PREAI
分享收藏Welding characteristics of aluminum, copper, nickel and aluminum alloy with alumina coating using ultrasonic complex vibration welding equipments铝、铜、镍及铝合金氧化铝涂层超声复合振动焊接特性研究
PREAI
分享收藏
查看更多内容