.png){kind=icon}
返回
TPQA: Efficient attention architecture with task-aware pattern-guided quantization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.future.2025.108352.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Attention mechanisms have become a cornerstone of modern deep learning models, yet their computational intensity poses significant deployment challenges for resource-limited devices. While quantization offers a potential solution, current approaches typically employ uniform precision assignment schemes across all attention heads, neglecting critical variations in head-specific contributions across different tasks. This oversight results in substantial computational redundancy for those attention heads with fewer contributions, impacting overall performance. Through systematic analysis of head pattern characteristics in transformer models, we reveal two key insights: different attention heads exhibit distinct task-aware patterns, and their varying contributions to model performance directly dictate differentiated quantization demands across heads. Building on these findings, we propose TPQA, a novel algorithm and accelerator co-design architecture for efficient deployment of transformer models. TPQA strategically assigns adaptive precision levels to each head based on pre-identified patterns, thereby reducing computational overhead while preserving model accuracy. Furthermore, TPQA employs a data reordering strategy to transform irregular workloads into structured formats and introduces a dedicated accelerator with an attention-weights-stationary dataflow to efficiently process these structured workloads. Comprehensive evaluations demonstrate TPQA's superior performance, achieving up to 2.1x speedup and 3.4x energy efficiency improvement over state-of-the-art accelerators while maintaining <1% accuracy loss on various tasks.
Keyword:
Multi-head attention
Quantization
Accelerator
Head pattern
Systolic array
Attention architecture
期刊
F
IF:
6.1
论文数:
6.8K
被引数:
2.3W
机构
引用论文
VLCQ: Post-training quantization for deep neural networks using variable length codingVLCQ: 使用可变长度编码的深度神经网络的训练后量化
PREAI
分享收藏What Does BERT Look at? An Analysis of BERT’s Attention伯特在看什么？浅析伯特的注意力.png)
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
OAAI
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLPIF0
OAAI分享收藏Ramulator 2.0: A modern, modular, and extensible dram simulatorRamulator 2.0：一款现代、模块化且可扩展的DRAM模拟器
IF0
PREAI
分享收藏Block-Wise Mixed-Precision Quantization: Enabling High Efficiency for Practical ReRAM-Based DNN Accelerators分块混合精度量化：实现实用ReRAM基DNN加速器的高效性OAAI
OAAI分享收藏
查看更多内容