.png){kind=icon}
返回
Allspark: Workload Orchestration for Visual Transformers on Processing In-Memory Systems
.png){kind=icon}
PRE
AI
DOI：10.1109/TC.2024.3483633.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
The advent of Transformers has revolutionized computer vision, offering a powerful alternative to convolutional neural networks (CNNs), especially with the local attention mechanism that excels at capturing local structures within the input and achieve state-of-the-art performance. Processing in-memory (PIM) architecture offers extensive parallelism, low data movement costs, and scalable memory bandwidth, making it a promising solution to accelerate Transformer with memory-intensive operations. However, the crucial issue lies in efficiently deploying an entire model onto resource-limited PIM system while parallelizing each transformer block with potentially many computational branches based on local-attention mechanisms. We present Allspark, which focuses on workload orchestration for visual Transformers on PIM systems, aiming at minimizing inference latency. Firstly, to fully utilize the massive parallelism of PIM, Allspark employs a fine-grained partitioning scheme for computational branches, and formats a systematic layout and interleaved dataflow with maximized data locality and reduced data movement. Secondly, Allspark formulates the scheduling of the complete model on a resource-limited distributed PIM system as an integer linear programming (ILP) problem. Thirdly, as local-global data interactions exhibit complex yet regular dependencies, Allspark provides a two-stage placement method, which simplifies the challenging placement of computational branches on the PIM system into the structured layout and greedy-based binding, to minimize NoC communication costs. Extensive experiments on 3D-stacked DRAM-based PIM systems show that Allspark brings $1.2\times$1.2x $\sim$similar to $24.0\times$24.0x inference speedup for various visual Transformers over baselines. Compared to Nvidia V100 GPU, Allspark-enriched PIM system yields average speedups of $2.3\times$2.3x and energy savings of $20\times$20x $\sim$similar to $55\times$55x.
Keyword:
Transformers
Visualization
Parallel processing
Computer architecture
Computational modeling
Bandwidth
Costs
Memory management
Layout
System-on-chip
Processing in-memory
scheduling
visual transformer
spatial architecture
model parallelism
期刊
IF:
3.8
论文数:
5.4K
被引数:
9.8K
机构
引用论文
Towards Identifying Grass Seed Infestation in the Australian Sheep Using Terahertz Radiation基于太赫兹辐射识别澳大利亚绵羊草籽侵染的研究
OAAI
.png){kind=icon}
OAAI分享收藏
Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏分享收藏
查看更多内容