.png){kind=icon}
返回
A Cost-Efficient FPGA-Based CNN-Transformer Using Neural ODE
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3480977.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Transformer has been adopted to image recognition tasks and shown to outperform CNNs and RNNs while it suffers from high training cost and computational complexity. To address these issues, a hybrid approach has become a recent research trend, which replaces a part of ResNet with an MHSA (Multi-Head Self-Attention). In this paper, we propose a lightweight hybrid model which uses Neural ODE (Ordinary Differential Equation) as a backbone instead of ResNet so that we can increase the number of iterations of building blocks while reusing the same parameters, mitigating the increase in parameter size per iteration. The proposed model is deployed on a modest-sized FPGA device for edge computing. The model is further quantized by QAT (Quantization Aware Training) scheme to reduce FPGA resource utilization while suppressing the accuracy loss. The quantized model achieves 79.68% top-1 accuracy for STL10 dataset that contains 96x96 pixel images. The weights of the feature extraction network are stored on-chip to minimize the memory transfer overhead, allowing faster inference. By eliminating the overhead of memory transfers, inference can be executed seamlessly, leading to accelerated inference. The proposed FPGA implementation accelerates the backbone and MHSA parts by 34.01x , and achieves an overall 9.85x speedup when taking into account the software pre- and post-processing. The FPGA acceleration leads to 7.10x better energy efficiency compared to the ARM Cortex-A53 CPU. The proposed lightweight Transformer model is demonstrated on Xilinx ZCU104 board for the image recognition of 96x96 pixel images in this paper and can be applied to different image sizes by modifying the pre-processing layer.
Keyword:
Transformers
Field programmable gate arrays
Computational modeling
Accuracy
Attention mechanisms
Quantization (signal)
Costs
Training
Load modeling
Mathematical models
Artificial intelligence
machine learning
tiny machine learning
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
.png){kind=icon}
Inflexibility of mental planning: A characteristic disorder with prefrontal lobe lesions?心理计划的僵化: 前额叶病变的特征性障碍？
PREAI
分享收藏Metal-organic chemical vapor deposition of indium selenide films using a single-source precursor使用单源前体对硒化铟薄膜进行金属有机化学气相沉积
PREAI
分享收藏Research on the Efficiency of Wireless Power Transfer System Based on Multi-Auxiliary Transmitting Coils基于多辅助发射线圈的无线电能传输系统效率研究
PREAI
分享收藏The power of obfuscation techniques in malicious JavaScript code: A measurement study恶意JavaScript代码中混淆技术的功能: 一项测量研究
OAAI
OAAI分享收藏分享收藏
查看更多内容