logo
科言猫学术研究的AI总结
language
手机端手机端
  • Phone Feature微信小程序
简体中文简体中文
notice消息通知
博客博客
delete
arrow
返回

MPN-S: An Algorithm-Hardware Co-Designed Architecture for High-Efficiency Softmax Computation

delete
delete0
PRE
AI
Y
Yuchen Ma
D
Defa Wu
Y
Yucheng Du
X
Xing Wang
X
Xin Si
C
Chao Chen
DOI：10.1109/TCSII.2025.3594105delete
delete原文链接
delete原文求助
delete分享
delete收藏
摘要

摘要

En 中文
作为Transformer中的基本算子，softmax因其计算复杂度而面临关键硬件挑战。softmax中广泛使用的指数和除法运算不仅占用了大量计算资源，还导致硬件实现时的高执行延迟。本研究提出了一种高精度算法与硬件协同设计的softmax架构MPN-S，包括映射优化、动态剪枝以及结合流水线调度的动态牛顿迭代法。所提出的MPN-S通过最小误差整数映射方法、动态剪枝和压缩查找表简化了softmax计算。结合动态牛顿迭代和流水线设计，MPN-S将softmax处理时间压缩为一次寻找最大值的时间。实现结果表明，在28 nm CMOS工艺下0.5 GHz频率时，MPN-S在不包括缓冲区的2292 μm²面积上可达到3827.18 Gps/(mm²·mW)的能效。神经网络实验显示，将softmax函数替换为MPN-S后，MobileVIT分类任务的准确率达到92.9%，Yolo V10的mAP50val达到0.959。
Keyword:
Softmax function
transformer
neural network
collaborative optimization
hardware implementation

期刊

I
IEEE Transactions on Circuits and Systems and Express Briefs
IF:
4.9
论文数:
8.8K
被引数:
2.5W

机构

S
Southeast University
学者数:
2.1W
论文数: 8.6K
被引数: 480
引用论文

引用论文

TEA-S: A Tiny and Efficient Architecture for PLAC-Based Softmax in Transformers
errIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS II-EXPRESS BRIEFS
IF4.9
err2023-09-01
err3
PREAI
errMei, Zhengyu; Dong, Hongxi; Wang, Yuxuan; Pan, Hongbing
err分享
err收藏
Design and Implementation of an Approximate Softmax Layer for Deep Neural Networks
err2020 IEEE International Symposium on Circuits and Systems (ISCAS)
IF0
err2020-10-01
err0
PREAI
errYue Gao; Weiqiang Liu; Fabrizio Lombardi
err分享
err收藏
DNN Acceleration: A High-Accuracy Implementation for Base-2 Softmax Layer
err2023 International Mobile, Intelligent, and Ubiquitous Computing Conference (MIUCC)
IF0
err2023-09-27
err0
PREAI
errThabit,Doha; Alabassy,Bassma; El-Kharashi,M. Watheq; Safar,Mona; Dessouky,Mohamed
err分享
err收藏
Attention Is All You Need注意是你所需要的
err
IF0
err
err0
errOAAI
err
err分享
err收藏
A Low-complexity and Reconfigurable Design for Nonlinear Function Approximation in Transformers
err2025 IEEE International Symposium on Circuits and Systems (ISCAS)
IF0
err2025-05-25
err0
PREAI
errWu,Qi-Xian; Teng,Shu-Sian; Shieh,Ming-Der; Huang,Chih-Tsun; Lu,Juin-Ming
err分享
err收藏
err
IF0
err
err0
PREAI
err
err分享
err收藏
Hardware-Aware Softmax Approximation for Deep Neural Networks
errLecture Notes in Computer Science
IF0
err2019-05-25
err0
PREAI
errXue Geng; Jie Lin; Bin Zhao; Anmin Kong; Mohamed M. Sabry Aly; Vijay Chandrasekhar
err分享
err收藏
Base-2 Softmax Function: Suitability for Training and Efficient Hardware Implementation
errIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS I-REGULAR PAPERS
IF5.2
err2022-09-01
err16
PREAI
errZhang, Yuan; Zhang, Yonggang; Peng, Lele; Quan, Lianghua; Zheng, Shubin; Lu, Zhonghai; Chen, Hui
err分享
err收藏
28nm FDSOI technology platform for high-speed low-voltage digital applications
err2012 Symposium on VLSI Technology (VLSIT)
IF0
err2012-06-01
err0
PREAI
errN. Planes; O. Weber; V. Barral; S. Haendler; D. Noblet; D. Croain; M. Bocat; P.-O. Sassoulas; X. Federspiel; A. Cros; A. Bajolet; E. Richard; B. Dumont; P. Perreau; D. Petit; D. Golanski; C. Fenouillet-Beranger; N. Guillot; M. Rafik; V. Huard; S. Puget; X. Montagner; M.-A. Jaud; O. Rozeau; O. Saxod; F. Wacquant; F. Monsieur; D. Barge; L. Pinzelli; M. Mellier; F. Boeuf; F. Arnaud; M. Haond
err分享
err收藏
Approximate Softmax Functions for Energy-Efficient Deep Neural Networks
errIEEE Transactions on Very Large Scale Integration (VLSI) Systems
IF0
err2023-01-01
err0
PREAI
errKe Chen; Yue Gao; Haroon Waris; Weiqiang Liu; Fabrizio Lombardi
err分享
err收藏
学者 查看更多内容