A Learned Performance Model With Transfer Learning Across GPUs on Tensorized Instructions

Yang Bai

doi:10.1109/TPDS.2025.3578630

2025-09-01

PRE

DOI：10.1109/TPDS.2025.3578630

原文链接

原文求助

摘要

En 中文

训练和推理效率日益增大的深度神经网络的性能高度依赖于特定硬件加速器上张量操作的性能。因此，具有张量化指令编译的自动张量生成性能调优框架对于高效部署是必要的。这些新颖的张量化指令，以及新兴的机器学习模型，为基于编译的方法带来了巨大的工程挑战。它们在探索巨大的设计空间时面临精度粗糙的测量和特定硬件约束下专门指令间差劲的转移能力。本文提出了一种用于张量化指令自动代码优化的新型性能模型。该性能模型的核心是分配特征，它不仅通过计算和数据移动抽象清晰地指定了指令的行为，还正式定义了从算法到张量化指令的匹配问题。同时，一个简单而高效的基于注意力启发模块的设计，通过捕获完整调度空间内的全局和长距离依赖关系，来精确预测优化后张量程序的性能。与最先进技术相比，我们的性能模型能够预测具有张量化指令代码配置的最优实现，以在现代DNN基准测试中将推理延迟和搜索时间最多减少1.21倍和3.41倍。此外，通过预训练参数，我们的性能能够通过迁移学习快速适应不同工作负载和平台上的张量化指令。

Keyword:

Attention

code generation

compiler

performance model

tensorized instructions

期刊

IEEE Transactions on Parallel and Distributed Systems

IF:

论文数:

5.2K

被引数:

1.1W

机构

The Chinese University of Hong Kong

学者数:

3.8K

论文数: 1.9K

被引数: 3

The University of Hong Kong

学者数:

6.6K

论文数: 3.2K

被引数: 7

引用论文

In-Datacenter Performance Analysis of a Tensor Processing Unit

Proceedings of the 44th Annual International Symposium on Computer Architecture

IF0

2017-06-24

OAAI

Norman P. Jouppi; Cliff Young; Nishant Patil; David Patterson; Gaurav Agrawal; Raminder Bajwa; Sarah Bates; Suresh Bhatia; Nan Boden; Al Borchers; Rick Boyle; Pierre-luc Cantin; Clifford Chao; Chris Clark; Jeremy Coriell; Mike Daley; Matt Dau; Jeffrey Dean; Ben Gelb; Tara Vazir Ghaemmaghami; Rajendra Gottipati; William Gulland; Robert Hagmann; C. Richard Ho; Doug Hogberg; John Hu; Robert Hundt; Dan Hurt; Julian Ibarz; Aaron Jaffey; Alek Jaworski; Alexander Kaplan; Harshit Khaitan; Daniel Killebrew; Andy Koch; Naveen Kumar; Steve Lacy; James Laudon; James Law; Diemthu Le; Chris Leary; Zhuyuan Liu; Kyle Lucke; Alan Lundin; Gordon MacKean; Adriana Maggiore; Maire Mahony; Kieran Miller; Rahul Nagarajan; Ravi Narayanaswami; Ray Ni; Kathy Nix; Thomas Norrie; Mark Omernick; Narayana Penukonda; Andy Phelps; Jonathan Ross; Matt Ross; Amir Salek; Emad Samadiani; Chris Severn; Gregory Sizikov; Matthew Snelham; Jed Souter; Dan Steinberg; Andy Swing; Mercedes Tan; Gregory Thorson; Bo Tian; Horia Toma; Erick Tuttle; Vijay Vasudevan; Richard Walter; Walter Wang; Eric Wilcox; Doe Hyun Yoon

FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2019-06-01

OAAI

Bichen Wu; Kurt Keutzer; Xiaoliang Dai; Peizhao Zhang; Yanghan Wang; Fei Sun; Yiming Wu; Yuandong Tian; Peter Vajda; Yangqing Jia

TensorIR: An Abstraction for Automatic Tensorized Program Optimization

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

IF0

2023-01-30