.png){kind=icon}
Return
Kitsune: Enabling Dataflow Execution on GPUs with Spatial
.png){kind=icon}
PRE
AI
DOI：10.1145/3777466.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
State-of-the-art DL models are growing in size and complexity, with many modern models also increasing in heterogeneity of behavior. GPUs are still the dominant platform for DL applications, relying on a bulk-synchronous execution model which has many drawbacks and is ill-suited for the graph structure of DL applications. Many industry and academic works attempt to overcome these by employing vertical fusion-combining multiple sequential operations into a single kernel-but this approach still fails to realize three untapped opportunities: (1) the fact that many resources on the GPU are idle while only one operator executes due to temporal multiplexing of the SM; (2) lower energy from more intelligent on-chip data-movement which lends to higher performance in a power-provisioned environment. (3) inability to exploit reduction dimensions as a source of parallelism to ease pressure on batch size. This article explores relatively uncharted territory, answering the following key question: Can modest adjustments to the current GPU architecture enable efficient dataflow execution, thereby circumventing the constraints of vertical fusion without necessitating a clean-slate architecture design. We develop Kitsune-a set of primitives to construct spatial pipelines which enable dataflow execution on GPUs, and an end-to-end compiler based on PyTorch Dynamo. Across 5 challenge applications, Kitsune can provide up to 2.8x and 2.2x performance improvement as well as up to 99% and 45% off-chip traffic reduction for inference and training, respectively.
Keywords:
GPU architecture
spatial pipelining
artificial intelligence
Journal
A
IF:
1.8
Papers:
96
Citations:
1.1K
Organization
Cited Papers
ShareSaveShareSaveCerebras Architecture Deep Dive: First Look Inside the Hardware/Software Co-Design for Deep Learning.png)
IEEE Micro
IEEE MicroIF0
PREAI
ShareSaveShareSave
ShareSaveShareSave
View more