Ember: A Compiler for Embedding Operations on Decoupled Access-Execute Architectures

doi:10.1109/CGO68049.2026.11395192

Ember: A Compiler for Embedding Operations on Decoupled Access-Execute Architectures

2026-01-01

1

PRE

AI

DOI：10.1109/CGO68049.2026.11395192

Original

Original request for help

Save

Abstract

En 中文

Decoupled Access-Execute (DAE) architectures separate memory accesses from computation in two specialized units. This design is becoming increasingly popular among hyperscalers to accelerate irregular embedding lookups in recommendation models. In this paper, we first broaden the scope by demonstrating the benefits of DAE architectures across a wider range of irregular embedding operations in several machine learning models. Then, we propose the Ember compiler to automatically compile all of these embedding operations to DAE architectures. Conversely from other DAE compilers, Ember features multiple intermediate representations specifically designed for different optimization levels. In this way, Ember can implement all optimizations to match the performance of hand-written code, unlocking the full potential of DAE architectures at scale.

Keywords:

Recommendation models

Large language models

Graph learning models

Tensor compilers

Hardware-software co-design

Machine learning accelerators

Ember: A Compiler for Embedding Operations on Decoupled Access-Execute Architectures

Abstract

Journal

Organization