.png){kind=icon}
.png){kind=icon}
Return
Efficient Transformers: A Survey
.png){kind=icon}
OA
AI
DOI：10.1145/3530811.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision, and reinforcement learning. In the field of natural language processing for example, Transformers have become an indispensable staple in the modern deep learning stack. Recently, a dizzying number of X- former models have been proposed-Reformer, Linformer, Performer, Longformer, to name a few which improve upon the original Transformer architecture, many of which make improvements around computational and memory efficiency. With the aim of helping the avid researcher navigate this flurry, this article characterizes a large and thoughtful selection of recent efficiency-flavored X-former models, providing an organized and comprehensive overview of existing work and models across multiple domains.
Keywords:
Transformers
attention
deep learning
neural networks