.png){kind=icon}
Return
PredLM: A Sparse Large Language Model Decoding Accelerator With Hardware-Efficient Zero-Activation Prediction
.png){kind=icon}
PRE
AI
DOI：10.1109/TCSII.2026.3690531.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This brief presents PredLM, a hardware-algorithm co-designed accelerator that aggressively reduces external memory access (EMA) to achieve real-time large language model (LLM) decoding. First, we introduce a simple but effective Key-Value (KV) cache management policy so that the size of KV cache is fixed regardless of the context length, significantly reducing the EMA for the KV cache. Second, we propose a dynamic zero-activation prediction scheme for feed-forward networks, allowing the hardware to dynamically skip unnecessary multiply-accumulate operations and weight fetches. To seamlessly support these predictive algorithms, PredLM features a homogeneous processing element array able to support the execution of both structural sparse and dense vector-matrix multiplications. Implemented in a 28-nm CMOS technology and evaluated with the OPT-1.3B and Llama2-7B models, PredLM achieves a time-per-output-token of 1.76 and 9.41 ms/token and an energy efficiency of 19.77 and 105.09 mJ/token with negligible perplexity degradation, delivering up to $18.1\times $ higher throughput and $4.92\times $ better energy efficiency compared to state-of-the-art LLM accelerators.
Keywords:
Large language models (LLMs)
key-value (KV) cache reduction
zero-activation prediction
Journal
I
IF:
0
Papers:
153
Citations:
0
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSaveMobile Transformer Accelerator Exploiting Various Line Sparsity and Tile-Based Dynamic Quantization
PREAI
ShareSaveShareSaveShareSaveShareSave no more
no more