.png){kind=icon}
返回
Near-Memory Computing With Compressed Embedding Table for Personalized Recommendation
.png){kind=icon}
PRE
AI
DOI：10.1109/TETC.2023.3345870.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Deep learning (DL)-based recommendation models play an important role in many real-world applications. However, an embedding layer, which is a key part of the DL-based recommendation models, requires sparse memory accesses to a very large memory space followed by the pooling operations (i.e., reduction operations). It makes the system overprovision memory capacity for model deployment. Moreover, with conventional CPU-based architecture, it is difficult to exploit the locality, causing a huge burden for data transfer between the CPU and memory. To resolve this problem, we propose an embedding vector element quantization and compression method to reduce the memory footprint (capacity) required by the embedding tables. In addition, to reduce the amount of data transfer and memory access, we propose near-memory acceleration hardware with an SRAM buffer that stores the frequently accessed embedding vectors. Our quantization and compression method results in compression ratios of 3.95-4.14 for embedding tables in widely used datasets while negligibly affecting the inference accuracy. Our acceleration technique with 3D stacked DRAM memories, which facilitates the near-memory processing in the logic die with high DRAM bandwidth, leads to 4.9 x -5.4 x embedding layer speedup as compared to the 8-core CPU-based execution while reducing the memory energy consumption by 5.9 x -12.1 x, on average.
Keyword:
Memory management
Random access memory
Quantization (signal)
Data transfer
Energy consumption
Table lookup
Hardware
Compression
embedding table
inference
near-memory processing
personalized recommendation model
期刊
IF:
5.4
论文数:
1.1K
被引数:
3.4K
机构
引用论文
.png){kind=icon}
分享收藏Latent dimensions of social anxiety disorder: A re-evaluation of the Social Phobia Inventory (SPIN)社交焦虑障碍的潜在维度: 对社交恐惧症量表 (SPIN) 的重新评估
OAAI
OAAI分享收藏
查看更多内容