.png){kind=icon}
.png){kind=icon}
Return
RAG Embeddings Storage Optimization Through Quantization and Dimensionality Reduction
.png){kind=icon}
PRE
AI
DOI：10.1007/978-3-032-08465-1_16.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Retrieval-Augmented Generation is a powerful framework that enhances language models by retrieving relevant information from large external knowledge bases, relying heavily on high-dimensional vector embeddings. However, storing these embeddings at scale poses significant challenges due to their substantial memory requirements. To address this, we investigate two complementary strategies: quantization of embeddings using low-bit floating point formats, and dimensionality reduction via Principal Component Analysis (PCA). Our experiments, measuring performance using nDCG@10 (the standard MTEB Retrieval benchmark metric for ranking quality) relative to the float32 baseline, show that low-bit formats like float8 achieve 4x storage reduction with minimal (<0.3%) performance loss, significantly outperforming int8 at the same compression level. Notably, combining float8 with moderate PCA (e.g., retaining 50% dimensions) yields even better trade-offs, achieving 8x total compression with less performance degradation than int8 alone (e.g., a 0.62% drop vs 1.53% for nomic-embed-text-v1.5) while requiring only half the storage space. These findings highlight the effectiveness of combining float8 quantization and dimensionality reduction for efficient RAG embedding storage with minimal impact on retrieval quality.
Keywords:
Artificial Intelligence
Machine Learning
Retrieval-Augmented Generation
Quantization
Dimensionality Reduction
Storage Optimization
Journal
H
IF:
0
Papers:
25
Citations:
0