Learning hash index based on a shallow autoencoder

Yuming Lin

doi:10.1007/s10489-022-04274-w

Learning hash index based on a shallow autoencoder

2022-11-07

1

PRE

AI

DOI：10.1007/s10489-022-04274-w

Original

Original request for help

Save

Abstract

En 中文

The hash index plays an important role in improving query efficiency in databases. Because traditional hash algorithms cannot use the original data distribution, there is often a high collision rate in large-scale datasets. Additionally, some traditional hashing algorithms are data-dependent and cannot be accelerated in parallel. The learned index provides a new method for index design in data management systems. The key idea of the learned index is to consider the index as a model that can be learned. In this paper, we propose a learning hash algorithm based on a shallow autoencoder that can make full use of the original data characteristics and take advantage of the parallelism of matrix operations. Therefore, compared with traditional hash functions, the proposed method has a lower collision rate and higher efficiency. Finally, we verify the effectiveness of the proposed method through a series of experiments on synthetic datasets and real datasets. Experimental results show that the proposed hash algorithm has considerable advantages in reducing the collision rate and computing time while improving the retrieval efficiency.

Keywords:

Hash index

Learned index

Autoencoder

Machine learning

Learning hash index based on a shallow autoencoder

Abstract

Journal

Organization