.png){kind=icon}
返回
High-Performance Tensor-Train Primitives Using GPU Tensor Cores
.png){kind=icon}
PRE
AI
DOI：10.1109/TC.2024.3441831.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Learning tensor-train (TT) structure (a.k.a matrix product state (MPS) representation) from large-scale high-dimensional data has been a common task in big data analysis, deep learning, and quantum machine learning. However, tensor-train algorithms are compute-intensive, which hinders their real-world applications. In this paper, we present high-performance tensor-train primitives using GPU tensor cores and demonstrate three applications. First, we use GPU tensor cores to optimize tensor-train primitives, including tensor contraction, singular value decomposition, and data transfer and computing. Second, we utilize the optimized primitives to accelerate tensor-train decomposition algorithms for big data analysis. Further, we propose a shard mode for high-order tensor computations on multiple GPUs. Third, we apply the optimized primitives to accelerate the tensor-train layer for compressing deep neural networks. Last, we utilize the optimized primitives to accelerate a quantum machine learning algorithm called Density Matrix Renormalization Group (DMRG). In performance evaluations, our third-order TT tensor decomposition achieves up to 3.34x and 6.91x speedups over two popular libraries (namely T3F and tntorch) on an A100 GPU, respectively. The proposed sixth-order tensor-train decomposition achieves up to a speedup of 5.01x over T3F on multiple A100 GPUs. Our tensor-train layer for a fully connected neural network achieves a compression ratio of 65.3x at the cost of 0.3% drop in accuracy and a speedup of 1.53x over a PyTorch implementation on CUDA cores. The optimized DMRG algorithm achieves up to a speedup of 14.0x over TensorNetwork, indicating the potential of the optimized tensor primitives for the classical simulation of quantum machine learning algorithms.
Keyword:
Tensors
Graphics processing units
Matrix decomposition
Machine learning algorithms
Big Data
Libraries
Matrix converters
Tensor-train
matrix product state
GPU tensor cores
big data
tensor layer
quantum machine learning
期刊
IF:
3.8
论文数:
5.4K
被引数:
9.8K
机构
引用论文
Parallel time-dependent variational principle algorithm for matrix product states.png)
PHYSICAL REVIEW B
OAAI
PHYSICAL REVIEW BIF3.7
OAAI分享收藏分享收藏
查看更多内容