.png){kind=icon}
.png){kind=icon}
Return
An efficient parallel DeepFM for recommendation systems based on Spark
.png){kind=icon}
PRE
AI
Q
Y
Z
S
Z
M
DOI：10.1016/j.jpdc.2026.105270.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
DeepFM is an effective recommendation algorithm, however, as the scale of recommendation data increases, it fails to exploit its expected performance fully. To tackle the aforementioned challenges, we introduce a novel parallel algorithm for DeepFM based on Spark, called SDFM, which comprises three primary components. Firstly, we partition the preprocessed data and compute the embedding matrices in parallel. Secondly, we simplify the computation process of the FM component by extracting the effective values of each feature, thereby streamlining the formula while maintaining its accuracy. Lastly, building on data parallelism, we further partition and parallelize the computation of each hidden layer node within the Deep component. These three components fully leverage the distributed and parallel nature of Spark, enabling DeepFM to achieve enhanced performance on large-scale datasets. Ultimately, we extensively validate the performance of SDFM through abundant experiments. Comparing our algorithm with DeepFM deployed on similar distributed platforms such as Flink and Hadoop, our approach consistently achieves superior performance in the same amount of time.
Keywords:
DeepFM
Spark
parallel algorithm
recommendation systems
distributed computing
Journal
IF:
4
Papers:
3.8K
Citations:
4.8K
Organization
Cited Papers