.png){kind=icon}
.png){kind=icon}
Return
.png){kind=icon}
.png){kind=icon}
.png){kind=icon}
.png){kind=icon}
.png){kind=icon}
Abstract
En 中文
Multimodal-based recommendation integrates visual, textual, and acoustic modalities of items to comprehensively capture user preferences, thereby playing a crucial role in modern multimedia online platforms. In this paper, we propose a Diffusion-based Multi-agent with Reinforcement Learning for Multimodal-based Recommendation (DRMRec). Specifically, our DRMRec first leverages a hierarchical Diffusion-based Multi-Agent (DMA) to reconstruct the user-item interaction graph. Subsequently, these reconstructed graphs are fused with multimodal information to form contrastive views. During the reconstruction, each agent is injected with multimodal information through a Cross-Modal Aligner (CMA), thereby bringing cross-modal information closer to interaction embeddings and facilitating alignment across modalities. Then, we introduce a Metric-Aware Diffusion Reinforcer (MADR), a reinforcement learning framework that leverages validation-set recommendation metrics as reward signals to enable dynamic and individual fine-tuning of each agent, thereby actively aligning model optimization with recommendation tasks. Next, we apply cross-modal graph contrastive learning to contrastive views, alleviating data sparsity while further enhancing cross-modal alignment. Extensive experiments on three real-world multimedia platform datasets demonstrate that DRMRec consistently outperforms state-of-the-art approaches in multimodal-based recommendation. It is noteworthy that the performance improvements across all three metrics on both the TikTok and Sports datasets exceed 10%.
Keywords:
Multimodal-based recommendation
diffusion model
reinforcement learning
multi-agent
Journal
I
IF:
5.7
Papers:
860
Citations:
3.0K