arrow
Return

Reward-guided direct preference optimization

delete2025-11-04
delete0
PRE
AI
Z
Zhe Yuan Ding
S
Su Pan
Y
Yongpan Zhang
H
Hui Ji
C
Cheng Ding
DOI:10.1016/j.eswa.2025.130295delete
deleteOriginal
deleteOriginal request for help
deleteShare
deleteSave
Abstract

Abstract

En 中文
• Analyzed the likelihood changes of chosen and rejected responses in SFT and DPO. • Proposed a dynamic switching method for balancing likelihood and ranking objectives. • RG-DPO outperforms DPO for preference learning. • Proposed algorithm achieves competitive performance on HH-RLHF and TLDR datasets.
Keywords:
Alignment
Large language model
Preference optimization
Natural language processing

Journal

Expert Systems with Applications cover
Expert Systems with Applications
IF:
7.5
Papers:
3.0W
Citations:
10.2W

Organization

N
nanjing university of posts and telecommunications
Scholars:
3.8K
Papers: 1.6K
Citations: 0
C
china telecom co, ltd jiangsu branch
Scholars:
2
Papers: 1
Citations: 0
Cited Papers

Cited Papers

Diffusion Model Alignment Using Direct Preference Optimization
err2024-06-16
err0
PREAI
errBram Wallace; Meihua Dang; Rafael Rafailov; Linqi Zhou; Aaron Lou; Senthil Purushwalkam; Stefano Ermon; Caiming Xiong; Shafiq Joty; Nikhil Naik
errShare
errSave
TL;DR: Mining Reddit to Learn Automatic Summarization
err2017-01-01
err0
errOAAI
errMichael Völske; Martin Potthast; Shahbaz Syed; Benno Stein
errShare
errSave