.png){kind=icon}
Return
Active preference-based Gaussian process regression for reward learning and optimization
.png){kind=icon}
PRE
AI
DOI：10.1177/02783649231208729.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Designing reward functions is a difficult task in AI and robotics. The complex task of directly specifying all the desirable behaviors a robot needs to optimize often proves challenging for humans. A popular solution is to learn reward functions using expert demonstrations. This approach, however, is fraught with many challenges. Some methods require heavily structured models, for example, reward functions that are linear in some predefined set of features, while others adopt less structured reward functions that may necessitate tremendous amounts of data. Moreover, it is difficult for humans to provide demonstrations on robots with high degrees of freedom, or even quantifying reward values for given trajectories. To address these challenges, we present a preference-based learning approach, where human feedback is in the form of comparisons between trajectories. We do not assume highly constrained structures on the reward function. Instead, we employ a Gaussian process to model the reward function and propose a mathematical formulation to actively fit the model using only human preferences. Our approach enables us to tackle both inflexibility and data-inefficiency problems within a preference-based learning framework. We further analyze our algorithm in comparison to several baselines on reward optimization, where the goal is to find the optimal robot trajectory in a data-efficient way instead of learning the reward function for every possible trajectory. Our results in three different simulation experiments and a user study show our approach can efficiently learn expressive reward functions for robotic tasks, and outperform the baselines in both reward learning and reward optimization.
Keywords:
Reward learning
active learning
inverse reinforcement learning
preference-based learning
human-robot interaction
Gaussian processes
Journal
IF:
5
Papers:
2.4K
Citations:
1.5W
Organization
Cited Papers
ShareSaveNeonatal Hearing Screening: Prevalence of Unilateral and Bilateral Hearing Loss and Associated Risk Factors.png)
Cureus
OAAI
CureusIF0
OAAIShareSave
ShareSaveShareSaveShareSave
View more