.png){kind=icon}
.png){kind=icon}
Return
Prospect utility with hyperbolic tangent function
.png){kind=icon}
OA
AI
DOI：10.1016/j.ijar.2025.109440.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
One branch of safety in reinforcement learning is through integrating risk sensitivity within the Markov Decision Process framework. The objective is to mitigate low-probability events that could lead to severe negative outcomes. Eliminating such risky events is usually done by incorporating a utility function on the expected return; therefore, reshaping the reward structures according to the risk levels associated with different outcomes. The temporal difference learning algorithm can be modified with a utility to capture risk. Notably, such utility functions are either convex or concave depending on the desired risk behavior. Given the outcome space and depending on the risk-sensitivity mode, concave utilities may promote risk-averse behavior and convex utilities may encourage risk-seeking strategies. Such function structure is demonstrated in Prospect Theory, and this motivates a novel formulation using the hyperbolic tangent function called PTanh. Using PTanh, experiments are performed to assess the effect of the diminishing marginal property on the risk-averse policies. It is concluded that there is a correlation between the marginal and selecting the risk-averse parameters. The marginals influence the effectiveness of the averse policies. When the marginals are considered, PTanh can demonstrate better results in terms of a ratio of average reward per prohibited state rate. Furthermore, using empirical evidence, the policy experiments shown with PTanh generalize to other utilities of the Prospect Shape.
Keywords:
Risk-sensitivity
Reliability and risk analysis
Reinforcement learning
Temporal difference
Prospect theory utility function
Hyperbolic tangent function
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
3
Papers:
2.9K
Citations:
5.1K