Effective service composition using multi-agent reinforcement learning

Hongbing Wang

doi:10.1016/j.knosys.2015.10.022

2016-01-01

PRE

王

王红兵 (Hongbing Wang) *

Xiaojun Wang

Xingzhi Zhang

Qi Yu

DOI：10.1016/j.knosys.2015.10.022

原文链接

原文求助

摘要

En 中文

As online services may keep evolving, service composition should maintain certain adaptivity especially for a dynamic composition environment. Meanwhile, the large number of potential candidate services poses scalability concerns, which demand efficient composition solutions. This paper presents a multi-agent reinforcement learning model for Web service composition that effectively addresses the above challenges. In particular, we model a service composition as a Markov Decision Process. Based on the model, agents in a team would benefit from one another. In contrast to single-agent reinforcement-learning, our method can speed up the convergence to an optimal policy. We develop two multi-agent reinforcement learning algorithms. The first one introduces the concept of articulate state and distributed Q-learning to speed up the convergence time. The second one proposes the experience sharing strategy to improve the efficiency of learning. As the learning process continues throughout the life-cycle of a service composition, our algorithms can automatically adapt to the change of environment and the evolving component services. We conduct a simulation study to compare our algorithm with other similar reinforcement learning algorithms, including the traditional Q-learning algorithm, a multi-agent Sarsa algorithm, a Q-learning algorithm based on gaussian process, and a multi-agent Q-learning algorithm, to justify the effectiveness of our model and algorithm. (C) 2015 Elsevier B.V. All rights reserved.

Keyword:

Web service composition

Multi-agent

Reinforcement learning

Q-learning