.png){kind=icon}
.png){kind=icon}
Return
Dynamic Communication Optimization with Collision Avoidance for Parallel Programs in Distributed Systems
.png){kind=icon}
PRE
AI
DOI：10.1007/s10766-025-00808-0.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In distributed systems with redundant network paths, dynamically selecting optimal communication routes for parallel programs is essential for minimizing latency and avoiding congestion. However, this is challenging due to unpredictable network conditions and concurrent workloads that create time-varying performance characteristics. This paper presents a reinforcement learning framework that enables programs to adaptively select communication routes based on historical performance without requiring global network state monitoring. We employ the Upper Confidence Bound 1 (UCB1) algorithm for small candidate sets and an improved \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\epsilon $$\end{document}-greedy algorithm for larger sets, providing logarithmic regret bounds in stationary environments and sublinear regret in dynamic scenarios. We demonstrate this approach on the 3-Quads cluster, a distributed system with three redundant subnetworks, where simulation and visualization programs run concurrently. Experiments show that our method reduces data transmission delay by 30-45% compared to random routing, with failure rates below 6% and overhead less than 0.6%. The approach converges to near-optimal routes within 500 communication rounds across different data sizes. While demonstrated experimentally on 3-Quads, theoretical analysis indicates that the framework can generalize to other redundant network topologies including fat-tree and dragonfly networks, with performance guarantees whose dependence on network topology is limited to the candidate set size m.
Keywords:
Distributed systems
Reinforcement learning
Communication optimization
Multi-path routing
Online learning
Journal
I
IF:
0.9
Papers:
18
Citations:
476