NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning

doi:10.1007/978-3-031-99872-0_20

2026-01-01

PRE

DOI：10.1007/978-3-031-99872-0_20

原文链接

原文求助

摘要

En 中文

大规模分布式机器学习模型的训练对网络基础设施提出了巨大需求，通常导致突发流量激增，进而引发拥塞、延迟增加和吞吐量降低，最终影响收敛时间和整体训练性能。尽管梯度压缩技术常被用于缓解网络负载，但它们因梯度信息损失而频繁牺牲模型精度。本文介绍了NetSenseML，一种新颖的网络自适应分布式深度学习框架，该框架能根据实时网络状况动态调整量化、剪枝和压缩策略。通过主动监控网络状况，NetSenseML仅在因网络拥塞而影响收敛速度时才应用梯度压缩，从而有效平衡数据载荷减少与模型精度保持。我们的方法通过根据当前网络条件调整缩减技术，确保资源高效利用，实现更短的收敛时间和更高的训练效率。本文阐述了NetSenseML自适应数据缩减函数的设计，实验评估表明，在带宽受限条件下，对于典型的DDL训练任务，NetSenseML相比当前最先进的压缩启用系统，可将训练吞吐量提升1.55至9.84倍。

Keyword:

Distributed Systems

Systems for Machine Learning

DNN Training

Gradient Compression

期刊

EURO-PAR 2025: PARALLEL PROCESSING, PT III

IF:

论文数:

被引数:

机构

hong kong university of science & technology (guangzhou)

学者数:

214

论文数: 92

被引数: 0

引用论文

The QUIC Transport Protocol

Proceedings of the Conference of the ACM Special Interest Group on Data Communication

IF0

2017-08-07

OAAI

Adam Langley; Alistair Riddoch; Alyssa Wilk; Antonio Vicente; Charles Krasic; Dan Zhang; Fan Yang; Fedor Kouranov; Ian Swett; Janardhan Iyengar; Jeff Bailey; Jeremy Dorfman; Jim Roskind; Joanna Kulik; Patrik Westin; Raman Tenneti; Robbie Shade; Ryan Hamilton; Victor Vasiliev; Wan-Teh Chang; Zhongyi Shi

Crux: GPU-Efficient Communication Scheduling for Deep Learning Training

Proceedings of the ACM SIGCOMM 2024 Conference

IF0

2024-08-04

PREAI

Cao,Jiamin; Guan,Yu; Qian,Kun; Gao,Jiaqi; Xiao,Wencong; Dong,Jianbo; Fu,Binzhang; Cai,Dennis; Zhai,Ennan

Hi-Speed DNN Training with Espresso: Unleashing the Full Potential of Gradient Compression with Near-Optimal Usage Strategies

Proceedings of the Eighteenth European Conference on Computer Systems

IF0

2023-05-08

PREAI

Wang,Zhuang; Lin,Haibin; Zhu,Yibo; Ng,T. S. Eugene

Beyond Throughput and Compression Ratios: Towards High End-to-end Utility of Gradient Compression

IF0

2024-11-18

PREAI

Wenchen Han; Shay Vargaftik; Michael Mitzenmacher; Brad Karp; Ran Ben Basat

BBR: Congestion-Based Congestion Control

Queue

IF0

2016-10-01

OAAI

Neal Cardwell; Yuchung Cheng; C. Stephen Gunn; Soheil Hassas Yeganeh; Van Jacobson

Sparse Communication for Distributed Gradient Descent

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

IF0

2017-01-01

OAAI

Alham Fikri Aji; Kenneth Heafield

DC2: Delay-aware Compression Control for Distributed Machine Learning

IEEE INFOCOM 2021 - IEEE Conference on Computer Communications

IF0

2021-05-10

PREAI

Ahmed M. Abdelmoniem; Marco Canini

PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Transformation and Graph Compilation

Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2

IF0

2024-04-27

PREAI

Jason Ansel; Edward Yang; Horace He; Natalia Gimelshein; Animesh Jain; Michael Voznesensky; Bin Bao; Peter Bell; David Berard; Evgeni Burovski; Geeta Chauhan; Anjali Chourdia; Will Constable; Alban Desmaison; Zachary DeVito; Elias Ellison; Will Feng; Jiong Gong; Michael Gschwind; Brian Hirsh; Sherlock Huang; Kshiteej Kalambarkar; Laurent Kirsch; Michael Lazos; Mario Lezcano; Yanbo Liang; Jason Liang; Yinghai Lu; C. K. Luk; Bert Maher; Yunjie Pan; Christian Puhrsch; Matthias Reso; Mark Saroufim; Marcos Yukio Siraichi; Helen Suk; Shunting Zhang; Michael Suo; Phil Tillet; Xu Zhao; Eikan Wang; Keren Zhou; Richard Zou; Xiaodong Wang; Ajit Mathews; William Wen; Gregory Chanan; Peng Wu; Soumith Chintala

CUBIC

ACM SIGOPS Operating Systems Review

IF0

2008-07-01

PREAI

Sangtae Ha; Injong Rhee; Lisong Xu

Data center TCP (DCTCP)

ACM SIGCOMM Computer Communication Review

IF0

2010-08-30

PREAI

Mohammad Alizadeh; Albert Greenberg; David A. Maltz; Jitendra Padhye; Parveen Patel; Balaji Prabhakar; Sudipta Sengupta; Murari Sridharan

没有更多内容