.png){kind=icon}
返回
Fluid-Shuttle: Efficient Cloud Data Transmission Based on Serverless Computing Compression
.png){kind=icon}
PRE
AI
DOI：10.1109/TNET.2024.3402561.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Nowadays, there exists a lot of cross-region data transmission demand on the cloud. It is promising to use serverless computing for data compressing to save the total data size. However, it is challenging to estimate the data transmission time and monetary cost with serverless compression. In addition, minimizing the data transmission cost is non-trivial due to the enormous parameter space. This paper focuses on this problem and makes the following contributions: 1) We propose empirical data transmission time and monetary cost models based on serverless compression. It can also predict compression information, e.g., ratio and speed using chunk sampling and machine learning techniques. 2) For single-task cloud data transmission, we propose two efficient parameter search methods based on Sequential Quadratic Programming (SQP) and Eliminate then Divide and Conquer (EDC) with proven error upper bounds. Besides, we propose a parameter fine-tuning strategy to deal with transmission bandwidth variance. 3) Furthermore, for multi-task scenarios, a parameter search method based on dynamic programming and numerical computation is proposed. We have implemented the system called Fluid-Shuttle, which includes straggler optimization, cache optimization, and the autoscaling decompression mechanism. Finally, we evaluate the performance of Fluid-Shuttle with various workloads and applications on the real-world AWS serverless computing platform. Experimental results show that the proposed approach can improve the parameter search efficiency by over 3 $\times$ compared with the state-of-art methods and achieves better parameter quality. In addition, our approach achieves higher time efficiency and lower monetary cost compared with competing cloud data transmission approaches.
Keyword:
Data transmission
serverless compression
cloud function configuration
Data transmission
serverless compression
cloud function configuration
期刊
I
IF:
3.6
论文数:
4.4K
被引数:
9.5K
机构
引用论文
.png){kind=icon}
Lipase as a green and sustainable material for production of levulinate compounds: State of the art脂肪酶作为生产乙酰丙酸化合物的一种绿色和可持续材料：最新研究进展
OAAI
OAAI分享收藏
查看更多内容