Adding topology and memory awareness in data aggregation algorithms

doi:10.1016/j.future.2024.05.016

Adding topology and memory awareness in data aggregation algorithms

2024-10-01

0

OA

AI

DOI：10.1016/j.future.2024.05.016

Original

Save

View PDF

Abstract

En 中文

With the growing gap between computing power and the ability of large-scale systems to ingest data, I/O is becoming the bottleneck for many scientific applications. Improving read and write performance thus becomes decisive, and requires consideration of the complexity of architectures. In this paper, we introduce TAPIOCA, an architecture-aware data aggregation library. TAPIOCA offers an optimized implementation of the two-phase I/O scheme for collective I/O operations, taking advantage of the many levels of memory and storage that populate modern HPC systems, and leveraging network topology. We show that TAPIOCA can significantly improve the I/O bandwidth of synthetic benchmarks and I/O kernels of scientific applications running on leading supercomputers. For example, on HACC-IO, a cosmology code, TAPIOCA improves data writing by a factor of 13 on nearly a third of the target supercomputer.

Keywords:

Data movement

I/O

Data aggregation

Deep memory and storage hierarchy

Architecture-aware placement

Adding topology and memory awareness in data aggregation algorithms

Abstract

AI Summary

Journal

Organization