GraphAr: An Efficient Storage Scheme for Graph Data in Data Lakes

doi:10.14778/3712221.3712223

2025-04-07

PRE

DOI：10.14778/3712221.3712223

原文链接

原文求助

摘要

En 中文

数据湖因其能够存储和分析多种类型数据的能力而日益被采用，通常使用列式存储格式（如Parquet和ORC）来处理关系表。然而，这些传统架构在高效管理图数据方面存在不足，尤其是针对符合Labeled Property Graph（LPG）模型的数据。为解决这一差距，本文介绍了GraphAr，一种专门设计的存储方案，旨在增强现有数据湖以实现高效的图数据管理。GraphAr利用Parquet的优势，精确捕获LPG语义，并支持邻接点检索和标签过滤等图特定操作。通过创新的数据组织、编码和解码技术，GraphAr显著提升了性能。我们的评估表明，GraphAr在邻接点检索方面比传统Parquet和Acero方法快4452倍，在标签过滤方面快14.8倍，在端到端工作负载方面快29.5倍。这些发现突显了GraphAr通过实现高效图数据管理来扩展数据湖实用性的潜力。

Keyword:

BIG DATA

期刊

Proceedings of the VLDB Endowment

IF:

3.3

论文数:

563

被引数:

1.2W

机构

alibaba grp

学者数:

论文数: 24

被引数: 1

引用论文

Exchange and Consumption of Huge RDF Data

Lecture Notes in Computer Science

IF0

2012-01-01

OAAI

Miguel A. Martínez-Prieto; Mario Arias Gallego; Javier D. Fernández

Decoding billions of integers per second through vectorization

Software: Practice and Experience

IF0

2013-05-28

OAAI

D. Lemire; L. Boytsov

Photon: A Fast Query Engine for Lakehouse Systems

Proceedings of the 2022 International Conference on Management of Data

IF0

2022-06-11

PREAI

Alexander Behm; Shoumik Palkar; Utkarsh Agarwal; Timothy Armstrong; David Cashman; Ankur Dave; Todd Greenstein; Shant Hovsepian; Ryan Johnson; Arvind Sai Krishnan; Paul Leventis; Ala Luszczak; Prashanth Menon; Mostafa Mokhtar; Gene Pang; Sameer Paranjpye; Greg Rahn; Bart Samwel; Tom van Bussel; Herman van Hovell; Maryann Xue; Reynold Xin; Matei Zaharia

Big Data, Fast Data and Data Lake Concepts

Procedia Computer Science

IF0

2016-01-01

OAAI

Natalia Miloslavskaya; Alexander Tolstoy

The LDBC Social Network Benchmark

Proceedings of the VLDB Endowment

IF0

2023-02-21

OAAI

Gábor Szárnyas; Jack Waudby; Benjamin A. Steer; Dávid Szakállas; Altan Birler; Mingxi Wu; Yuchen Zhang; Peter Boncz

Modern techniques for querying graph-structured relations 图结构关系查询的现代技术

Proceedings of the VLDB Endowment

IF0

2022-09-29

PREAI

Amine Mhedhbi; Semih Salihoğlu

FlexPushdownDB

Proceedings of the VLDB Endowment

IF0

2021-10-27

PREAI

Yifei Yang; Matt Youill; Matthew Woicik; Yizhou Liu; Xiangyao Yu; Marco Serafini; Ashraf Aboulnaga; Michael Stonebraker

Azure Data Lake Store

Proceedings of the 2017 ACM International Conference on Management of Data

IF0

2017-05-09

OAAI

Raghu Ramakrishnan; Baskar Sridharan; John R. Douceur; Pavan Kasturi; Balaji Krishnamachari-Sampath; Karthick Krishnamoorthy; Peng Li; Mitica Manu; Spiro Michaylov; Rogério Ramos; Neil Sharman; Zee Xu; Youssef Barakat; Chris Douglas; Richard Draves; Shrikant S. Naidu; Shankar Shastry; Atul Sikaria; Simon Sun; Ramarathnam Venkatesan

The LDBC Social Network Benchmark

Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data

IF0

2015-05-27

OAAI

Orri Erling; Alex Averbuch; Josep Larriba-Pey; Hassan Chafi; Andrey Gubichev; Arnau Prat; Minh-Duc Pham; Peter Boncz

Cypher

Proceedings of the 2018 International Conference on Management of Data

IF0

2018-05-27

OAAI

Nadime Francis; Alastair Green; Paolo Guagliardo; Leonid Libkin; Tobias Lindaaker; Victor Marsault; Stefan Plantikow; Mats Rydberg; Petra Selmer; Andrés Taylor

查看更多内容