.png){kind=icon}
.png){kind=icon}
Return
Optimizing large-scale graph database ingestion through edge value ranking: a proposed framework
.png){kind=icon}
OA
AI
DOI：10.3389/fdata.2026.1768571.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
This paper proposes a preprocessing framework for optimizing large-scale graph database ingestion through intelligent edge filtering based on value ranking. We combine adapted PageRank algorithms with business-specific metrics and edge type importance to evaluate and rank edges, enabling selective retention of high-value relationships. The framework introduces three PageRank variants (maximum weight normalization, weighted average, and log-based normalization) with type-specific business value normalization to handle heterogeneous graphs. Current graph database ingestion approaches struggle with scale: loading 6.2TB of data (38 billion objects) requires over 3 weeks, forcing organizations to limit historical data retention. Our approach addresses this through preprocessing-stage filtering before database ingestion. While requiring experimental validation, preliminary analysis suggests potential for 40%-80% data volume reduction depending on graph characteristics, with corresponding improvements in loading efficiency and storage costs. The paper details the theoretical framework, computational complexity analysis, formal property preservation guarantees, and comprehensive validation methodology. This work represents a novel direction in graph database optimization: value-based preprocessing rather than runtime query optimization.
Keywords:
data preprocessing
E-commerce marketplaces
edge ranking
graph databases
graph neural networks
graph optimization
heterogeneous relational graph convolution networks
large datasets