.png){kind=icon}
.png){kind=icon}
Return
Efficient multiple density-based models over large datasets with data stream applications
.png){kind=icon}
PRE
AI
DOI：10.1016/j.asoc.2025.114019.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Density-based hierarchical clustering offers comprehensive insight into the clusters and outlier structures within datasets through density functions. These algorithms construct a hierarchical graph representation, where edges are weighted by the minimum density required for connected data points to form clusters; this density threshold depends on the minimum number of objects within a neighborhood, denoted mpts, which acts as a smoothing parameter for the density estimate. CORE-SG, a spanning graph for the fast computation of HDBSCAN* results, allows efficient and seamless extraction of multiple hierarchical solutions with varying densities and across an arbitrary range of density smoothing levels, surpassing its predecessors in computational performance. However, much like its predecessors, CORE-SG requires neighborhood estimation based on pairwise similarity calculations that are constrained by a quadratic asymptotic complexity relative to dataset size, which can be impractical for scenarios involving large volumes of data. This paper proposes a streamlined version of CORE-SG, designed to achieve computational efficiency through data abstraction, and investigates the impact of data summarization on the quality of unsupervised hierarchical models across multiple density levels. Our goal is to improve the scalability of CORE-SG while preserving its core properties. We evaluated our approach on clustering and outlier detection tasks, comparing it to the original version. Furthermore, we adapted and evaluated its effectiveness in a data stream scenario, where data arrives continuously and indefinitely, requiring ongoing model updates. Our algorithm was benchmarked against the latest HDBSCAN*-based algorithm for data streams, demonstrating superior performance and improved clustering quality.
Keywords:
Density-based data models
Unsupervised learning
Clustering
Data summarization
Data stream
Big data
Journal
IF:
6.6
Papers:
1.4W
Citations:
4.8W