.png){kind=icon}
.png){kind=icon}
Return
Efficient local intrinsic dimensionality estimation in evolving deep representations
.png){kind=icon}
PRE
AI
DOI：10.1016/j.is.2026.102795.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Local intrinsic dimensionality (LID) provides insight into the behavior of individual training points in deep neural networks, with applications that include adversarial detection, prevention of dimensional collapse in self-supervised learning, and identification of untruthful responses from large language models (LLMs). In such contexts, efficient LID estimation often relies on the use of mini-batches, due to the high cost of computing neighborhoods in latent space. However, estimation with respect to small subsets of the training data usually reflects the dimensionality of the global manifold structure rather than the intended local distribution around each point. In this paper, we propose the Nearest-Distance Cache (NDC), a method that improves the locality of LID estimation by reusing nearest-neighbor distances observed in past mini-batches. This strategy faces two key challenges: representations evolve over time, and limited memory prevents storing all past distances. To address these issues, NDC maintains a compact cache of nearest distances per example and uses window-based change detection to discard outdated samples affected by distributional drift. We evaluate two variants of our framework, one that explicitly maintains a temporal history of distances within a sliding window, and the other based on a randomized cache eviction strategy. We also develop a new parametric statistical test designed specifically to detect changes in the lower tail of the distance distribution, enabling more sensitive and computationally efficient detection of drift in the distance values used for LID estimation. Experiments on both synthetic and real data show that by effectively handling temporal distribution shifts, NDC produces higher quality LID estimates than methods that rely solely on neighborhoods computed within mini-batches, while incurring only minor computational overhead.
Keywords:
Local intrinsic dimensionality
Deep representations
Nearest distance cache
Distributional drift detection
Journal
I
IF:
3.4
Papers:
117
Citations:
0
Organization
No organization information available