Online Learning From Evolving Feature Spaces With Deep Variational Models

doi:10.1109/TKDE.2023.3326365

Online Learning From Evolving Feature Spaces With Deep Variational Models

2024-08-01

0

PRE

AI

DOI：10.1109/TKDE.2023.3326365

原文链接

原文求助

摘要

En 中文

In this paper, we explore a novel online learning setting, where the online learners are presented with doubly-streaming data. Namely, the data instances constantly streaming in are described by feature spaces that over-time evolve, with new features emerging and old features fading away. The main challenge of this problem lies in the fact that the newly emerging features are described by very few samples, resulting in weak learners that tend to make error predictions. A seemingly plausible idea to overcome the challenge is to establish a relationship between the old and new feature spaces, so that an online learner can leverage the knowledge learned from the old features to better the learning performance on the new features. Unfortunately, this idea does not scale up to high-dimensional feature spaces that entail very complex feature interplay. Specifically. a tradeoff between onlineness, which biases shallow learners, and expressiveness, which requires deep models, is inevitable. Motivated by this, we propose a novel paradigm, named Online Learning Deep models from Data of Double Streams ((OLDS)-S-3), where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building an intermediate feature mapping relationship. A key trait of (OLDS)-S-3 is to treat the model capacityas a learnable semantics, aiming to yield optimal model depth and parameters jointly in accordance with the complexity and non-linearity of the input data streams in an online fashion. To ablate its efficacy and applicability, two variants of (OLDS)-S-3 are proposed namely, OLD-Linear that learns the relationship by a linear function; and OLD-FD learns that two consecutive feature spaces pre-and-post evolution with fixed deep depth. Besides, instead of re-starting the entire learning process from scratch, (OLDS)-S-3 learns multiple newly emerging feature spaces in a lifelong manner, retaining the knowledge from the learned and vanished feature space to enjoy a jump-start of the new features' learning process. Both theoretical analysis and empirical studies substantiate the viability and effectiveness of our proposed approach.

Keyword:

Data models

Streams

Convergence

Training

Adaptation models

Data mining

Image reconstruction

Data streams

online learning

streaming algorithms

open feature spaces

Online Learning From Evolving Feature Spaces With Deep Variational Models

摘要

期刊

机构

引用论文