Revisiting data imbalance in token-based self-supervised learning

doi:10.1016/j.neucom.2026.133408

Revisiting data imbalance in token-based self-supervised learning

2026-03-25

0

PRE

AI

DOI：10.1016/j.neucom.2026.133408

Original

Original request for help

Save

Abstract

En 中文

• Identifies token-class imbalance as a key issue in token-based SSL. • Demonstrates that rare visual tokens are semantically rich and crucial. • Applies class-balanced loss to mitigate token imbalance in SSL. • Proposes semantic-aware label smoothing utilizing token similarity. • Improves both representation learning and image generation performance.

Keywords:

token-class imbalance

self-supervised learning

semantic-rich tokens

class-balanced loss

label smoothing

Revisiting data imbalance in token-based self-supervised learning

Abstract

Journal

Organization