Less is More: Unlabeled Data Selection for Efficient Tabular Self-supervised Learning

doi:10.1007/s10994-026-07133-8

Less is More: Unlabeled Data Selection for Efficient Tabular Self-supervised Learning

2026-08-29

0

OA

AI

S

Sintija Stevanoska *

C

Christian L. Camacho Villalón

DOI：10.1007/s10994-026-07133-8

Original

Save

View PDF

Abstract

En 中文

Self-supervised learning (SSL) addresses label scarcity by leveraging large amounts of unlabeled data to learn transferable representations. However, pretraining on very large unlabeled datasets can be computationally expensive and may include noisy or unrepresentative samples that degrade learning. In this work, we explore whether selecting a subset of the available unlabeled examples for the pretext task can reduce computational cost while maintaining strong performance of SSL methods for tabular data. In particular, we investigate whether uncertainty-, diversity-, and transport-based criteria can guide this selection and improve representation quality. To this end, we conduct large-scale experiments on 25 tabular benchmark datasets using four SSL models, across varying amounts of labeled data under both biased and unbiased label selection, and multiple strategies for sampling unlabeled data. To better understand when and how unlabeled data subsampling is effective, we relate dataset characteristics to experimentally observed performance gains through a meta-analysis. Our results show that reducing the unlabeled data pool yields substantial computational savings while maintaining, and often even improving, downstream performance.

Keywords:

Self-supervised learning

Tabular data

Unlabeled data selection

Active data selection

Less is More: Unlabeled Data Selection for Efficient Tabular Self-supervised Learning

Abstract

AI Summary

Journal

Organization

Cited Papers