Tabular Data Distillation: An Extensive Comparison

Corneliu Florea

doi:10.3390/make8040084

Return

Tabular Data Distillation: An Extensive Comparison

2026-03-26

Corneliu Florea *

Eduard Barnoviciu

DOI：10.3390/make8040084

Original

Save

View PDF

Abstract

En 中文

In this paper, we present an extensive evaluation of tabular data distillation methods for downstream classification and regression tasks. Our analysis considers multiple distillation approaches that are problem-type independent (i.e., unsupervised). For downstream learners, we focus on non-neural models such as Random Forest, XGBoost, and Support Vector Machines, as our goal is to evaluate the quality of the distilled data independently of the learner. The evaluation is conducted on 17 classification and nine regression problems. Our findings can be summarized as follows: (1) in all cases, applying a distillation method leads to a decrease in performance compared to the baseline; (2) overall, coreset-based methods are the most effective, with performance losses that are minimal—ranging from around 3% in classification accuracy or regression correlation to, in some cases, being negligible; (3) performance loss is moderately correlated with dataset tailness, measured as the proportion of outliers; (4) all distillation methods alter dataset consistency, narrowing the range of hyperparameter values that yield good performance; and (5) the Coreset Leverage Score remains fast, regardless of the size of the original set and of the distilled set.

Keywords:

tabular data distillation

coreset

distribution matching

classification

regression

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.