Approximate Importance-Based Sampling for Neural Network Training

Piotr Duda

doi:10.1007/978-3-032-03705-3_7

Approximate Importance-Based Sampling for Neural Network Training

2026-01-01

0

PRE

AI

DOI：10.1007/978-3-032-03705-3_7

Original

Original request for help

Save

Abstract

En 中文

This paper introduces a simplified version of the BBATDD algorithm (Boosting-Based Algorithm Trained with Drift Detector), called A-BBATDD, which aims to speed up neural network training. Instead of updating the importance of each training example individually, the new version uses an approximation based on the whole mini-batches. This makes the method easier to use with standard machine learning tools such as TensorFlow or PyTorch. The algorithm still focuses on sampling the most difficult examples for the network and includes a drift detection mechanism (CUSUM) to reset the importance values when the network performance changes significantly. Experiments on the MNIST dataset show that the performance of the method is especially sensitive to the mini-batch size.

Keywords:

Neural network training

sampling methods

drift detection

Approximate Importance-Based Sampling for Neural Network Training

Abstract

Journal

Organization