Mpbs:research on mini-batch partitioning algorithm based on self-organizing map network

Hui Du

doi:10.1007/s00607-024-01362-2

2024-11-15

PRE

DOI：10.1007/s00607-024-01362-2

原文链接

原文求助

摘要

En 中文

Mini-batch partitioning is a widely used technique in deep learning that involves dividing a dataset into smaller subsets. This method is crucial in training deep learning models such as deep neural networks and convolutional neural networks. It is favored for its ability to accelerate model convergence, reduce memory overhead, and minimize convergence errors. The primary advantage of mini-batch partitioning is that it allows the model to learn dataset features more evenly, thereby speeding up the convergence process. However, determining the optimal method and size for mini-batch partitioning remains a challenging problem. This paper proposes a novel mini-batch partitioning method focused on feature reorganization. By leveraging a Self-Organizing Map network for feature extraction, data with similar characteristics are initially grouped into the same batch. The purity index of each batch is then calculated based on the number of features and labels, allowing for a comprehensive evaluation of batch homogeneity. Batches with significant differences in purity are selectively reorganized to ensure that each batch contains a diverse set of features, reducing intra-batch feature correlation and ultimately enhancing data representation.Furthermore, through SOM network mapping, the dataset can be effectively partitioned into subsets that are well-suited for model training. Experimental comparisons of various batch partitioning methods on multiple UCI datasets demonstrate that our proposed method, termed MPBS (Mini-Batch Partitioning Algorithm based on Self-Organizing Map Network). Compared with other algorithms, the accuracy, loss and training time are improved by 14.06%, 24.31% and 31.22%.

Keyword:

Mini-batch partitioning

Self-organizing map

Purity index

Feature reorganization

Enhancing data

Model training