Binary kernel logistic regression: A sparsity-inducing formulation and a convergent decomposition training algorithm

Antonio Consolo

doi:10.1016/j.cor.2025.107363

Return

Binary kernel logistic regression: A sparsity-inducing formulation and a convergent decomposition training algorithm

2025-12-24

Antonio Consolo *

Andrea Manno

E. Amaldi

DOI：10.1016/j.cor.2025.107363

Original

Save

View PDF

Abstract

En 中文

Kernel logistic regression (KLR) is a widely used supervised learning method for binary and multi-class classification, which provides estimates of the conditional probabilities of class membership for the data points. Unlike other kernel methods such as Support Vector Machines (SVMs), KLRs are generally not sparse. Previous attempts to deal with sparsity in KLR include a heuristic method referred to as the Import Vector Machine (IVM) and ad hoc regularizations such as the ℓ1/2-based one. Achieving a good trade-off between prediction accuracy and sparsity is still a challenging issue with a potential significant impact from the application point of view. In this work, we revisit binary KLR and propose an extension of the training formulation proposed by Keerthi et al., which is able to induce sparsity in the trained model, while maintaining good testing accuracy. To efficiently solve the dual of this formulation, we devise a decomposition algorithm of Sequential Minimal Optimization type which exploits second-order information, and for which we establish global convergence. Numerical experiments conducted on 12 datasets from the literature show that the proposed binary KLR approach achieves a competitive trade-off between accuracy and sparsity with respect to IVM, ℓ1/2-based regularization for KLR, and SVM while retaining the advantages of providing informative estimates of the class membership probabilities.

Keywords:

Machine learning

Kernel logistic regression

Sparsity

Decomposition methods

Nonlinear programming

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.