An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition

Qingsong Wang

doi:10.1007/s10915-025-03067-w

Return

An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition

2025-09-27

PRE

DOI：10.1007/s10915-025-03067-w

Original

Original request for help

Save

Abstract

En 中文

Despite the remarkable success of low-rank estimation in data mining, its effectiveness diminishes when applied to sparse non-negative data that inherently lacks low-rank structure. To address this limitation, in this paper, we focus on non-negative sparse matrices and aim to investigate the intrinsic low-rank characteristics of the rectified linear unit (ReLU) activation function. We first propose a novel nonlinear matrix decomposition framework incorporating a comprehensive regularization term designed to simultaneously promote useful structures in clustering and compression tasks, such as low-rankness, sparsity, and non-negativity in the resulting factors. This formulation presents significant computational challenges due to its multi-block structure, non-convexity, non-smoothness, and the absence of global gradient Lipschitz continuity. We address these issues by developing an accelerated alternating partial Bregman proximal gradient method (AAPB), whose distinctive feature lies in its capability to enable simultaneous updates of multiple variables. Under mild and theoretically justified assumptions, we establish both sublinear and global convergence properties of the proposed algorithm. Through careful selection of kernel generating distances tailored to various regularization terms, we derive corresponding closed-form solutions while maintaining the L-smooth adaptable property. Numerical experiments on graph regularized clustering and sparse NMF basis compression confirm the effectiveness of our model and algorithm.

Keywords:

Low-rank

ReLU activation function

Alternating minimization

Bregman method

Acceleration