Gradient-based multi-label feature selection considering three-way variable interaction

Yizhang Zou

doi:10.1016/j.patcog.2023.109900

2024-01-01

PRE

Yizhang Zou

Xuegang Hu

李培培 (Peipei Li) *

DOI：10.1016/j.patcog.2023.109900

原文链接

原文求助

摘要

En 中文

Nowadays, Multi-Label Feature Selection (MLFS) attracts more and more attention to tackle the high dimensional problem in multi-label data. A key characteristic of existing gradient-based MLFS methods is that they typically consider two-way variable correlations between features and labels, including feature-feature and label-label correlations. However, two-way correlations are not sufficient to steer feature selection since such correlations vary given different additional variables in practical scenarios, which leads to the selected features with relatively-poor classification performance. Motivated by this, we capture three-way variable interactions including feature-feature-label and feature-label-label interactions to further characterize the fluctuating correlations in the context of another variable, and propose a new gradient-based MLFS approach incorporating the above three-way variable interactions into a global optimization objective. Specifically, based on information theory, we develop second-order regularization penalty terms to regard three-way interactions while jointly combining with the main loss term in regard to feature relevance. Then the objective function can be efficiently optimized via a block-coordinate gradient descent schema. Meanwhile, we provide a theoretical analysis demonstrating the effectiveness of the regularization terms in exploiting three-way interaction. In addition, experiments conducted on a series of benchmark data sets also verify the validity of the proposed method on multiple evaluation metrics.

Keyword:

Multi-label learning

Feature selection

Gradient-based methods

Three-way interaction

Information theory