.png){kind=icon}
.png){kind=icon}
Return
Finite Sample Based Mutual Information
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2021.3107031.png)
.png){kind=icon}
.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Mutual information is a popular metric in machine learning. In case of a discrete target variable and a continuous feature variable the mutual information can be calculated as a sum-integral of weighted log likelihood ratio of joint and marginal density distributions. However, in practice the true density distributions are unavailable and only a finite sample of the population is given. In this paper, we propose a novel method for calculating the mutual information for continuous variables using a finite sample of the population. The proposed method is based on approximating the underlying continuous density distribution using Kernel Density Estimation. Unlike previous kernel-based approaches for estimating mutual information, our method calculates directly the integral involved in the formula. Numerical experiments demonstrate that the proposed method produces more accurate results than the currently used feature selection approaches. In addition, our method demonstrates substantially faster computation times than the benchmark methods.
Keywords:
Mutual information
Statistics
Sociology
Kernel
Entropy
Machine learning
Feature extraction
Continuous variable
feature evaluation
feature selection
finite sample
kernel density estimation
mutual information
AI Summary
Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.
Journal
IF:
3.6
Papers:
9.8W
Citations:
29.4W