.png){kind=icon}
.png){kind=icon}
Return
Density Distillation for Fast Nonparametric Density Estimation
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2022.3160939.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Nonparametric density estimation has been extensively used in various application scenarios and theoretical models. However, the modeling of these powerful methods is inseparable from the sample data and comes at the cost of repeated and intensive kernel calculations, which makes their efficiency greatly affected by the sample scale, data dimension, and evaluation scale. Inspired by the knowledge distillation method, a student-teacher paradigm model named density convolutional neural network (DCNN) is proposed in this article. The method extracts the density knowledge of the samples based on the density convolution rule and transfers it to a compact and small deep neural network, in order to separate the sample data from the modeling and avoid the cumbersome kernel calculations. Experimental results show the superiority of the proposed method to various nonparametric estimation methods in terms of accuracy, stability, processing efficiency, and low-storage advantage. Especially, for the estimation speed, a univariate density estimation on 1.0E + 08 evaluation points using GPU only takes 1.57 s, and a 10-D multivariate density estimation on 1.0E + 08 evaluation points only takes 10.50 s, which makes our method very suitable for real-time and large-scale repetitive density estimation tasks.
Keywords:
Estimation
Kernel
Convolution
Feature extraction
Data models
Computational modeling
Bandwidth
Deep learning
knowledge distillation (KD)
model compression and acceleration
multivariate nonparametric density estimation
Journal
IF:
8.9
Papers:
7.5K
Citations:
7.2W