Realizing Data Features by Deep Nets

Zheng-Chu Guo

doi:10.1109/TNNLS.2019.2951788

Realizing Data Features by Deep Nets

2020-10-01

12

OA

AI

DOI：10.1109/TNNLS.2019.2951788

Original

Save

View PDF

Abstract

En 中文

This article considers the power of deep neural networks (deep nets) in realizing data features. Based on refined covering number estimates, we find that, to realize data features such as the locality, rotation invariance, and manifold structure, deep nets essentially improve the performances of shallow neural networks (shallow nets) without requiring additional capacity costs. Conversely, to realize some data features, such as the smoothness, we show that deep nets perform similar as shallow nets, provided the depth is not extremely large. Both sides show the advantages and limitations of deep nets in realizing data features and demonstrate that deep nets are not always better than shallow nets.

Keywords:

Neural networks

Machine learning

Manifolds

Computer vision

Sparse matrices

Learning systems

Approximation rates

covering numbers

data feature

deep neural networks (deep nets)

neural networks

Realizing Data Features by Deep Nets

Abstract

AI Summary

Journal

Organization