Convolutional spectral kernel learning with generalization guarantees

Jian Li

doi:10.1016/j.artint.2022.103803

Convolutional spectral kernel learning with generalization guarantees

2022-12-01

8

PRE

AI

DOI：10.1016/j.artint.2022.103803

Original

Original request for help

Save

Abstract

En 中文

Kernel methods are powerful tools to capture nonlinear patterns behind given data but often lead to poor performance on complicated tasks compared to convolutional neural networks. The reason is that kernel methods are still shallow and fully connected models, failing to reveal hierarchical features and local interdependencies. In this paper, to acquire hierarchical and local knowledge, we incorporate kernel methods with deep architectures and convolutional operators in a spectral kernel learning framework. Based on the inverse Fourier transform and Rademacher complexity theory, we provide the generalization error bounds for the proposed model and prove that under suitable initialization, deeper networks lead to tighter error bounds. Inspired by theoretical findings, we finally completed the convolutional spectral kernel network (CSKN) with two additional regularizers and an initialization strategy. Extensive ablation results validate the effectiveness of non-stationary spectral kernel, multiple layers, additional regularizers, and the convolutional filters, which coincide with our theoretical findings. We further devise a VGG-type 8-layers CSKN, and it outperforms the existing kernel-based networks and popular CNN models on the medium-sized image classification tasks. (c) 2022 Elsevier B.V. All rights reserved.

Keywords:

Convolutional operator

Non-stationary spectral kernel

Cosine activation

Generalization analysis

Rademacher complexity

Convolutional spectral kernel learning with generalization guarantees

Abstract

Journal

Organization