.png){kind=icon}
.png){kind=icon}
Return
fastESN: Fast Echo State Network
.png){kind=icon}
PRE
AI
DOI：10.1109/TNNLS.2022.3167466.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Echo state networks (ESNs) are reservoir computing-based recurrent neural networks widely used in pattern analysis and machine intelligence applications. In order to achieve high accuracy with large model capacity, ESNs usually contain a large-sized internal layer (reservoir), making the evaluation process too slow for some applications. In this work, we speed up the evaluation of ESN by building a reduced network called the fast ESN (fastESN) and achieve an ESN evaluation complexity independent of the original ESN size for the first time. FastESN is generated using three techniques. First, the high-dimensional state of the original ESN is approximated by a low-dimensional state through proper orthogonal decomposition (POD)-based projection. Second, the activation function evaluation number is reduced through the discrete empirical interpolation method (DEIM). Third, we show the directly generated fastESN has instability problems and provide a stabilization scheme as a solution. Through experiments on four popular benchmarks, we show that fastESN is able to accelerate the sparse storage-based ESN evaluation with a high parameter compression ratio and a fast evaluation speed.
Keywords:
Reservoirs
Complexity theory
Training
Recurrent neural networks
Principal component analysis
Biological neural networks
Very large scale integration
Acceleration
complexity reduction
echo state networks (ESNs)
recurrent neural networks (RNNs)
reservoir computing
Journal
IF:
8.9
Papers:
7.5K
Citations:
7.2W