.png){kind=icon}
返回
Direct Feedback Learning With Local Alignment Support
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3409819.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
While backpropagation (BP) algorithm has been pivotal in enabling the success of modern deep learning technologies, it encounters challenges related to computational inefficiency and biological implausibility. Especially, the sequential propagation of error signals using forward weights in BP is not biologically plausible and prevents efficient parallel updates of learning parameters. To solve these problems, the direct feedback alignment (DFA) method is proposed to directly propagate the error signal from output layer to each hidden layer through random feedback weight, but the performance of DFA is still not competent to BP, especially in complicate tasks with large number of outputs and the convolutional neural network models. In this paper, we propose a method to adjust the feedback weights in DFA using additional local modules that are connected to the hidden layers. The local module attached to each hidden layer has a single-layer structure and learns to mimic the final output of the network. Then, the weights of a local module behave like a direct path connecting each hidden layer to the network output, which has an inverse relationship to the direct feedback weights of DFA. We use this relationship to update the feedback weight of DFA. From the experimental investigation, we confirm that the proposed adaptive feedback weights improve the alignment of the error signal of DFA with that of BP. Furthermore, comparative experiments show that the proposed method significantly outperforms the original DFA on well-known benchmark datasets. The code used for the experiments is available at https://github.com/leibniz21c/direct-feedback-learning-with-local-alignment-support.
Keyword:
Vectors
Biological information theory
Backpropagation
Benchmark testing
Convolutional neural networks
Training
Task analysis
Feedback
biologically plausible learning
random feedback weight
direct feedback alignment
local alignment support module
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
引用论文
BNIP3 Regulates AT101 [(-)-Gossypol] Induced Death in Malignant Peripheral Nerve Sheath Tumor Cells.png)
PLoS ONEOAAI
PLoS ONEIF0
OAAI分享收藏Investigation of Ti mesh-supported anodes for direct borohydride fuel cells直接硼氢化物燃料电池用钛网支撑阳极的研究
PREAI
分享收藏
Two-dimensional bricklayer arrangements of tolans using halogen bonding interactions使用卤素键相互作用的tolans的二维瓦工层布置
PREAI
分享收藏LAFD: Local-Differentially Private and Asynchronous Federated Learning With Direct Feedback AlignmentIEEE ACCESSOAAI
IEEE ACCESSIF3.6
OAAI分享收藏Developing Wind and/or Solar Powered Crop Irrigation Systems for the Great Plains为大平原开发风能和/或太阳能作物灌溉系统
PREAI
分享收藏
查看更多内容