SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

doi:10.1109/TNNLS.2024.3446171

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

2024-01-01

0

OA

AI

DOI：10.1109/TNNLS.2024.3446171

Original

Original request for help

Save

Abstract

En 中文

Catastrophic forgetting is a phenomenon in which a neural network, upon learning a new task, struggles to maintain its performance on previously learned tasks. It is a common challenge in the realm of continual learning (CL) through neural networks. The mammalian brain addresses catastrophic forgetting by consolidating memories in different parts of the brain, involving the hippocampus and the neocortex. Taking inspiration from this brain strategy, we present a CL framework that combines a plastic model simulating the fast learning capabilities of the hippocampus and a stable model representing the slow consolidation nature of the neocortex. To supplement this, we introduce a variational autoencoder (VAE)-based pseudo memory for rehearsal purposes. In addition by applying lateral inhibition masks on the gradients of the convolutional layer, we aim at damping the activity of adjacent neurons and introduce a sleep phase to reorganize the learned representations. Empirical evaluation demonstrates the positive impact of such additions on the performance of our proposed framework; we evaluate the proposed model on several class-incremental and domain-incremental datasets and compare it with the standard benchmark algorithms, showing significant improvements. With the aim to showcase practical applicability, we implement the algorithm in a physical environment for object classification using a soft pneumatic gripper. The algorithm learns new classes incrementally in real time and also exhibits significant backward knowledge transfer (KT).

Keywords:

Task analysis

Training

Heuristic algorithms

Data models

Brain modeling

Real-time systems

Feature extraction

Catastrophic forgetting

complementary learning system (CLS)

continual learning (CL)

perception

pseudo episodic memory

soft gripper

variational autoencoder (VAE)

SynapNet: A Complementary Learning System Inspired Algorithm With Real-Time Application in Multimodal Perception

Abstract

Journal

Organization