.png){kind=icon}
.png){kind=icon}
Return
Layerwise-priority-based gradient adjustment for few-shot learning
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2025.127053.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
To resolve the problem that deep neural networks (DNN) require a large amount of data, few-shot learning has been studied in many ways. Model-Agnostic Meta-Learning (MAML), one of the successful methods in gradient- based meta-learning, consists of the inner loop adapting new tasks by learning task-specific knowledge and the outer loop finding meta-initialization primed for rapid learning. Recent work BOIL (Body Only update in Inner Loop) tried to control the inner loop by freezing the classifier (last layers) to achieve representation change while ANIL (Almost No Inner Loop) only updates the classifier for feature reuse. We bring our intuition about adapting to new tasks from previous works that the degree of weight change in different layers differs a lot. We define the priority as the degree of change in each layer and propose a novel priority-based adaptation to adapt to new tasks named Gradient Adjustment in Inner Loop (GAIL) by multiplying the per-layer priority to the corresponding layer's gradient vector. We figure out that adjusted gradients with GAIL in the inner loop for different tasks have similar directions and compared to the original gradient, the mean direction of the adjusted gradients for different tasks from GAIL is statistically better aligned with the direction of the global solution in multiple tasks with a convex quadratic loss function. Experimental results show that GAIL has a faster and better convergence behavior than previous inner loop variation methods. The code is provided in here https://drive.google.com/file/d/1zkn2M_VEXRzzdBediIrJz4PhXN8uYEJ1/view?usp=sharing.
Keywords:
Meta-learning
Representation change
Gradient adjustment
Convolutional neural networks
Journal
IF:
7.5
Papers:
2.9W
Citations:
10.2W