.png){kind=icon}
返回
GRASFormer: A computational framework for gradient-regularized and entropy-stabilized multi-task transformer optimization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.jocs.2026.102805.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Multi-task learning (MTL) with transformer architectures represents a computationally powerful paradigm for joint optimization across related objectives. However, the underlying optimization dynamics remain difficult to stabilize due to inter-task gradient conflicts and excessively sharp attention landscapes, which lead to high-curvature loss surfaces and degraded generalization. This paper introduces GRASFormer, a Gradient Regularized Attention Sharpness Transformer, as a unified computational framework that systematically addresses these challenges through dual regularization. GRASFormer models the interdependence among task gradients via pairwise cosine similarity and dynamically reprojects conflicting directions to achieve smoother multi-objective descent. Concurrently, an entropy-based attention regularizer is formulated to control attention sharpness, thereby ensuring numerically stable and broadly distributed focus across transformer heads. The resulting framework constitutes a generalizable computational model for stable and efficient optimization in multi-task deep learning. Comprehensive experiments on synthetic optimization landscapes and benchmark datasets, including MultiMNIST, FashionMNIST-Edge, CelebA, and SVHN-Length, demonstrate that GRASFormer achieves faster convergence, improved accuracy, and enhanced robustness compared to existing state-of-the-art MTL optimizers. Statistical analyses and ablation diagnostics further confirm the theoretical consistency and computational efficiency of the proposed approach.
期刊
J
IF:
3.7
论文数:
252
被引数:
0
机构
引用论文
Dynamic Split Computing Framework for Multi-Task Learning Models: A Deep Reinforcement Learning Approach.png)
IEEE Access
IEEE AccessIF0
PREAI
分享收藏Sharing to Learn and Learning to Share; Fitting Together Meta, Multi-Task, and Transfer Learning: A Meta Review分享学习和学习分享; 整合元、多任务和迁移学习: 元综述IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏Robust Semantic Segmentation of Wafer Transmission Electron Microscopy Image Using Multi-Task Learning With Edge Detection基于边缘检测的多任务学习在晶圆透射电子显微镜图像的鲁棒语义分割中的应用IEEE AccessOAAI
IEEE AccessIF3.6
OAAI分享收藏分享收藏
查看更多内容