.png){kind=icon}
返回
Label smoothing regularization-based no hyperparameter domain generalization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2024.112877.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Domain generalization learns from one or multiple source domains. It aims to extract a domain-invariant model that can be employed in an unknown target domain. During network training, the models may become overly reliant on the distribution of the training datasets, ultimately leading to an inability to capture the appropriate patterns present in the unseen data accurately. To alleviate these issues, we introduce weighted label smoothing regularization, which reduces the confidence of the model in the correct category and increases the confidence in other categories to mitigate overfitting. More importantly, the proposed weighted label smoothing regularization allows the model to focus more on the relative relationships between categories rather than on absolute differences, which helps the model learn general features and improves its ability to generalize to new data. After introducing label smoothing regularization, we observe that the gradient signals of the label loss and regularization loss are antagonistic to each other, which may suggest that they correspond to different optimal solutions. A reasonable explanation for this is that label smoothing regularization focuses on the generalization performance of the model, whereas the label loss is more concentrated on the accuracy under the current data distribution. To balance this relationship, we propose weight adaptation. Weight adaptation adjusts the weights adaptively based on the gradient performance and the importance of each loss without manually setting a hyperparameter. This allows the model to allocate more attention to the more crucial loss in the optimization process while reducing the influence of the less important loss. Additionally, by dynamically adjusting the weights during training, weight adaptation can enhance model robustness and prevent overfitting. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method.
Keyword:
Domain generalization
Label smoothing regularization
Weight adaptation
期刊
K
IF:
7.6
论文数:
1.3W
被引数:
4.5W
机构
引用论文
First-Principles Calculations of Acoustic and Anharmonic Properties of Ferromagnetic Cu2MnZ (Z = Al and In) Heusler Alloys铁磁Cu2MnZ (Z = Al和In) Heusler合金的声学和非谐特性的第一性原理计算
PREAI
分享收藏Duplex adversarial domain discriminative network for cross-domain partial transfer fault diagnosis用于跨域部分转移故障诊断的双工对抗域判别网络
PREAI
分享收藏A cancer pharmacogenomic screen powering crowd-sourced advancement of drug combination prediction.png)
OAAI
IF0
OAAI分享收藏
分享收藏A Dual-View Style Mixing Network for unsupervised cross-domain fault diagnosis with imbalanced data用于不平衡数据无监督跨域故障诊断的双视图混合网络
PREAI
分享收藏
查看更多内容