.png){kind=icon}
.png){kind=icon}
Return
Label smoothing regularization-based no hyperparameter domain generalization
.png){kind=icon}
PRE
AI
DOI：10.1016/j.knosys.2024.112877.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Domain generalization learns from one or multiple source domains. It aims to extract a domain-invariant model that can be employed in an unknown target domain. During network training, the models may become overly reliant on the distribution of the training datasets, ultimately leading to an inability to capture the appropriate patterns present in the unseen data accurately. To alleviate these issues, we introduce weighted label smoothing regularization, which reduces the confidence of the model in the correct category and increases the confidence in other categories to mitigate overfitting. More importantly, the proposed weighted label smoothing regularization allows the model to focus more on the relative relationships between categories rather than on absolute differences, which helps the model learn general features and improves its ability to generalize to new data. After introducing label smoothing regularization, we observe that the gradient signals of the label loss and regularization loss are antagonistic to each other, which may suggest that they correspond to different optimal solutions. A reasonable explanation for this is that label smoothing regularization focuses on the generalization performance of the model, whereas the label loss is more concentrated on the accuracy under the current data distribution. To balance this relationship, we propose weight adaptation. Weight adaptation adjusts the weights adaptively based on the gradient performance and the importance of each loss without manually setting a hyperparameter. This allows the model to allocate more attention to the more crucial loss in the optimization process while reducing the influence of the less important loss. Additionally, by dynamically adjusting the weights during training, weight adaptation can enhance model robustness and prevent overfitting. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed method.
Keywords:
Domain generalization
Label smoothing regularization
Weight adaptation
Journal
K
IF:
7.6
Papers:
1.2W
Citations:
4.5W