.png){kind=icon}
返回
Robust Semantic Segmentation With Multi-Teacher Knowledge Distillation
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2021.3107841.png)
.png){kind=icon}
.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recent studies have recently exploited knowledge distillation (KD) technique to address time-consuming annotation task in semantic segmentation, through which one teacher trained on a single dataset could be leveraged for annotating unlabeled data. However, in this context, knowledge capacity is restricted, and knowledge variety is rare in different conditions, such as cross-model KD, in which the single teacher KD prohibits the student model from distilling information using cross-domain context. In this study, we aim to train a robust, lightweight student under the supervision of several expert teachers, which provide better instructive guidance compared to a single student-teacher learning framework. To be more specific, we first train five distinct convolutional neural networks (CNNs) as teachers for semantic segmentation on several datasets. To this end, several state-of-the-art augmentation transformations have also been utilized in training phase of our teachers. The impacts of such training scenarios are then assessed in terms of student robustness and accuracy. As the main contribution of this paper, our proposed multi-teacher KD paradigm endows the student with the ability to amalgamate and capture a variety of knowledge illustrations from different sources. Results demonstrated that our method outperforms the existing studies on both clean and corrupted data in the semantic segmentation task while benefiting from our proposed score weight system. Experiments validate that our multi-teacher framework results in an improvement of 9% up to 32.18% compared to the single-teacher paradigm. Moreover, it is demonstrated that our paradigm surpasses previous supervised real-time studies in the semantic segmentation challenge.
Keyword:
Robustness
Semantics
Data models
Training
Task analysis
Computational modeling
Knowledge engineering
Autonomous vehicles
convolutional neural networks
knowledge distillation
semantic segmentation
semi-supervised learning
corruption robustness
machine vision
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
3.6
论文数:
9.8W
被引数:
29.4W
机构
暂无机构信息
引用论文
分享收藏Gross primary production of Mediterranean watersheds: Using isotope mass balance approach to improve estimations.png)
EcohydrologyOAAI
EcohydrologyIF0
OAAI分享收藏分享收藏
查看更多内容