.png){kind=icon}
返回
Modality adaptation in multimodal data
.png){kind=icon}
PRE
AI
DOI：10.1016/j.eswa.2021.115126.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Recently, multimodal data has received much attention. In classical machine learning, it is assumed that all data comes from one modality while in multimodal machine learning, the information comes from different modalities. In multimodal machine learning, transiting, or fusing knowledge from different modalities is an important step. Hence, in these steps, the different marginal distributions between different modalities should be taken into account. However, in recent years, modality adaptation has not gotten enough attention. The motivation of this work is to consider modality adaptation to effectively encode the shared common or complementary knowledge in multimodal data. To reduce the modality shift, we present a new perspective on the modality adaptation algorithm. In multimodal data, by applying the existing domain adaptation techniques to reduce the modality shift, a problem arises because of the insufficient capability of those techniques in preserving complementary knowledge. Our proposed modality adaptation is designed such that it simultaneously considers both the shared and complementary knowledge of each modality while preserving the discriminative ability of each modality in the label space. To evaluate the proposed approach, we have applied it to two different multimodal applications: multi-view object detection and RGBD image semantic segmentation. Our results show that the proposed modality adaptation technique is successful in transferring and fusing knowledge.
Keyword:
Modality adaptation
Multimodal learning
Knowledge transferring
Knowledge fusion
AI总结
对已上传原文的论文进行重点信息的提取，主要内容包括：简要概述、研究摘要、背景介绍、关键亮点、图文解析、展望与总结。
期刊
IF:
7.5
论文数:
2.9W
被引数:
10.2W
机构
引用论文
RGB-D Scene Recognition via Spatial-Related Multi-Modal Feature Learning基于空间相关多模态特征学习的rgb-d场景识别.png)
IEEE ACCESS
OAAI
IEEE ACCESSIF3.6
OAAI分享收藏分享收藏分享收藏分享收藏Deep Symmetric Adaptation Network for Cross-Modality Medical Image Segmentation用于交叉模态医学图像分割的深度对称自适应网络
OAAI
OAAI分享收藏
查看更多内容