.png){kind=icon}
返回
HiM: hierarchical multimodal network for document layout analysis
.png){kind=icon}
PRE
AI
DOI：10.1007/s10489-023-04782-3.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In document layout analysis, both computer vision based and natural language processing based methods are employed individually or integrated to enrich the feature information resources and to enforce object detection. To simultaneously leverage visual and textual modalities, this paper proposes a hierarchical multimodal (HiM) network to aggregate representative features from multi-source inputs with the introduction of complementary semantics and non-local context dependencies across grained scales. Different channel and spatial attention mechanisms are adapted to different modalities. The visual modality is based on conventional convolution network, while the textual modality focuses on embedding hierarchical textual vectors and positioning. The feature representations from multiple modalities are then integrated adaptively in feature pyramid network for subsequent region proposal processing. We have made database adaptation on PubLayNet, including inserting semi-structure elements and extending ground truth annotations by parsing PDF pages. On three popular benchmarks, including Article Regions, PubLayNet and DocBank, extensive experiments are carried out to verify the effectiveness and adaptability of HiM.
Keyword:
Multimodal
Object detection
Document layout analysis
Deep learning
期刊
IF:
3.5
论文数:
7.6K
被引数:
1.7W
机构
引用论文
分享收藏Sodium channel kinetic changes that produce Brugada syndrome or progressive cardiac conduction system disease产生Brugada综合征或进行性心脏传导系统疾病的钠通道动力学变化
PREAI
分享收藏X-ray scattering as a probe for warm dense mixtures and high-pressure miscibilityX射线散射作为温暖致密混合物和高压混溶性的探针
OAAI
OAAI分享收藏Thermal comfort, perceived air quality, and cognitive performance when personally controlled air movement is used by tropically acclimatized persons当热带适应的人使用个人控制的空气运动时，热舒适性，感知的空气质量和认知表现.png)
Indoor Air
OAAI
Indoor AirIF0
OAAI分享收藏
查看更多内容