.png){kind=icon}
.png){kind=icon}
Return
Bidirectional parallel multi-layer multi-scale hybrid network
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2025.132255.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Although Vision Transformer (ViT) can directly process images, the division of images into patches lacks internal interaction within patches and has a single feature scale, leading to suboptimal performance in dense prediction tasks. Most existing research focuses on serial networks that combine the strengths of CNN and ViT to address these issues, but this often disrupts the ViT structure and introduces additional pretraining costs. In this pa per, we propose BiPNet (the Bidirectional Parallel Multi-layer Multi-scale Hybrid Network), which addresses the aforementioned issues by facilitating information interaction between CNNs and Transformers and can directly leverage existing ViT pre-trained weights. Compared to existing methods, our Bidirectional Parallel Multi-Layer Multi-Scale Hybrid Network has the following advantages: 1.The CNN and Transformer are used in parallel to fully retain the ViT architecture, making use of existing pre-trained models. 2.A 3M (multi-layer, multi-scale con volutional module) is proposed to handle the spatial pyramid information of CNNs, addressing the problem of insufficient local feature interaction and single feature representation within ViT. 3.A simple CNN-Transformer BiLGM (bidirectional local-global interaction module) is introduced, which performs both local-global interac tion and balances high-and low-frequency semantics, making it beneficial for handling dense prediction tasks. It achieves 63.9 % APb on COCO val2017 and 62.0 % mIoU on ADE20K with its super-large model without using additional training data.
Keywords:
Hybrid networks
Transformer
Bidirectional
Multi-layer
Backbone
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W