SAB Net: A Semantic Attention Boosting Framework for Semantic Segmentation

doi:10.1109/TNNLS.2022.3144003

SAB Net: A Semantic Attention Boosting Framework for Semantic Segmentation

2025-03-01

10

PRE

AI

DOI：10.1109/TNNLS.2022.3144003

Original

Original request for help

Save

Abstract

En 中文

Semantic segmentation has achieved great progress by effectively fusing features of contextual information. In this article, we propose an end-to-end semantic attention boosting (SAB) framework to adaptively fuse the contextual information iteratively across layers with semantic regularization. Specifically, we first propose a pixelwise semantic attention (SAP) block, with a semantic metric representing the pixelwise category relationship, to aggregate the nonlocal contextual information. In addition, we improve the computation complexity of SAP block from O(n(2)) to O(n) for images with size n. Second, we present a categorywise semantic attention (SAC) block to adaptively balance the nonlocal contextual dependencies and the local consistency with a categorywise weight, overcoming the contextual information confusion caused by the feature imbalance within intra-category. Furthermore, we propose the SAB module to refine the segmentation with SAC and SAP blocks. By applying the SAB module iteratively across layers, our model shrinks the semantic gap and enhances the structure reasoning by fully utilizing the coarse segmentation information. Extensive quantitative evaluations demonstrate that our method significantly improves the segmentation results and achieves superior performance on the PASCAL VOC 2012, Cityscapes, PASCAL Context, and ADE20K datasets.

Keywords:

Semantics

Boosting

Image segmentation

Aggregates

Measurement

Context modeling

Fuses

Contextual dependencies

feature fusion

semantic attention

semantic attention boosting network (SAB Net)

semantic segmentation

SAB Net: A Semantic Attention Boosting Framework for Semantic Segmentation

Abstract

Journal

Organization