Geometry-Injected Image-Based Point Cloud Semantic Segmentation

doi:10.1109/TGRS.2023.3264292

Geometry-Injected Image-Based Point Cloud Semantic Segmentation

2023-01-01

5

PRE

AI

DOI：10.1109/TGRS.2023.3264292

Original

Original request for help

Save

Abstract

En 中文

Image-based methods have replicated the success from 2-D domain to 3-D point cloud semantic segmentation. However, when we directly apply 2-D techniques to the projected pseudo-image, inherent differences between the point cloud and the image cause geometric distortion. This article analyzes the geometric distortion between the point cloud and the pseudo-image, including truncation, dislocation, and hole. To ensure geometric fidelity, we propose the Geometry-injected Image-based point cloud semantic segmentation Network (GINet). We design a cyclic convolution to optimize the convolution operation, dealing with truncation. For dislocation and hole, we propose dual geometric constraints, including local spatial attention and local affinity regularization, to incorporate the geometric information into semantic feature learning. Local spatial attention generates an attention map from the point coordinates to modulate the feature map before convolution. Local affinity regularization supervises the semantic similarity of pixels in the convolution kernel range. The GINet rectifies the geometric distortion with these mechanisms while taking advantage of the successful 2-D semantic segmentation methods. Quantitative and qualitative experiments on SemanticKITTI and SemanticPOSS demonstrate the effectiveness of GINet.

Keywords:

Point cloud compression

Three-dimensional displays

Convolution

Semantic segmentation

Feature extraction

Distortion

Semantics

geometric fidelity

point cloud

semantic segmentation

Geometry-Injected Image-Based Point Cloud Semantic Segmentation

Abstract

Journal

Organization