Semantic-aware self-supervised depth estimation for stereo 3D detection

Hanqing Sun

doi:10.1016/j.patrec.2023.02.006

2023-03-01

PRE

Hanqing Sun

Jiale Cao

Yanwei Pang *

DOI：10.1016/j.patrec.2023.02.006

原文链接

原文求助

摘要

En 中文

Besides the 3D object supervision, the auxiliary disparity supervision is usually indispensable when train-ing a stereo-based 3D object detector. The disparity supervision is either transformed from LiDAR points or generated from pre-trained models. However, the former suffers from the high cost and over-sensitivity to airborne particles of LiDAR devices, and the latter from the limited cross-dataset transferability of con-temporary stereo matching models. To alleviate those problems, we propose a self-supervision frame-work for stereo-based 3D detection that relies on neither LiDARs nor external models. A Depth-based Self-supervision (DSelf) is proposed to unify the coordinate spaces of self-supervised losses and detection into a 3D space. However, the DSelf supervision is dense compared with the sparse LiDAR points, which introduces redundancy and irrelevancy into the stereo features. A Semantic-Aware Sampler (SASampler) is proposed to address the problems by an unbalanced sampling of foreground and background pixels. Combining our SASampler and DSelf supervision, the resultant detector (named S3D) achieves state-of-the-art detection results without explicit disparity supervisions. (c) 2023 Published by Elsevier B.V.

Keyword:

Stereo vision

3D object detection

Self-supervised learning

Depth estimation

期刊

Pattern Recognition Letters

IF:

3.3

论文数:

8.0K

被引数:

1.6W

机构

tianjin university

学者数:

8.0W

论文数: 5.8W

被引数: 88

引用论文

Self-Monitoring of Blood Glucose in Type 2 Diabetes: Recent Studies

Journal of Diabetes Science and Technology

IF0

2013-03-01

OAAI

Oliver Schnell; Hasan Alawi; Tadej Battelino; Antonio Ceriello; Peter Diem; Anne-Marie Felton; Wladyslaw Grzeszczak; Kari Harno; Peter Kempler; Ilhan Satman; Bruno Vergès

Changes in the water status and osmotic solute contents in response to drought and salicylic acid treatments in four different cultivars of wheat (Triticum aestivum)

Journal of Plant Research

IF0

2011-03-29

PREAI

Naglaa Loutfy; Mohamed A. El-Tayeb; Ahmed M. Hassanen; Mahmoud F. M. Moustafa; Yoh Sakuma; Masahiro Inouhe

Unsupervised Stereo Matching Using Confidential Correspondence Consistency

IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS

IF8.4

2020-05-01

PREAI

Joung, Sunghun; Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon