Robust-DS: A Robust Multi-Sensor Multi-Task Learning Framework for Depth Estimation and Semantic Segmentation

Cheng Feng

doi:10.1109/tcsvt.2026.3737613

Robust-DS: A Robust Multi-Sensor Multi-Task Learning Framework for Depth Estimation and Semantic Segmentation

2026-09-25

0

PRE

AI

DOI：10.1109/tcsvt.2026.3737613

Original

Original request for help

Save

Abstract

En 中文

While depth estimation and semantic segmentation are common tasks in autonomous systems, specialized multi-sensor-based frameworks for low-cost systems are still lacking. Additionally, multi-sensor-based methods often suffer from sensor measurement misalignment and sensor failures, which limit their practical applications. In this paper, we propose Robust-DS, a baseline framework tailored for autonomous systems. It integrates a confidence-guided depth consistency learning scheme and a robust training strategy to address sensor measurement misalignment and failures. Robust-DS processes data from a single image and range sensor, using two encoders to extract features, which are then merged in a convolutional gated recurrent unit fusion module. This setup supports multi-task predictions through various decoder heads. To tackle sensor measurement misalignment and failure, we implement the confidence-guided scheme to reduce the effect of unreliable range measurements and a robust training strategy for complete failure of one sensor while the other sensor remains available. We evaluate our proposed method on four datasets with different sensor inputs, demonstrating consistent performance improvements over state-of-the-art baselines.

Keywords:

Multi-sensor

Depth Estimation

Semantic Segmentation

Sensor failures

Robust training

Robust-DS: A Robust Multi-Sensor Multi-Task Learning Framework for Depth Estimation and Semantic Segmentation

Abstract

Journal

Organization

Cited Papers