Leveraging Stable Diffusion for Monocular Depth Estimation via Image Semantic Encoding

doi:10.1007/978-3-032-01486-3_5

2026-01-01

PRE

DOI：10.1007/978-3-032-01486-3_5

原文链接

原文求助

摘要

En 中文

单目深度估计涉及从单幅RGB图像预测深度，在自动驾驶、机器人导航、3D重建等应用中起着关键作用。基于学习的方法的最新进展显著提升了深度估计性能。生成模型，特别是稳定扩散（Stable Diffusion），通过在多样化数据集上进行大规模训练，展现出在恢复精细细节和重建缺失区域方面的巨大潜力。然而，依赖文本嵌入的模型（如CLIP）在需要丰富上下文信息的复杂户外环境中存在局限性，这些局限性降低了其在这些挑战性场景中的有效性。为此，我们提出了一种基于图像的语义嵌入方法，直接从视觉特征中提取上下文信息，显著提升了复杂环境中的深度预测精度。在KITTI和Waymo数据集上的评估表明，我们的方法在性能上可媲美当前最优模型，同时克服了CLIP嵌入在处理户外场景时的不足。通过直接利用视觉语义，我们的方法在深度估计任务中展现出增强的鲁棒性和适应性，并展示了其在其他视觉感知任务中的应用潜力。

Keyword:

Monocular depth estimation

Stable diffusion

Semantic embedding

Generative model

期刊

TOWARDS AUTONOMOUS ROBOTIC SYSTEMS, TAROS 2025

IF:

论文数:

被引数:

机构

university of york - uk

学者数:

1.5W

论文数: 1.5W

被引数: 15

引用论文

Vision Transformers for Dense Prediction 用于密集预测的视觉变压器

2021 IEEE/CVF International Conference on Computer Vision (ICCV)

IF0

2021-10-01

OAAI

Rene Ranftl; Alexey Bochkovskiy; Vladlen Koltun

AdaBin: Improving Binary Neural Networks with Adaptive Binary Sets

Lecture Notes in Computer Science

IF0

2022-11-03

PREAI

Zhijun Tu; Xinghao Chen; Pengju Ren; Yunhe Wang

Deep learning for monocular depth estimation: A review 用于单目深度估计的深度学习: 综述

NEUROCOMPUTING

IF6.5

2021-05-01

219

OAAI

Ming, Yue; Meng, Xuyang; Fan, Chunxiao; Yu, Hui

High-Resolution Image Synthesis with Latent Diffusion Models 具有潜在扩散模型的高分辨率图像合成

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2022-06-01

OAAI

Robin Rombach; Andreas Blattmann; Dominik Lorenz; Patrick Esser; Bjorn Ommer

Digging Into Self-Supervised Monocular Depth Estimation

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

IF0

2019-10-01

OAAI

Clement Godard; Oisin Mac Aodha; Michael Firman; Gabriel Brostow

Prompt-Free Diffusion: Taking “Text” Out of Text-to-Image Diffusion Models

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2024-06-16

PREAI

Xingqian Xu; Jiayi Guo; Zhangyang Wang; Gao Huang; Irfan Essa; Humphrey Shi

Scalability in Perception for Autonomous Driving: Waymo Open Dataset 自动驾驶感知的可扩展性: Waymo开放数据集

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

IF0

2020-06-01

OAAI

Pei Sun; Henrik Kretzschmar; Xerxes Dotiwalla; Aurelien Chouard; Vijaysai Patnaik; Paul Tsui; James Guo; Yin Zhou; Yuning Chai; Benjamin Caine; Vijay Vasudevan; Wei Han; Jiquan Ngiam; Hang Zhao; Aleksei Timofeev; Scott Ettinger; Maxim Krivokon; Amy Gao; Aditya Joshi; Yu Zhang; Jonathon Shlens; Zhifeng Chen; Dragomir Anguelov

Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue

Lecture Notes in Computer Science

IF0

2016-09-17

OAAI

Ravi Garg; Vijay Kumar B.G.; Gustavo Carneiro; Ian Reid

Monocular Depth Prediction Using Generative Adversarial Networks

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

IF0

2018-06-01

PREAI

Aran CS Kumar; Suchendra M. Bhandarkar; Mukta Prasad