Local Selective Vision Transformer for Depth Estimation Using a Compound Eye Camera

doi:10.1016/j.patrec.2023.02.010

Local Selective Vision Transformer for Depth Estimation Using a Compound Eye Camera

2023-03-01

5

PRE

AI

DOI：10.1016/j.patrec.2023.02.010

原文链接

原文求助

摘要

En 中文

A compound eye camera is a hemispherical camera made by mimicking the structure of an insect's eye. In general, a compound eye camera is composed of a set of single eye cameras. The compound eye cam-era has various advantages due to its unique structure and can be used in various vision tasks. In order to apply the compound eye camera to various vision tasks using 3D information, depth estimation is required. However, due to the difference between the compound eye image and the 2D RGB image, it is hard to use the existing depth estimation methods directly. In this paper, we propose a transformer-based neural network for eye-wise depth estimation, which is suitable for the compound eye image. We modify the self-attention module with local selective self-attention to take advantage of the compound eye's hemispherical structure. In addition, we reduce the computational amount and increase the per-formance through the eye selection module. Using the proposed local selective self-attention and eye selection modules, we are able to improve the performance without large-scale pre-training. Compared to the ResNet-based depth estimation network, our method showed 2.8% and 1.4% higher performance on the GAZEBO and Matterport3D datasets, respectively, with 15.3% fewer network parameters.(c) 2023 Elsevier B.V. All rights reserved.

Keyword:

Compound Eye

Depth Estimation

Vision Transformer

Local Selective Vision Transformer for Depth Estimation Using a Compound Eye Camera

摘要

期刊

机构

引用论文