CollaborativeBEV: Collaborative bird eye view for reconstructing crowded environment

doi:10.1016/j.imavis.2024.105060

CollaborativeBEV: Collaborative bird eye view for reconstructing crowded environment

2024-07-01

1

PRE

AI

DOI：10.1016/j.imavis.2024.105060

原文链接

原文求助

摘要

En 中文

Constructing a virtual world for the Metaverse based on real-world data is crucial, yet creating virtual environments for crowded scenes poses challenges in accurately tracking individuals using egocentric wearable cameras due to occlusions caused by crowded pedestrians. To address this, we propose a collaborative perception strategy that leverages multiple agents equipped with multi-view cameras to construct an occupancy map for a crowded environment. To fuse the multi-view perceptions of multiple agents, we propose a Collaborative Bird Eye View fusion network, called CollaborativeBEV (C-BEV), in which, we leverage a depth-based BEV network as a feature extractor, and propose a feature enhancement module to improve perception fusion in overlapping area. A designed loss function is introduced to address data imbalance during training, and a BEV enhancement strategy is proposed to augment the sample pool for training the BEV decoder. Experiment on the Sean2.0 dataset demonstrates that our C-BEV method performs better than the baseline method in terms of a 5.3% IoU increase. Our code will be released on github. https://github.com/RYaNzzZ1/CollaborativeBEV.

Keyword:

Metaverse

3D reconstruction

Occupancy map

Collaborative BEV

Point cloud understanding

CollaborativeBEV: Collaborative bird eye view for reconstructing crowded environment

摘要

期刊

机构

引用论文