Benchmarking equivariance for Deep Learning based optical flow estimators

doi:10.1016/j.image.2022.116892

Benchmarking equivariance for Deep Learning based optical flow estimators

2023-02-01

0

PRE

AI

DOI：10.1016/j.image.2022.116892

Original

Original request for help

Save

Abstract

En 中文

The Optical Flow (OF) estimation is an important but challenging task used in various computer vision applications. Recently, the introduction of Deep Learning models has resulted in a veritable paradigm shift. However, such an approach can be biased towards the specific training data, which can lead to a considerable inaccuracy of the estimated OF in real world applications. This paper proposes a novel benchmark that can be used to identify and measure different biases of Deep Learning models for OF estimation. The benchmark focuses on testing the network equivariance, i.e. the model capability to handle data transformations. We have performed experiments based on public datasets to (1) investigate to what extent the state-of-the-art networks lack spatial equivariance when reflections are applied to the data; (2) propose new metrics and a methodology to assess the phenomenon; and (3) benchmark the state-of-the-art optical estimators and their core components for equivariance. The results show that some state-of-the-art Deep Learning techniques present a substantial degree of bias towards certain directions of motion. The proposed framework can help researchers and practitioners to develop more effective models for OF estimation. The testing and training scripts provided at https://github.com/stsavian/benchmarking_equivariance_for_of_estimators can be used to evaluate and compare different models for OF estimation.

Keywords:

Optical flow

Deep Learning

Equivariance

Benchmark

Benchmarking equivariance for Deep Learning based optical flow estimators

Abstract

Journal

Organization