Hierarchical Equalization Loss for Long-Tailed Instance Segmentation

doi:10.1109/TMM.2024.3358080

2024-01-01

PRE

DOI：10.1109/TMM.2024.3358080

原文链接

原文求助

摘要

En 中文

In long-tailed image instance segmentation, the existing methods deal with imbalance problem from a single perspective, which results in the limitation of performance. Considering that imbalances exist not only between positive and negative classes, but also between foreground and background subclasses, as well as between hard and easy examples, we argue that the losses of samples should be hierarchically equalized at multi-levels (HEL). We first propose a focus based hierarchical-equalization loss (FHEL), which employs a class gradient ratio based reweighting mechanism to achieve the balance between classes, and uses a subclass-balance term and a sample-balance term to separately deal with the inter-subclass and inter-sample imbalances. FHEL can improve the performance of long-tailed instance segmentation in an end-to-end manner, avoiding the overfitting risk and manual hard division in the traditional methods. On the basis of FHEL, we further explore the relationship between inter-subclass imbalance and inter-sample imbalance, and propose a constrained-focus based hierarchical-equalization loss (CFHEL) that copes with the imbalances at multi-levels simultaneously with fewer hyperparameters. We conduct extensive experiments on LVIS v1.0 and COCO-LT datasets with different benchmarks. Both FHEL and CFHEL are superior to the existing methods. On LVIS v1.0, with ResNet50Mask R-CNN, ResNet101Mask R-CNN, ResNeXt101Mask R-CNN and ResNet101 CascadeMask R-CNN, CFHEL outperforms its baselines respectively with 19.8 % , 18.5 % , 21.6 % and 21.2 % AP r gains, and with 6.7 % , 6.6 % , 7.6 % and 6.5 % AP gains, achieving the new state-of-the-arts. On COCO-LT, our CFHEL outperforms the baseline with 13.2 % APr gains and 3.3 % AP gains, also achieving the new best performances.

Keyword:

Instance segmentation

Object detection

long-tailed distribution

imbalanced learning

deep learning