Open-Set Tattoo Semantic Segmentation

Anderson Brilhador

doi:10.1109/ACCESS.2024.3438557

2024-01-01

Anderson Brilhador *

Rodrigo Tchalski da Silva

Carlos Roberto Modinez-Junior

Gabriel de Almeida Spadafora

Heitor Silvério Lopes

André Eugênio Lazzaretti

DOI：10.1109/ACCESS.2024.3438557

原文链接

原文求助

摘要

En 中文

Tattoos can serve as an essential source of biometric information for public security, aiding in identifying suspects and victims. In order to automate tattoo classification, tasks like classification require more detailed image content analysis, such as semantic segmentation. However, a dataset with appropriate semantic segmentation annotations is currently lacking. Also, there are countless ways to categorize tattoo classes, and many are not directly categorizable, either because they belong to a specific artistic trait or characterize an object with previously undefined semantics. An effective way to overcome these limitations is to build recognition systems based on open-set assumptions. Nevertheless, state-of-the-art open set approaches are not directly applicable in tattoo semantic segmentation, mainly due to the significant class imbalance (predominant background). To the best of our knowledge, this paper is the first to explore semantic segmentation in closed and open-set scenarios for tattoos. In this sense, this paper presents two key contributions: (i) a novel large-margin loss function and generalized open-set classifier approach and (ii) an open-set tattoo semantic segmentation dataset with a publicly accessible test set, enabling comparisons and future research in this area. The proposed approach outperforms other methods, achieving 0.8013 of AUROC, 0.6318 of Macro F1, 0.4900 of mIoU, and notably 0.2753 of IoU for the unknown class, demonstrating the feasibility of this approach for automatic tattoo analysis. The paper also highlights key limitations and open research areas in this challenging field.

Keyword:

Semantic segmentation

Semantics

Flowering plants

Biometrics (access control)

Annotations

Task analysis

Complexity theory

Open-world

open-set

semantic segmentation

large-margin learning