Triple-Sigmoid Activation Function for Deep Open-Set Recognition

doi:10.1109/ACCESS.2022.3192621

Triple-Sigmoid Activation Function for Deep Open-Set Recognition

2022-01-01

4

OA

AI

DOI：10.1109/ACCESS.2022.3192621

原文链接

原文求助

摘要

En 中文

Traditional models for various machine learning problems such as image classification perform well only under the assumption of a closed set. This implies that inputs must belong to the classes for which the models were trained. Data collected in the real world may not belong to any finite set of classes, and training a model with an infinite number of classes would obviously be impossible. Rather than incorrectly classifying outlier inputs that belong to unknown classes as members of one of the classes on which the model was trained, learning models must recognize and reject such data samples, or request human assistance in labeling them. For example, when a self-driving vehicle detects unfamiliar scenes or objects, it must notify the driver and hand over control. Various models have been proposed to address the open-set problem. However, the existing models are generally complex or use complicated techniques such as generative adversarial networks and auto-encoders, and their efficiency is not proportional to their complexity. In this study, we propose Triple-Sigmoid as a simple activation function comprised of three Sigmoid functions. Using Triple-Sigmoid in the last activation layer of any deep neural network model enables the model to recognize outliers. Although Triple-Sigmoid can be applied to a variety of machine learning problems, including semi-supervised learning, active learning, and incremental learning, we only investigated the open-set recognition problem in this work. The results of numerous experiments are presented to validate that substituting Triple-Sigmoid for conventional activations such as Softmax and Sigmoid not only retained performance in closed-set settings, but also significantly improved performance in open-set configurations. Furthermore, these results demonstrate that Triple-Sigmoid significantly outperformed existing state-of-the-art methods. Source code for the proposed model is available at https://github.com/dinhtuantran/triple-sigmoid.

Keyword:

Anomaly detection

Training

Semisupervised learning

Machine learning

Feature extraction

Deep learning

Generative adversarial networks

Active learning

anomaly detection

incremental learning

novelty detection

open-set recognition

out-of-distribution detection

outlier detection

semi-supervised learning

Triple-Sigmoid Activation Function for Deep Open-Set Recognition

摘要

期刊

机构

引用论文