Autonomous anomaly detection for streaming data

Muhammad Yunus Iqbal Basheer

doi:10.1016/j.knosys.2023.111235

Autonomous anomaly detection for streaming data

2024-01-01

1

PRE

AI

M

Muhammad Yunus Iqbal Basheer

A

Azliza Mohd Ali *

N

Nurzeatul Hamimah Abdul Hamid

M

Muhammad Azizi Mohd Ariffin

DOI：10.1016/j.knosys.2023.111235

Original

Original request for help

Save

Abstract

En 中文

Anomaly detection from data streams is a hotly studied topic in the machine learning domain. It is widely considered a challenging task because the underlying patterns exhibited by the streaming data may dynamically change at any time. In this paper, a new algorithm is proposed to detect anomalies autonomously for streaming data. The proposed algorithm is nonparametric and does not require any threshold to be preset by users. The algorithmic procedure of the proposed algorithm is composed of the following three complementary stages. Firstly, the potentially anomalous samples that represent highly different patterns from others are identified from data streams based on data density. Then, these potentially anomalous samples are clustered online using the evolving autonomous data partitioning algorithm. Finally, true anomalies are identified from these minor clusters with the least amounts of samples associated with them. Numerical examples based on three benchmark datasets demonstrated the potential of the proposed algorithm as a highly effective approach for anomaly detection from data streams.

Keywords:

Anomaly detection

Data stream

Online clustering

Autonomous anomaly detection for streaming data

Abstract

Journal

Organization