Stream clustering guided supervised learning for classifying NIDS alerts

Risto Vaarandi

doi:10.1016/j.future.2024.01.032

Stream clustering guided supervised learning for classifying NIDS alerts

2024-06-01

4

PRE

AI

R

Risto Vaarandi *

A

Alejandro Guerra-Manzanares

DOI：10.1016/j.future.2024.01.032

原文链接

原文求助

摘要

En 中文

A Network Intrusion Detection System (NIDS) is a network monitoring technology for identifying cyber attacks, botnet command and control traffic, and other unwanted network activity. Unfortunately, organizational NIDS solutions can often generate tens or hundreds of thousands of alerts on a daily basis, with a significant part of them having low importance or being false positives. Therefore, high priority alerts become hard to spot, which overloads security analysts and complicates their work. The current paper addresses this problem and introduces a machine learning framework for classifying NIDS alerts with the help of stream clustering and supervised learning. We propose a stream -clustering -guided method for creating labeled NIDS alert data sets. The small data sets created using this method can be used for training high-performance supervised NIDS alert classifiers. This significantly reduces the human labeling effort and eases the application of supervised machine learning for NIDS alert classification. The proposed machine learning framework was evaluated on NIDS alerts collected over 2 months from the network of a large academic organization. The experimental results indicate that combining stream clustering and supervised learning into a NIDS alert classification framework significantly decreases the number of false positives, and thus reduces the workload of human security analysts. The framework also features low CPU time and memory consumption and can thus be run on commodity hardware. In conclusion, the proposed framework provides a cost-effective means of integrating machine learning into Security Operation Centers (SOCs). This enables the identification of critical NIDS alerts using high-performance classifiers, thereby assisting in the automation of alert handling tasks for SOC personnel. To address the lack of public data sets in the problem domain and foster further research, we publicly share the large labeled NIDS alert data set used in our experimental setup.

Keyword:

Network security

Intrusion detection

Network Intrusion Detection System

IDS

NIDS

NIDS alert

High-priority NIDS alert

NIDS alert classification

Supervised learning

Small training data set

Stream clustering

Data set generation

Data labeling

Security Operations Center

SOC

Workload reduction

Stream clustering guided supervised learning for classifying NIDS alerts

摘要

期刊

机构

引用论文