Efficient on-the-fly Web bot detection

Grażyna Suchacka

doi:10.1016/j.knosys.2021.107074

Return

Efficient on-the-fly Web bot detection

2021-07-01

DOI：10.1016/j.knosys.2021.107074

Original

Save

View PDF

Abstract

En 中文

A large fraction of traffic on present-day Web servers is generated by bots - intelligent agents able to traverse the Web and execute various advanced tasks. Since bots' activity may raise concerns about server security and performance, many studies have investigated traffic features discriminating bots from human visitors and developed methods for automated traffic classification. Very few previous works, however, aim at identifying bots on-the-fly, trying to classify active sessions as early as possible. This paper proposes a novel method for binary classification of streams of Web server requests in order to label each active session as bot'' or human''. A machine learning approach has been developed to discover traffic patterns from historical usage data. The model, built on a neural network, is used to classify each incoming HTTP request and a sequential probabilistic analysis approach is then applied to capture relationships between subsequent HTTP requests in an ongoing session to assess the likelihood of the session being generated by a bot or a human, as soon as possible. A performance evaluation study with real server traffic data confirmed the effectiveness of the proposed classifier in discriminating bots from humans at early stages of their visits, leaving very few of them undecided, with very low number of false positives. (C) 2021 The Author(s). Published by Elsevier B.V.

Keywords:

Web bot

Internet robot

Real-time bot detection

Machine learning

Sequential analysis

Neural network

Early decision

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.