Pitfalls in Machine Learning for Computer Security

doi:10.1145/3643456

Pitfalls in Machine Learning for Computer Security

2024-10-25

0

OA

AI

Christian Wressnegger

DOI：10.1145/3643456

Original request for help

Save

Abstract

En 中文

With the growing processing power of computing systems and the increasing availability of massive datasets, machine-learning (ML) algorithms have led to major breakthroughs in many different areas. This development has influenced computer security, spawning a series of work on learning-based security systems, such as for malware detection, vulnerability discovery, and binary code analysis. Despite great potential, ML in security is prone to subtle pitfalls that undermine its performance and render learning-based sys-tems potentially unsuitable for security tasks and practical deployment.In this paper, we look at this problem with critical eyes. First, we identify common pitfalls in the design, imple-mentation, and evaluation of learning-based security sys-tems. We conduct a study of 30 papers from top-tier secu-rity conferences within the past 10 years, confirming that these pitfalls are widespread in the current security lit-erature. In an empirical analysis, we further demonstrate how individual pitfalls can lead to unrealistic performance and interpretations, obstructing the understanding of the security problem at hand. As a remedy, we propose action-able recommendations to support researchers in avoiding or mitigating the pitfalls where possible. Furthermore, we identify open problems when applying ML in security and provide directions for further research

Pitfalls in Machine Learning for Computer Security

Abstract

Journal

Organization