Knowledge Distillation-Based Anomaly Detection via Adaptive Discrepancy Optimization

doi:10.1109/TII.2025.3574540

Knowledge Distillation-Based Anomaly Detection via Adaptive Discrepancy Optimization

0

PRE

AI

DOI：10.1109/TII.2025.3574540

Original

Original request for help

Save

Abstract

En 中文

Knowledge distillation has emerged as a primary solution for anomaly detection, leveraging feature discrepancies between teacher–student (T–S) networks to locate anomalies. However, previous approaches suffer from ambiguous feature discrepancies, which hinder effective anomaly detection due to two main challenges: 1) overgeneralization, where the student network excessively mimics teacher features in anomalous regions, and 2) semantic bias between T–S networks in normal regions. To address these issues, we propose an Adaptive Discrepancy Optimization (Ado) block. The Ado block adaptively calibrates feature discrepancies by reducing overgeneralization in anomalous regions and selectively aligning semantic features in normal regions via learnable feature offsets. This versatile block can be seamlessly integrated into various distillation-based methods. Experimental results demonstrate that the Ado block significantly enhances performance across 11 different knowledge distillation frameworks on two widely used datasets. Notably, when integrated with the Ado block, RD4AD achieves a 22% relative improvement in pixel-level PRO on the VisA dataset. In addition, a real-world keyboard inspection application further validates the effectiveness of the Ado block.

Keywords:

Anomaly detection

defect detection

knowledge distillation

overgeneralization problem

semantic bias

Knowledge Distillation-Based Anomaly Detection via Adaptive Discrepancy Optimization

Abstract

Journal

Organization