.png){kind=icon}
.png){kind=icon}
Return
Wavelet-Based Distillation with Structured Frequency Alignment
.png){kind=icon}
PRE
AI
DOI：10.1007/978-981-95-3393-0_27.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Feature-level knowledge distillation is a promising approach for compressing object detectors. It transfers intermediate representations from high-capacity, computationally intensive teacher models to lightweight students with lower inference costs. However, existing methods predominantly emphasize spatial and channel-wise alignment, while largely overlooking the frequency characteristics that can be derived from deep features via spectral decomposition. In this work, we present a frequency-aware knowledge distillation approach that enhances existing frameworks through the integration of wavelet-based feature alignment. Specifically, teacher features are decomposed into multi-level subbands using two-dimensional Haar wavelet transforms, enabling the student to perform subband-wise alignment that captures both high-frequency details and low-frequency semantic cues. Additionally, a relational distillation mechanism operating on the principal subband is employed to model global dependencies and enhance semantic consistency. To evaluate the effectiveness of the proposed approach, we conduct experiments on the Inspection of Power Line Assets Dataset (InsPLAD), a real-world UAV dataset for transmission infrastructure inspection, using various detectors including both two-stage and one-stage models. Building upon the FGD framework, our method yields distilled lightweight student models that consistently achieve 1%-3% gains in average precision (AP) across various detectors. Furthermore, it demonstrates superior detection performance, particularly in handling elongated and structurally complex objects that typically pose challenges for compact models.
Keywords:
Haar Wavelet Decomposition
Knowledge Distillation
Object Detection
Subband Alignment
Power Asset Inspection