Efficient Cluster-Based Knowledge Distillation for Deep Face Recognition

Xianwei Lv

doi:10.1109/tsusc.2026.3677896

Efficient Cluster-Based Knowledge Distillation for Deep Face Recognition

2026-05-01

0

PRE

AI

DOI：10.1109/tsusc.2026.3677896

Original

Original request for help

Save

Abstract

En 中文

Knowledge distillation has been widely used to improve the performance of small compact models for face recognition. However, selecting key knowledge and effectively transferring it from teacher to student remains a challenging problem. In this work, we propose an efficient Cluster-based Knowledge Distillation (CKD) dedicated to aligning the student model with the teacher model in terms of both sample relations and class centers. Specifically, CKD first determines the key sample relations based on the similarities between the sample features extracted by the teacher and their cluster centers generated by existing clustering algorithms. Then, CKD effectively transfers the knowledge of the above relations from the teacher to the student by designing a cluster-based relation distillation loss. Finally, CKD further improves the quality of the student's class centers by constructing a center loss between the above representative cluster centers and the student's class centers. We validate the proposed CKD on multiple face benchmarks. For example, CKD improves the baseline student performance from 91.95% to 94.20% on MegaFace and consistently outperforms recent competitive distillation methods on multiple benchmarks. These results demonstrate the effectiveness and superiority of CKD.

Keywords:

Face recognition

Chronic kidney disease

Training

Measurement

Feature extraction

Computational modeling

Knowledge engineering

Clustering algorithms

Benchmark testing

Performance evaluation

Knowledge distillation

deep face recognition

cluster centers

Efficient Cluster-Based Knowledge Distillation for Deep Face Recognition

Abstract

Journal

Organization

Cited Papers