.png){kind=icon}
Return
FSP Matrix Compression Distillation Algorithm Based on Krylov Subspace Projection
.png){kind=icon}
OA
AI
DOI：10.1109/access.2026.3735117.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
The Segment Anything Model (SAM) has gained widespread recognition in general visual segmentation owing to its strong zero-shot generalisation and segmentation performance, yet its large network architecture severely limits deployment on mobile and edge devices. Most existing lightweight SAM methods adopt decoupled distillation, transferring knowledge solely by aligning the output features of a lightweight encoder with those of the SAM encoder. This alignment, however, fails to convey the deep structural information of the teacher model to the student, thereby constraining student performance. To address this problem, we propose a distillation method based on the Krylov subspace projection of the Flow of Solution Procedure (FSP) matrix, which achieves alignment of deep structural information between heterogeneous teacher–student model pairs. The proposed Krylov projection compression is designed to preserve the dominant structural information of the FSP matrix during dimensionality reduction, enabling FSP-based distillation to operate efficiently across models with mismatched channel dimensions without modifying the student’s inference architecture. Benefiting from this approach, MobileSAM achieves performance improvements of up to 4 percentage points on segmentation with ground-truth box prompts, 3 percentage points on segmentation with prompts generated by an external detector, and 6 to 10 percentage points in mIoU on everything-mode segmentation under varying point prompt densities. These results demonstrate the effectiveness of the proposed method and offer a new perspective on structured knowledge distillation between heterogeneous models.
Keywords:
Knowledge distillation
FSP matrix
Krylov subspace
image segmentation
SAM
model compression
Journal
IF:
3.6
Papers:
9.8W
Citations:
29.4W
Organization
Cited Papers
No cited papers available