Long kernel distillation in human activity recognition

doi:10.1016/j.knosys.2025.113397

Long kernel distillation in human activity recognition

2025-05-01

0

PRE

AI

DOI：10.1016/j.knosys.2025.113397

Original

Original request for help

Save

Abstract

En 中文

Most human activity recognition (HAR) models prioritize achieving high recognition rates with deep neural networks, often at the expense of increased computational complexity, which significantly limits their practical deployment on embedded devices. Knowledge distillation, a technique where dark knowledge is transferred from larger models to smaller ones, presents a promising solution to this challenge. However, existing distillation methods overlook two critical issues: (1) teacher models with limited receptive fields inherently constrain model performance, and (2) current distillation techniques fail to transfer an adequate amount of activity-related knowledge. To address these limitations, this paper pioneers the exploration of long-kernel HAR models as teacher models, empowering short-kernel HAR models with enhanced perceptual capabilities. Additionally, we propose a novel multi-teacher knowledge distillation algorithm that adaptively transfers richer and more diverse activity-related knowledge from multiple long-kernel networks to the student model. Extensive experiments conducted on three public benchmark datasets demonstrate consistent performance improvements (up to 4.15%) for the student model, while maintaining its lightweight design.

Keywords:

Human activity recognition

Convolutional neural network

Long kernel

Knowledge distillation

Sensor

Long kernel distillation in human activity recognition

Abstract

Journal

Organization