.png){kind=icon}
.png){kind=icon}
Return
Two-stream transformer network for sensor-based human activity recognition
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neucom.2022.09.099.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Human Activity Recognition (HAR) based on wearable devices has always been a hot topic in health applications, human-object interaction, and smart homes. Despite significant improvements achieved by convolutional neural networks, long short-term memory networks, transformer networks and various hybrid models, there are still two fundamental issues. First, the spatial-temporal dependencies of sensor signals are difficult to be effectively modeled. Second, in multimodal environments, sensors placed on various body positions contribute distinctly to the classification results. In this work, we propose a self-attention based Two-stream Transformer Network (TTN). In view of the former issue, we use two streams, named temporal stream and spatial stream, respectively, to extract the readings-over-time and time-over-readings features from sensor signals. These features extracted from two streams are com-plementary since the time-over-readings features are able to express additional information which can-not be captured from sensor signals directly. To deal with the latter issue, we assign attention weights to each sensor axis in the spatial channel based on their classification scores. It makes sense that different axis-readings with distinct recognition contributions caused by data heterogeneity be treated unequally. Extensive experiments on four available benchmark datasets (PAMAP2, Opportunity, USC-HAD, and Skoda) reveal that our proposed model is better suited for multimodal HAR than previous state-of-the-art methods.(c) 2022 Elsevier B.V. All rights reserved.
Keywords:
Human activity recognition
Wearable devices
Self-attention
Multimodal sensors
Journal
IF:
6.5
Papers:
2.5W
Citations:
6.5W
Organization
No organization information available