.png){kind=icon}
返回
A Multimodal Data Processing System for LiDAR-Based Human Activity Recognition
.png){kind=icon}
OA
AI
DOI：10.1109/TCYB.2021.3085489.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Increasingly, the task of detecting and recognizing the actions of a human has been delegated to some form of neural network processing camera or wearable sensor data. Due to the degree to which the camera can be affected by lighting and wearable sensors scantiness, neither one modality can capture the required data to perform the task confidently. That being the case, range sensors, like light detection and ranging (LiDAR), can complement the process to perceive the environment more robustly. Most recently, researchers have been exploring ways to apply convolutional neural networks to 3-D data. These methods typically rely on a single modality and cannot draw on information from complementing sensor streams to improve accuracy. This article proposes a framework to tackle human activity recognition by leveraging the benefits of sensor fusion and multimodal machine learning. Given both RGB and point cloud data, our method describes the activities being performed by subjects using regions with a convolutional neural network (R-CNN) and a 3-D modified Fisher vector network. Evaluated on a custom captured multimodal dataset demonstrates that the model outputs remarkably accurate human activity classification (90%). Furthermore, this framework can be used for sports analytics, understanding social behavior, surveillance, and perhaps most notably by autonomous vehicles (AVs) to data-driven decision-making policies in urban areas and indoor environments.
Keyword:
Sensors
Laser radar
Wearable sensors
Three-dimensional displays
Micromechanical devices
Activity recognition
Cameras
Convolutional neural network
faster RCNN
Fisher vector
human activity recognition (HAR)
multimodal machine learning (ML)
期刊
IF:
10.5
论文数:
1.1W
被引数:
5.0W
机构
引用论文
分享收藏Improved Use of Foot Force Sensors and Mobile Phone GPS for Mobility Activity Recognition改进使用脚力传感器和手机GPS进行移动活动识别.png)
IEEE SENSORS JOURNAL
OAAI
IEEE SENSORS JOURNALIF4.5
OAAI分享收藏
查看更多内容