.png){kind=icon}
返回
Surface Material Perception Through Multimodal Learning
.png){kind=icon}
PRE
AI
DOI：10.1109/JSTSP.2022.3171682.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Accurately perceiving object surface material is critical for scene understanding and robotic manipulation. However, it is ill-posed because the imaging process entangles material, lighting, and geometry in a complex way. Appearance-based methods cannot disentangle lighting and geometry variance and have difficulties in textureless regions. We propose a novel multimodal fusion method for surface material perception using the depth camera shooting structured laser dots. The captured active infrared image was decomposed into diffusive and dot modalities and their connection with different material optical properties (i.e. reflection and scattering) were revealed separately. The geometry modality, which helps to disentangle material properties from geometry variations, is derived from the rendering equation and calculated based on the depth image obtained from the structured light camera. Further, together with the texture feature learned from the gray modality, a multimodal learning method is proposed for material perception. Experiments on synthesized and captured datasets validate the orthogonality of learned features. The final fusion method achieves 92.5% material accuracy, superior to state-of-the-art appearance-based methods (78.4%).
Keyword:
Scattering
Cameras
Reflection
Geometry
Lighting
Robots
Optical scattering
Material recognition
structured light camera
subsurface scattering
multimodal learning
期刊
IF:
13.7
论文数:
1.9K
被引数:
1.1W
机构
引用论文
.png){kind=icon}
The influence of carrier level and frequency on modulation and beat-detection thresholds for sinusoidal carriers载波电平和频率对正弦载波调制和拍频检测阈值的影响
PREAI
分享收藏分享收藏分享收藏
查看更多内容