.png){kind=icon}
返回
Micro-object pose estimation with sim-to-real transfer learning using small dataset
.png){kind=icon}
OA
AI
DOI：10.1038/s42005-022-00844-z.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Three-dimensional (3D) pose estimation of micro/nano-objects is essential for the implementation of automatic manipulation in micro/nano-robotic systems. However, out-of-plane pose estimation of a micro/nano-object is challenging, since the images are typically obtained in 2D using a scanning electron microscope (SEM) or an optical microscope (OM). Traditional deep learning based methods require the collection of a large amount of labeled data for model training to estimate the 3D pose of an object from a monocular image. Here we present a sim-to-real learning-to-match approach for 3D pose estimation of micro/nanoobjects. Instead of collecting large training datasets, simulated data is generated to enlarge the limited experimental data obtained in practice, while the domain gap between the generated and experimental data is minimized via image translation based on a generative adversarial network (GAN) model. A learning-to-match approach is used to map the generated data and the experimental data to a low-dimensional space with the same data distribution for different pose labels, which ensures effective feature embedding. Combining the labeled data obtained from experiments and simulations, a new training dataset is constructed for robust pose estimation. The proposed method is validated with images from both SEM and OM, facilitating the development of closed-loop control of micro/nano-objects with complex shapes in micro/nano-robotic systems.
Keyword:
DEEP
SYSTEM
期刊
IF:
5.8
论文数:
2.8K
被引数:
9.2K
机构
引用论文
Modernizing Substation Automation Systems: Adopting IEC Standard 61850 for Modeling and Communication变电站自动化系统的现代化: 采用IEC标准61850进行建模和通信
PREAI
分享收藏Deep Learning Enables High-Throughput Analysis of Particle-Aggregation-Based Biosensors Imaged Using Holography.png)
ACS PHOTONICS
OAAI
ACS PHOTONICSIF6.7
OAAI分享收藏CHIMERIC ANTIGEN RECEPTOR (CAR) T CELL TREATMENT IN VERY HIGH TITER INHIBITOR MURINE MODEL REQUIRES CD19 AND BCMA TARGETING在非常高滴度抑制剂的鼠模型中，嵌合抗原受体（CAR）T细胞治疗需要靶向CD19和BCMA
PREAI
分享收藏分享收藏
查看更多内容