.png){kind=icon}
返回
Recognizing temporary construction site objects using CLIP-based few-shot learning and multi-modal prototypes
.png){kind=icon}
PRE
AI
DOI：10.1016/j.autcon.2024.105542.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
Visual understanding of temporary on-site objects is essential for robots and project management in construction. Implementation of deep learning algorithms is challenging on construction sites due to high data annotation cost, demanding computational power, and lack of large-scale training datasets. Recognizing on-site temporary objects demands the algorithms to learn in a data-efficient way. To fill this gap, a Contrastive Language-Image Pretraining (CLIP)-based few-shot learning algorithm to recognize temporary objects with limited image samples is proposed. The study builds ImageNet-based Similarity Cache with inter-class similarity distribution. The proposed algorithm is evaluated on a newly created TOCS dataset and on the public SODA dataset. Compared with CLIP zero-shot algorithm, the classification accuracy improves from 23.17% to 73.09% with 16-shot learning on SODA, and from 58.33% to 83.33% with 1-shot learning on TOCS. The study indicates that fewshot learning with vision language models (VLM) is promising to improve visual intelligence on construction sites.
Keyword:
Construction sites
CLIP
Few -shot image classification (FSIC)
Computer vision
Robots
期刊
IF:
11.5
论文数:
6.3K
被引数:
4.2W
机构
引用论文
SODA: A large-scale open site object detection dataset for deep learning in constructionSODA: 用于建筑深度学习的大规模开放站点对象检测数据集
OAAI
OAAI分享收藏分享收藏Deep metric learning for few-shot image classification: A Review of recent developments用于少镜头图像分类的深度度量学习: 最新进展综述.png)
PATTERN RECOGNITIONOAAI
PATTERN RECOGNITIONIF7.6
OAAI分享收藏A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities
OAAI
OAAI分享收藏Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications建筑和施工4.0的人工智能和智能视觉: 机器和深度学习方法及应用OAAI
OAAI分享收藏
查看更多内容