.png){kind=icon}
返回
Image clustering using generated text centroids
.png){kind=icon}
PRE
AI
DOI：10.1016/j.image.2024.117128.png)
.png){kind=icon}
原文求助.png){kind=icon}
分享.png){kind=icon}
收藏
摘要
En 中文
In recent years, deep neural networks pretrained on large-scale datasets have been used to address data deficiency and achieve better performance through prior knowledge. Contrastive language-image pretraining (CLIP), a vision-language model pretrained on an extensive dataset, achieves better performance in image recognition. In this study, we harness the power of multimodality in image clustering tasks, shifting from a single modality to a multimodal framework using the describability property of image encoder of the CLIP model. The importance of this shift lies in the ability of multimodality to provide richer feature representations. By generating text centroids corresponding to image features, we effectively create a common descriptive language for each cluster. It generates text centroids assigned by the image features and improves the clustering performance. The text centroids use the results generated by using the standard clustering algorithm as a pseudo-label and learn a common description of each cluster. Finally, only text centroids were added when the image features on the same space were assigned to the text centroids, but the clustering performance improved significantly compared to the standard clustering algorithm, especially on complex datasets. When the proposed method is applied, the normalized mutual information score rises by 32% on the Stanford40 dataset and 64% on ImageNet-Dog compared to the k-means clustering algorithm.
Keyword:
Deep neural network
Image clustering
Multimodal task
Vision-language model
期刊
S
IF:
2.7
论文数:
2.8K
被引数:
4.2K
机构
引用论文
Spectrophotometric Determination of Carbon Disulfide Using Bis(4-(4 -Nitrophenyl)azo-2-Nitrophenyl) Disulfide双 (4-硝基苯基) azo-2-Nitrophenyl) 二硫化物分光光度法测定二硫化碳
PREAI
分享收藏Spatiotemporal Characteristics of Air Pollutants (PM10, PM2.5, SO2, NO2, O3, and CO) in the Inland Basin City of Chengdu, Southwest China.png)
Atmosphere
OAAI
AtmosphereIF0
OAAI分享收藏Millennial-scale changes of surface- and deep-water flow in the western tropical Atlantic linked to Northern Hemisphere high-latitude climate during the HoloceneGeology
GeologyIF0
PREAI
分享收藏
查看更多内容