.png){kind=icon}
Return
Image clustering using generated text centroids
.png){kind=icon}
PRE
AI
DOI：10.1016/j.image.2024.117128.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
In recent years, deep neural networks pretrained on large-scale datasets have been used to address data deficiency and achieve better performance through prior knowledge. Contrastive language-image pretraining (CLIP), a vision-language model pretrained on an extensive dataset, achieves better performance in image recognition. In this study, we harness the power of multimodality in image clustering tasks, shifting from a single modality to a multimodal framework using the describability property of image encoder of the CLIP model. The importance of this shift lies in the ability of multimodality to provide richer feature representations. By generating text centroids corresponding to image features, we effectively create a common descriptive language for each cluster. It generates text centroids assigned by the image features and improves the clustering performance. The text centroids use the results generated by using the standard clustering algorithm as a pseudo-label and learn a common description of each cluster. Finally, only text centroids were added when the image features on the same space were assigned to the text centroids, but the clustering performance improved significantly compared to the standard clustering algorithm, especially on complex datasets. When the proposed method is applied, the normalized mutual information score rises by 32% on the Stanford40 dataset and 64% on ImageNet-Dog compared to the k-means clustering algorithm.
Keywords:
Deep neural network
Image clustering
Multimodal task
Vision-language model
Journal
S
IF:
2.7
Papers:
2.8K
Citations:
4.2K
Organization
Cited Papers
ShareSaveShareSaveShareSaveSpatiotemporal Characteristics of Air Pollutants (PM10, PM2.5, SO2, NO2, O3, and CO) in the Inland Basin City of Chengdu, Southwest China.png)
Atmosphere
OAAI
AtmosphereIF0
OAAIShareSaveMillennial-scale changes of surface- and deep-water flow in the western tropical Atlantic linked to Northern Hemisphere high-latitude climate during the HoloceneGeology
GeologyIF0
PREAI
ShareSaveShareSaveShareSave
View more