.png){kind=icon}
Return
ChatDiff: A ChatGPT-based diffusion model for long-tailed classification
.png){kind=icon}
PRE
AI
DOI：10.1016/j.neunet.2024.106794.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Long-tailed data distributions have been a major challenge for the practical application of deep learning. Information augmentation intends to expand the long-tailed data into uniform distribution, which provides a feasible way to mitigate the data starvation of underrepresented classes. However, most existing augmentation methods face two significant challenges: (1) limited diversity in generated samples, and (2) the adverse effect of generated negative samples on downstream classification performance. In this paper, we propose a novel information augmentation method, named ChatDiff, to provide diverse positive samples for underrepresented classes, and eliminate generated negative samples. Specifically, we start with a prompt template to extract textual prior knowledge from the ChatGPT-3.5 model, enhancing the feature space for underrepresented classes. Then using this prior knowledge, a conditional diffusion model generates semantic-rich image samples for tail classes. Moreover, the proposed ChatDiff leverages a CLIP-based discriminator to screen and remove generated negative samples. This process avoids neural network learning the invalid or erroneous features, and further, improves long-tailed classification performance. Comprehensive experiments conducted on long-tailed benchmarks such as CIFAR10-LT, CIFAR100-LT, ImageNet-LT, and iNaturalist 2018, validate the effectiveness of our ChatDiff method.
Keywords:
Long-tailed learning
Information augmentation
ChatGPT-3.5
Diffusion probabilistic model
Discriminator mechanism
Image classification
Journal
IF:
6.3
Papers:
8.3K
Citations:
3.0W
Organization
View more organizations
Cited Papers
ShareSave
ShareSaveBioactive compounds, antioxidant capacity and anti-inflammatory activity of native fruits from Brazil.png)
PLOS ONE
OAAI
PLOS ONEIF0
OAAIShareSaveShareSave View more