.png){kind=icon}
Return
Probabilistic Visual Prompt Tuning
.png){kind=icon}
PRE
AI
DOI：10.1007/978-981-95-5679-3_30.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Large-scale pre-trained models perform exceptionally well on various downstream tasks but their growing size demands significant computational and storage resources, limiting practical use. To address this, efficient fine-tuning methods have been developed that adapt models by modifying only a few parameters. Visual Prompt Tuning (VPT) is a promising approach that adds learnable prompts at the input and fine-tunes them alongside the classification head without altering the model's backbone. However, existing VPT methods typically only use low-dimensional representations, which restrict their ability to capture complex information and adapt to diverse datasets. In this study, we proposed Probabilistic Visual Prompt Tuning (PVPT), an enhanced VPT method that leverages Gaussian distribution sampling to generate robust and accurate visual prompts. PVPT first integrates two lightweight networks to extract domain-level and instance-level features from downstream datasets, which are then fused to define the parameters (mean and variance) of a Gaussian distribution. Probabilistic visual prompts are generated by sampling from this distribution, allowing the incorporation of uncertainty into the prompts. This probabilistic approach equips PVPT to capture more complex feature, characterize the underlying distribution of downstream data, thereby providing precise guidance for the efficient fine-tuning of the pre-trained model. Extensive experiments and ablation studies on multiple downstream tasks show that PVPT improves the performance of pre-trained models while maintaining efficient fine-tuning, highlighting its potential for practical applications.
Keywords:
Parameter-efficient fine-tuning
Visual prompt tuning
Probability distribution modeling
Journal
P
IF:
0
Papers:
31
Citations:
0
Organization
Cited Papers
.png){kind=icon}
ShareSaveShareSave
View more