.png){kind=icon}
.png){kind=icon}
Return
Multi-Task Image Restoration Algorithm Under Different Weather Influence Factors
.png){kind=icon}
OA
AI
DOI：10.1109/ACCESS.2024.3520484.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Bad weather, such as rain, snow, and fog, will reduce the quality of the image acquired, and it will affect the performance of many related visual fields. The existing researches on image restoration under severe weather either focus on the restoration under a certain kind of weather, which cannot be generalized under different weather, or need to introduce additional model structures, which increases the burden of practical applications. Therefore, we explores an integrated image restoration framework, which can restore images under different adverse weather effects. Specifically, the model is trained in two stages. In the pre-training stage, a basic general network that can deal with multiple weather is trained by supervised learning. In the fine-tuning stage, soft prompts are introduced to stimulate the model's ability to cope with different weather, so as to enhance the generalization ability. Specifically, the hidden information of the prompts is explored by low-rank decomposition, and the contras loss is added to prompt the prompts to converge on similar tasks. Furthermore, we address the distribution shift problem by aligning out-of-distribution (OOD) test sample statistics with those of the source data using test-time prompt tuning. Finally, we use the evaluation metrics, PSNR and SSIM, to evaluate the proposed method under four tasks: rain removal, snow removal, fog removal, and raindrop removal. The results demonstrate that the proposed method achieves superior performance compared to the existing state-of-the-art.
Keywords:
Transformer architecture
pretrain-finetuning
soft prompts
contrast learning
distribution alignment strategy
Transformer architecture
pretrain-finetuning
soft prompts
contrast learning
distribution alignment strategy