SOEDiff: Efficient Distillation for Small Object Editing

doi:10.1145/3715915

SOEDiff: Efficient Distillation for Small Object Editing

2025-11-01

1

PRE

AI

DOI：10.1145/3715915

Original request for help

Save

Abstract

En 中文

In this article, we delve into a new task known as Small Object Editing (SOE), which focuses on text-based image inpainting within a constrained, small-sized area. Despite the remarkable success have been achieved by current image inpainting approaches, their application to the SOE task generally results in failure cases such as Object Missing, Text-Image Mismatch, and Distortion. These failures stem from the limited use of small-sized objects in training datasets and the down-sampling operations employed by U-Net models, which hinders accurate generation. To overcome these challenges, we introduce a novel training-based approach, SOEDiff, aimed at enhancing the capability of baseline models like StableDiffusion in editing small-sized objects while minimizing training costs. Specifically, our method involves two key components: SO-LoRA, which efficiently fine-tunes low-rank matrices, and Cross-scale score distillation, which leverages high-resolution predictions from the pre-trained teacher diffusion model. Our method presents significant improvements on the test dataset collected from MSCOCO and OpenImage, validating the effectiveness of our proposed method in SOE. In particular, when comparing SOEDiff with SD-I model on the OpenImage-small-val dataset, we observe a 0.99 improvement in CLIP-Score and a reduction of 2.87 in FID.

Keywords:

Diffusion Model

Small Object Editing

Low-rank Adaptation

Distillation

SOEDiff: Efficient Distillation for Small Object Editing

Abstract

Journal

Organization