.png){kind=icon}
.png){kind=icon}
Return
Learning Adaptive Parameter Representation for Event-Based Video Reconstruction
.png){kind=icon}
PRE
AI
DOI：10.1109/LSP.2024.3433403.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Event-based video reconstruction aims to generate images from asynchronous event streams, which record the intensity changes exceeding specific contrast thresholds. However, the contrast thresholds are varied among pixels with manufacturing imperfections and circumstancing interference, which causes undesirable events. It may cause the existing works to output blurry frames with unpleasing artifacts. To address this, we propose a novel two-stage framework to reconstruct images with learnable parameter representations. The learnable representation of the contrast threshold is extracted with a transformer network from corresponding asynchronous events in the first stage. Then a UNet architecture is utilized in the second stage to fuse the representations with the event encoding features to refine the decoding features in spatiotemporal dimensions. The representation learned from asynchronous events can adapt to the variety of contrast thresholds when processing event data in diverse scenes, motivating the proposed framework to generate high-quality frames. Quantitative and qualitative experimental results on the four public datasets show that our approach achieves better performance.
Keywords:
Event-based vision
video reconstruction
transformer
contrast threshold
Event-based vision
video reconstruction
transformer
contrast threshold
Journal
IF:
9.6
Papers:
1.1W
Citations:
1.7W