Real-Time Decoding of Snapshot Compressive Imaging Using Tensor FISTA-Net

doi:10.1109/TNNLS.2023.3266998

Real-Time Decoding of Snapshot Compressive Imaging Using Tensor FISTA-Net

2024-10-01

1

PRE

AI

DOI：10.1109/TNNLS.2023.3266998

Original

Original request for help

Save

Abstract

En 中文

Snapshot compressive imaging (SCI) cameras compress high-speed videos or hyperspectral images into measurement frames. However, decoding the data frames from measurement frames is compute-intensive. Existing state-of-the-art decoding algorithms suffer from low decoding quality or heavy running time or both, which are not practical for real-time applications. In this article, we exploit the powerful learning ability of deep neural networks (DNN) and propose a novel tensor fast iterative shrinkage-thresholding algorithm net (Tensor FISTA-Net) as a real-time decoder for SCI cameras. Since SCI cameras have an accurate physical model, we can trade training time for the decoding time by generating abundant synthetic data and training a decoder on the cloud. Tensor FISTA-Net not only learns a sparse representation of the frames through convolution layers but also reduces the decoding time and memory consumption significantly through tensor operations, which makes Tensor FISTA-Net an appropriate approach for a real-time decoder. Our proposed Tensor FISTA-Net obtains an average PSNR improvement of 0.79-2.84 dB (video images) and 2.61-4.43 dB (hyperspectral images) over the state-of-the-art algorithms, along with more clear and detailed visual results on real SCI datasets, Hammer and Wheel, respectively. Our Tensor FISTA-Net reaches 45 frames per second in video datasets and 70 frames per second in hyperspectral datasets, meeting the real-time requirement. Besides, the trained model occupies only a 12-MB memory footprint, making it applicable to real-time Internet of Things (IoT) applications.

Keywords:

Deep unfolding

real-time decoding

snapshot compressive imaging (SCI)

tensor FISTA-net

training at workstation

Real-Time Decoding of Snapshot Compressive Imaging Using Tensor FISTA-Net

Abstract

Journal

Organization