.png){kind=icon}
Return
A Robust Self-Supervised Spectral Vision Sensor
.png){kind=icon}
PRE
AI
DOI：10.1002/lpor.71921.png)
.png){kind=icon}
Original request for help.png){kind=icon}
Share.png){kind=icon}
Save
Abstract
En 中文
Computational imaging simplifies hardware complexity by shifting the burden to algorithms, unlocking the potential for compact, high-dimensional visual perception. Self-supervised learning eliminates the reliance on large-scale datasets, but is typically constrained in high-dimensional recovery by the severely underdetermined inverse problem. Here, we present a generalized hardware-algorithm co-design paradigm for high-dimensional visual perception that integrates hybrid spatiotemporal encoding with physics-driven self-supervision, and develop a robust self-supervised spectral vision sensor. The core reconstruction engine, Motion-Aware Untrained Neural Network (MAUNN), performs scene-specific offline reconstruction in a zero-shot manner, combining a cascaded physics-informed module for spectral recovery with a coordinate-based implicit estimator for motion modeling. Under an exposure-matched evaluation, MAUNN achieves a PSNR of 44.21 dB and an SSIM of 0.992 using 16 hybrid-encoded measurements, reaching a reconstruction accuracy comparable to the state-of-the-art supervised baselines without requiring paired training data. Extensive experimental evaluations on complex dynamic scenes demonstrate robust spatio-temporal-spectral reconstruction with an average spectral fidelity of 0.997, an MSE of 7.11 × 10−3, and minimal frame-to-frame fluctuations with a standard deviation of 1.02 × 10−4. Furthermore, we validate the scalability in high-throughput biomedical microscopy across large-scale megapixel fields, where it maintains this high accuracy to enable precise pixel-level semantic segmentation for pathological diagnosis.
Keywords:
artificial neural network
artificial intelligence
computer vision
spectral imaging
Journal
L
IF:
10
Papers:
1.2K
Citations:
1
Organization
Cited Papers
ShareSaveDurable and programmable ultrafast nanophotonic matrix of spectral pixels.png)
NATURE NANOTECHNOLOGY
OAAI
NATURE NANOTECHNOLOGYIF34.9
OAAIShareSaveShareSaveShareSaveSnapshot temporal compressive microscopy using an iterative algorithm with untrained neural networksOPTICS LETTERS
OPTICS LETTERSIF3.3
PREAI
ShareSaveShareSaveCircular Polarimetric Imaging with a Metamaterial Integrated Long-Wavelength Infrared Focal Plane ArrayAdvanced ScienceOAAI
Advanced ScienceIF14.1
OAAIShareSave
View more