Unified multimodal conditional framework for unsupervised anomaly detection

doi:10.1007/s41060-025-00745-8

Unified multimodal conditional framework for unsupervised anomaly detection

2025-04-01

0

PRE

AI

DOI：10.1007/s41060-025-00745-8

Original

Original request for help

Save

Abstract

En 中文

Unsupervised anomaly detection using generative adversarial networks (GANs) has gained significant traction in medical applications. However, existing GAN-based methods are often limited to detecting a single pathology per model, restricting their utility in diverse clinical environments. This paper introduces an innovative unsupervised multimodal approach to the detection of medical anomalies and image generation through a unified conditional framework. Our proposed model, the residual attention conditional GAN (RA-cGAN), consists of two conditional networks: a GAN and an encoder-tailored to specific conditions such as dataset type or region of interest (ROI). The GAN generates realistic images conditioned on these inputs, enabling multimodal image generation alongside anomaly detection, while the encoder maps normal images to their latent representations, facilitating efficient anomaly detection across multiple modalities. Uniquely, RA-cGAN is trained exclusively on normal data in a fully unsupervised manner, enabling generalized anomaly detection and multimodal image generation across diverse clinical contexts without requiring separate models. This unified framework not only simplifies training but also leverages multimodal information to improve generalization. Furthermore, our model leverages depthwise separable convolution (DSC) to improve computational efficiency and integrates the convolutional block attention module (CBAM) to emphasize relevant image regions, all while maintaining low computational complexity. We validate RA-cGAN on pulmonary and brain datasets and achieve state-of-the-art results on the MVTec AD benchmark. These results demonstrate the efficacy of the model in multimodal unsupervised anomaly detection and image generation, highlighting its potential for diverse clinical applications.

Keywords:

Generative adversarial network

Unified conditional framework

Unsupervised multimodal anomaly detection

Depthwise separable convolution

Convolutional block attention module

Unified multimodal conditional framework for unsupervised anomaly detection

Abstract

Journal

Organization