Cross-Modality Image Registration via Generating Aligned Image Using Reference-Augmented Framework

doi:10.1109/jbhi.2025.3642431

Cross-Modality Image Registration via Generating Aligned Image Using Reference-Augmented Framework

2026-07-01

0

PRE

AI

Mohammed A. Al‐masni *

DOI：10.1109/jbhi.2025.3642431

Original

Original request for help

Save

Abstract

En 中文

Aligning a pair of cross-modality images (e.g., MR-CT, CBCT-CT) is important, yet conventional approaches, including registration or image-to-image (I2I) translation methods often have limitations. To overcome these challenges, we introduce a Register by Generation (RbG) framework, a novel 2D deep learning approach designed to generate images that are structurally well-aligned with the fixed image while preserving the detailed intensity and contrast of the moving image, which we refer to as the reference image. Our approach operates in two sequential key stages: first, we employ a novel semi-global reference-augmented image synthesis network incorporating Patch Adaptive Instance Normalization (PAdaIN). This method leverages a down-sampled reference image to guide local adaptive synthesis, generating a more accurately aligned image with a reduced risk of hallucinations. In the second stage, we introduce a detailed refining reference-augmented network featuring a Deformation-Aware Cross-Attention (DACA) block, which aims to recover finer details and textures that may be missing from the initial stage. This unique component (DACA block) enables the transfer of corresponding relevant features from the reference image, effectively performing a copy-and-paste operation within the latent feature space. Additionally, we propose a novel combination of loss functions that enables self-supervised training on misaligned datasets, eliminating the need for pre-aligned data. We rigorously evaluate our method on multiple misaligned datasets using metrics focused on structural alignment and distributional consistency, demonstrating comprehensively superior performance. Furthermore, we test its robustness by simulating intentional misalignments in a well-aligned dataset. Additionally, experiments from a case study and downstream segmentation tasks highlight the broad applicability of our approach.

Keywords:

Translation

Computed tomography

Deformation

Magnetic resonance imaging

Accuracy

Image registration

Transformers

Loss measurement

Deformable models

Artificial intelligence

Cross-modality registration

reference-augmented generation

self-supervised learning

Cross-Modality Image Registration via Generating Aligned Image Using Reference-Augmented Framework

Abstract

Journal

Organization

Cited Papers