Multisequence 3-T Image Synthesis from 64-mT Low-Field-Strength MRI Using Generative Adversarial Networks in Multiple Sclerosis
Inclusion flowchart for the (A) main group and the (B) validation group. MS = multiple sclerosis, NIH = National Institutes of Health, Penn = University of Pennsylvania.
Architecture overview: A generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN consists of three parallel pix2pix architectures, each trained on a different orthogonal plane in a two-dimensional fashion, and uses T1-weighted (T1), T2-weighted (T2), and fluid-attenuated inversion recovery (FLAIR) contrasts as channels. Low-field-strength (64-mT) scans are used as inputs to the generator, and outputs are compared with paired high-field-strength (3-T) scans by the discriminator. Each two-dimensional output section is concatenated into a volume that represents the output from each orthogonal plane. A wavelet-Fourier filter is then used to remove intensity differences between two-dimensional sections in the orthogonal planes of each output and subsequently average across axial, coronal, and sagittal outputs to create a final synthetic volume.
Improved image quality with a generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN. (A–C) Top: High-field-strength (3-T) and low-field-strength (64-mT) images and LowGAN outputs for a single participant across (A) fluid-attenuated inversion recovery (FLAIR), (B) T1-weighted (T1w), and (C) T2-weighted (T2w) contrasts. Bottom: Graphs demonstrate the structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and feature similarity index (FSIM) between high-field-strength and low-field-strength (64-mT) and high-field-strength and LowGAN outputs. a.u. = arbitrary units.****P ≤ .0001.
Tissue segmentations match 3-T values with a generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN. (A) Representative axial and coronal T1-weighted images (left) and corresponding SynthSeg segmentations (right) for 3-T, 64-mT, 64-mT plus SynthSR, and 64-mT plus LowGAN scans. (B–D) Estimated volumes for the (B) thalamus, (C) lateral ventricles, and (D) cerebral cortex. Left: Box plots show the measured volumes. Middle: Scatterplots show the relationship between the volume measured at 3 T and the volume measured in the 64-mT, SynthSR, and LowGAN synthesized outputs (reconstructed volume). The black line represents perfect correspondence between high-field-strength and synthesized volumes. Right: Box plots show the differences between 3-T volumetry and volumetry at 64 mT, 64 mT plus SynthSR, and 64 mT plus LowGAN. *P ≤ .05, **P ≤ .01, ****P ≤ .0001.
Generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN, preserves and enhances white matter lesions relative to 64 mT. (A–D) Axial (top) and coronal (bottom) high-field-strength (3-T), low-field-strength (64-mT), and LowGAN output axial fluid-attenuated inversion recovery (FLAIR) images for four different participants with white matter lesions (arrows); arrows of the same color point to the same lesion in a participant. The same sections are shown across field strengths and reconstructions. Images in a (A) 40-year-old female patient, (B) 38-year-old male patient, (C) 51-year-old female patient, and (D) 41-year-old male patient with relapsing-remitting multiple sclerosis (RRMS). Note that the LowGAN outputs are less noisy, have better delineation between gray and white matter, have increased lesion conspicuity, and do not have posterior flow-related venous hyperintensities relative to 64-mT inputs. The circle in (C) shows an example of lesions present at 3 T but not apparent on 64-mT or LowGAN outputs.
Comparison of white matter lesion segmentations. (A) Box plots show the white matter lesion contrast-to-noise ratio in fluid-attenuated inversion recovery (FLAIR) images for native 3-T, 64-mT, and generative adversarial network architecture for low- to high-field-strength image translation (LowGAN) outputs. Lines connect the same participants between modalities; P values are Bonferroni corrected. (B) Top: Scatterplot shows the logarithm of the lesion burden (total lesion volume) measured from 64-mT and LowGAN lesion masks as a function of the logarithm of the lesion burden measured from 3-T lesion masks. Bottom: Box plots show the difference between lesion burden measured at 3 T and 64 mT and between 3 T and LowGAN. (C) Box plots show the distribution of Dice scores between 64-mT and 3-T (64-mT) and between LowGAN and 3-T (LowGAN) lesion masks. Lines connect the same participants across comparisons. (D) Line graph shows the percentage of missed lesions as a function of the logarithm of lesion size for 64-mT and LowGAN-detected lesions. (E) Bar graph shows the individual Dice scores for each participant. (F) Distribution of detected lesions in Montreal Neurologic Institute space for 3 T, 64 mT, and LowGAN images. a.u. = arbitrary units. ***P ≤ .001.
Generative adversarial network architecture for low- to high-field-strength image translation (LowGAN) fluid-attenuated inversion recovery (FLAIR) outputs in an additional clinical group, successfully creating 3-T–like images. (A) Images in a 68-year-old male patient with a history of secondary progressive multiple sclerosis (SPMS) (first row), a 57-year-old female patient with a history of relapsing-remitting multiple sclerosis (RRMS) (second row); a 41-year-old female patient with a history of RRMS (third row); and a 37-year-old female patient with a history of RRMS (fourth row). (B) Images in a 30-year-old female patient with a clinically isolated syndrome (CIS) (first row), a 49-year-old female patient with a history of RRMS (second row); a 33-year-old female patient with a history of RRMS (third row), and a 24-year-old female patient with a history of RRMS (fourth row). Images in (C) a 46-year-old female patient with a history of RRMS, (D) a 63-year-old female patient with a history of cervical radiculopathy, (E) a 53-year-old female patient with a history of cervical spondylosis, (F) a 35-year-old male patient with a history of traumatic brain injury and migraine, and (G) a 41-year-old female patient with a history of lumbar radiculopathy. The performance of the LowGAN was similar to that in the main group in both preserving and enhancing (A) periventricular lesions, (B) deep white matter lesions (WMLs), as well as (C) other types of multiple sclerosis (MS) and (D–F) non-MS WMLs. One of the participants did not have WMLs (G), and LowGAN did not artificially generate any pathologic findings. Note that the LowGAN outputs are less noisy, have better delineation between gray and white matter, have increased lesion conspicuity, and do not have posterior flow-related venous hyperintensities relative to the 64-mT inputs.
Research Alert
Abstract
Background
News — Portable low-field-strength (64-mT) MRI scanners show promise for increasing access to neuroimaging for clinical and research purposes; however, these devices produce lower-quality images than conventional high-field-strength scanners.
Purpose
To develop and evaluate a deep learning architecture to generate high-field-strength quality brain images from low-field-strength inputs using paired data from patients with multiple sclerosis (MS) who underwent MRI at 64 mT and 3 T.
Materials and Methods
Adults with MS at two institutions were scanned using portable 64-mT and standard 3-T scanners, with T1-weighted, T2-weighted, and fluid-attenuated inversion recovery (FLAIR) acquisitions as part of an observational study (October 2020 to January 2022); a second validation group (January 2023 to January 2024) was also included. Using paired data, a generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN, was developed. Synthetic images were evaluated with respect to image quality (eg, structural similarity index), brain morphometry, and white matter lesions. Nonparametric Wilcoxon tests were used for comparison of image quality and morphometry, and Dice scores were used for comparison of lesion segmentations.
Results
A total of 50 participants (median age, 47 years [IQR, 38–56 years]; 38 female) were included in the main group, and 13 participants were included in the validation group (median age, 41 years [IQR 35–53 years]; 11 female). Compared with low-field-strength input images, LowGAN synthetic high-field-strength images were visually higher in quality and showed higher structural similarity index relative to actual high-field-strength images for T1-weighted (0.87 vs 0.82; P < .001) and FLAIR (0.88 vs 0.85; P < .001) contrasts. Cerebral cortex volumes in LowGAN outputs did not differ significantly from 3-T measurements (483.6 cm3 vs 482.1 cm3; P = .99). For white matter lesions, LowGAN increased lesion segmentation Dice scores relative to 3-T imaging when compared with native 64-mT images (0.32 vs 0.28; P < .001).
Conclusion
Application of LowGAN super-resolution to ultralow-field-strength MRI improved image quality compared with standard-of-care ultralow-field-strength images.
© RSNA, 2025
Article Multimedia
Credit:
Caption: Multisequence 3-T Image Synthesis from 64-mT Low-Field-Strength MRI Using Generative Adversarial Networks in Multiple Sclerosis
Credit:
Caption: Inclusion flowchart for the (A) main group and the (B) validation group. MS = multiple sclerosis, NIH = National Institutes of Health, Penn = University of Pennsylvania.
Credit:
Caption: Architecture overview: A generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN consists of three parallel pix2pix architectures, each trained on a different orthogonal plane in a two-dimensional fashion, and uses T1-weighted (T1), T2-weighted (T2), and fluid-attenuated inversion recovery (FLAIR) contrasts as channels. Low-field-strength (64-mT) scans are used as inputs to the generator, and outputs are compared with paired high-field-strength (3-T) scans by the discriminator. Each two-dimensional output section is concatenated into a volume that represents the output from each orthogonal plane. A wavelet-Fourier filter is then used to remove intensity differences between two-dimensional sections in the orthogonal planes of each output and subsequently average across axial, coronal, and sagittal outputs to create a final synthetic volume.
Credit:
Caption: Improved image quality with a generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN. (A–C) Top: High-field-strength (3-T) and low-field-strength (64-mT) images and LowGAN outputs for a single participant across (A) fluid-attenuated inversion recovery (FLAIR), (B) T1-weighted (T1w), and (C) T2-weighted (T2w) contrasts. Bottom: Graphs demonstrate the structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and feature similarity index (FSIM) between high-field-strength and low-field-strength (64-mT) and high-field-strength and LowGAN outputs. a.u. = arbitrary units.****P ≤ .0001.
Credit:
Caption: Tissue segmentations match 3-T values with a generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN. (A) Representative axial and coronal T1-weighted images (left) and corresponding SynthSeg segmentations (right) for 3-T, 64-mT, 64-mT plus SynthSR, and 64-mT plus LowGAN scans. (B–D) Estimated volumes for the (B) thalamus, (C) lateral ventricles, and (D) cerebral cortex. Left: Box plots show the measured volumes. Middle: Scatterplots show the relationship between the volume measured at 3 T and the volume measured in the 64-mT, SynthSR, and LowGAN synthesized outputs (reconstructed volume). The black line represents perfect correspondence between high-field-strength and synthesized volumes. Right: Box plots show the differences between 3-T volumetry and volumetry at 64 mT, 64 mT plus SynthSR, and 64 mT plus LowGAN. *P ≤ .05, **P ≤ .01, ****P ≤ .0001.
Credit:
Caption: Generative adversarial network architecture for low- to high-field-strength image translation, called LowGAN, preserves and enhances white matter lesions relative to 64 mT. (A–D) Axial (top) and coronal (bottom) high-field-strength (3-T), low-field-strength (64-mT), and LowGAN output axial fluid-attenuated inversion recovery (FLAIR) images for four different participants with white matter lesions (arrows); arrows of the same color point to the same lesion in a participant. The same sections are shown across field strengths and reconstructions. Images in a (A) 40-year-old female patient, (B) 38-year-old male patient, (C) 51-year-old female patient, and (D) 41-year-old male patient with relapsing-remitting multiple sclerosis (RRMS). Note that the LowGAN outputs are less noisy, have better delineation between gray and white matter, have increased lesion conspicuity, and do not have posterior flow-related venous hyperintensities relative to 64-mT inputs. The circle in (C) shows an example of lesions present at 3 T but not apparent on 64-mT or LowGAN outputs.
Credit:
Caption: Comparison of white matter lesion segmentations. (A) Box plots show the white matter lesion contrast-to-noise ratio in fluid-attenuated inversion recovery (FLAIR) images for native 3-T, 64-mT, and generative adversarial network architecture for low- to high-field-strength image translation (LowGAN) outputs. Lines connect the same participants between modalities; P values are Bonferroni corrected. (B) Top: Scatterplot shows the logarithm of the lesion burden (total lesion volume) measured from 64-mT and LowGAN lesion masks as a function of the logarithm of the lesion burden measured from 3-T lesion masks. Bottom: Box plots show the difference between lesion burden measured at 3 T and 64 mT and between 3 T and LowGAN. (C) Box plots show the distribution of Dice scores between 64-mT and 3-T (64-mT) and between LowGAN and 3-T (LowGAN) lesion masks. Lines connect the same participants across comparisons. (D) Line graph shows the percentage of missed lesions as a function of the logarithm of lesion size for 64-mT and LowGAN-detected lesions. (E) Bar graph shows the individual Dice scores for each participant. (F) Distribution of detected lesions in Montreal Neurologic Institute space for 3 T, 64 mT, and LowGAN images. a.u. = arbitrary units. ***P ≤ .001.
Credit:
Caption: Generative adversarial network architecture for low- to high-field-strength image translation (LowGAN) fluid-attenuated inversion recovery (FLAIR) outputs in an additional clinical group, successfully creating 3-T–like images. (A) Images in a 68-year-old male patient with a history of secondary progressive multiple sclerosis (SPMS) (first row), a 57-year-old female patient with a history of relapsing-remitting multiple sclerosis (RRMS) (second row); a 41-year-old female patient with a history of RRMS (third row); and a 37-year-old female patient with a history of RRMS (fourth row). (B) Images in a 30-year-old female patient with a clinically isolated syndrome (CIS) (first row), a 49-year-old female patient with a history of RRMS (second row); a 33-year-old female patient with a history of RRMS (third row), and a 24-year-old female patient with a history of RRMS (fourth row). Images in (C) a 46-year-old female patient with a history of RRMS, (D) a 63-year-old female patient with a history of cervical radiculopathy, (E) a 53-year-old female patient with a history of cervical spondylosis, (F) a 35-year-old male patient with a history of traumatic brain injury and migraine, and (G) a 41-year-old female patient with a history of lumbar radiculopathy. The performance of the LowGAN was similar to that in the main group in both preserving and enhancing (A) periventricular lesions, (B) deep white matter lesions (WMLs), as well as (C) other types of multiple sclerosis (MS) and (D–F) non-MS WMLs. One of the participants did not have WMLs (G), and LowGAN did not artificially generate any pathologic findings. Note that the LowGAN outputs are less noisy, have better delineation between gray and white matter, have increased lesion conspicuity, and do not have posterior flow-related venous hyperintensities relative to the 64-mT inputs.
MEDIA CONTACT
Register for reporter access to contact detailsCITATIONS
Download PDF
RSS