A Deep Learning Model for Multi-Domain MRI Synthesis Using Generative Adversarial Networks

Han, Le Hoang Ngoc; Hien, Ngo Le Huy; Huy, Luu Van; Hieu, Nguyen Van

doi:10.15388/24-INFOR556

Informatica

A Deep Learning Model for Multi-Domain MRI Synthesis Using Generative Adversarial Networks

Volume 35, Issue 2 (2024), pp. 283–309

Le Hoang Ngoc Han Ngo Le Huy Hien Luu Van Huy Nguyen Van Hieu

https://doi.org/10.15388/24-INFOR556

Pub. online: 29 April 2024 Type: Research Article

Open Access

Received
1 July 2023

Accepted
1 April 2024

Published
29 April 2024

Abstract

In recent years, Magnetic Resonance Imaging (MRI) has emerged as a prevalent medical imaging technique, offering comprehensive anatomical and functional information. However, the MRI data acquisition process presents several challenges, including time-consuming procedures, prone motion artifacts, and hardware constraints. To address these limitations, this study proposes a novel method that leverages the power of generative adversarial networks (GANs) to generate multi-domain MRI images from a single input MRI image. Within this framework, two primary generator architectures, namely ResUnet and StarGANs generators, were incorporated. Furthermore, the networks were trained on multiple datasets, thereby augmenting the available data, and enabling the generation of images with diverse contrasts obtained from different datasets, given an input image from another dataset. Experimental evaluations conducted on the IXI and BraTS2020 datasets substantiate the efficacy of the proposed method compared to an existing method, as assessed through metrics such as Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR) and Normalized Mean Absolute Error (NMAE). The synthesized images resulting from this method hold substantial potential as invaluable resources for medical professionals engaged in research, education, and clinical applications. Future research gears towards expanding experiments to larger datasets and encompassing the proposed approach to 3D images, enhancing medical diagnostics within practical applications.

References

Ali, H., Biswas, M.R., Mohsen, F., Shah U., Alamgir A., Mousa O., Shah Z. (2022). The role of generative adversarial networks in brain MRI: a scoping review. Insights Imaging, 13, 98. https://doi.org/10.1186/s13244-022-01237-0.

Avants, B.B., Tustison, N., Song, G., (2009). Advanced normalization tools (ANTS). Insight Journal, 2(365), 1–35.

Bharti, S.K., Inani, H., Gupta, R.K. (2022). Smart photo editor using generative adversarial network: a machine learning approach. In: 2022 IEEE 2nd International Symposium on Sustainable Energy, Signal Processing and Cyber Security (ISSSC), pp. 1–6.

Chen W., Wu S., Wang S., Li Z., Yang J., Yao H., Song X. (2023). Compound attention and neighbor matching network for multi-contrast MRI super-resolution. arXiv preprint arXiv:2307.02148.

Choi, Y., Choi, M., Kim, M., Ha, J.-W., Kim, S., Choo, J. (2018). StarGAN: unified generative adversarial networks for multi-domain image-to-image translation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8789–8797.

Choi, Y., Uh, Y., Yoo, J., Ha, J.-W. (2020). StarGAN v2: diverse image synthesis for multiple domains. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8188–8197.

Dai, X., Lei, Y., Fu, Y., Curran, W.J., Liu, T., Mao, H., Yang, X. (2020). Multimodal MRI synthesis using unified generative adversarial networks. Medical Physics, 47(12), 6343–6354.

Geethanath, S., Vaughan Jr., J.T. (2019). Accessible magnetic resonance imaging: a review. Journal of Magnetic Resonance Imaging, 49(7), 65–77.

Grigas, O., Maskeliūnas, R., Damaševičius, R. (2023). Improving structural MRI preprocessing with hybrid transformer GANs. Life, 13(9), 1893. https://doi.org/10.3390/life13091893.

Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H.R., Xu, D. (2021). Swin UNETR: swin transformers for semantic segmentation of brain tumors in MRI images. In: International MICCAI Brainlesion Workshop. Springer, Cham, pp. 272–284.

Hien, N.L.H., Hoang, H.M., Tien, N.V., Hieu, N.V. (2021). Keyphrase extraction model: a new design and application on tourism information. Informatica, 45(4), 563–569.

Hien, N.L.H., Tien, T.Q., Hieu, N.V. (2020). Web crawler: design and implementation for extracting article-like contents. Cybernetics and Physics, 9(3), 144–151.

Hien, N.L.H., Van Huy, L., Van Hieu, N. (2021). Artwork style transfer model using deep learning approach. Cybernetics and Physics, 10, 127–137.

Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A. (2017). Improved training of wasserstein GANs. arXiv preprint arXiv:1704.00028.

Katti, G., Ara, S.A., Shireen, A. (2011). Magnetic resonance imaging (MRI)– a review. International Journal of Dental Clinics, 3(1), 65–70.

Kingma, D.P., Ba, J.L. (2014). Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Li, H., Paetzold, J.C., Sekuboyina, A., Kofler, F., Zhang, J., Kirschke, J.S., Wiestler, B., Menze, B. (2019). DiamondGAN: unified multi-modal generative adversarial networks for MRI sequences synthesis. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part IV 22, pp. 795–803. Springer.

Liang, Z.-P., Lauterbur, P.C. (2000). Principles of Magnetic Resonance Imaging. SPIE Optical Engineering Press, Belllingham, WA.

Liu, Y., Lei, Y., Wang, Y., Shafai-Erfani, G., Wang, T., Tian, S., Patel, P., Jani, A.B., McDonald, M., Curran, W.J., Liu, T., Zhou, J., Yang, X. (2019). Evaluation of a deep learning-based pelvic synthetic CT generation technique for MRI-based prostate proton treatment planning. Physics in Medicine & Biology, 64(20), 205022.

Massa, H.A., Johnson, J.M., McMillan, A.B. (2020). Comparison of deep learning synthesis of synthetic CTs using clinical MRI inputs. Physics in Medicine & Biology, 65(23), 23N03.

Nelson, S., Menon, R. (2022). Bijective-constrained cycle-consistent deep learning for optics-free imaging and classification. Optica, 9(1), 26–31.

Shen, L., Zhu, W., Wang, X., Xing, L., Pauly, J.M., Turkbey, B., Harmon, S.A., Sanford, T.H., Mehralivand, S., Choyke, P.L., Wood, B., Xu, D. (2020). Multi-domain image completion for random missing input data. IEEE Transactions on Medical Imaging, 40(4), 1113–1122.

Sohail, M., Riaz, M.N., Wu, J., Long, C., Li, S. (2019). Unpaired multi-contrast MR image synthesis using generative adversarial networks. In: International Workshop on Simulation and Synthesis in Medical Imaging. Springer, Cham, pp. 22–31.

Wang, T., Lei, Y., Curran, W.J., Liu, T., Yang, X. (2021). Contrast-enhanced MRI synthesis from non-contrast MRI using attention CycleGAN. In: Proceedings od SPIE 11600, Medical Imaging 2021: Biomedical Applications in Molecular, Structural, and Functional Imaging, pp. 388–393.

Wei, X., Gong, B., Liu, Z., Lu, W., Wang, L. (2018). Improving the improved training of wasserstein gans: a consistency term and its dual effect. arXiv preprint arXiv:1803.01541.

Xiang, L., Li, Y., Lin, W., Wang, Q., Shen, D. (2018). Unpaired deep cross-modality synthesis with fast training. In: Stoyanov, D. et al. (Eds.), Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, DLMIA ML-CDS 2018. Lecture Notes in Computer Science, Vol. 11045. Springer, Cham, pp. 155–164.

Zhang, J., Chi, Y., Lyu, J., Yang, W., Tian, Y. (2023). Dual arbitrary scale super-resolution for multi-contrast MRI. arXiv preprint arXiv:2307.02334.

Zhang, X., He, X., Guo, J., Ettehadi, N., Aw, N., Semanek, D., Posner, J., Laine, A., Wang, Y. (2022). PTNet3d: a 3D high-resolution longitudinal infant brain MRI synthesizer based on transformers. IEEE Transactions on Medical Imaging, 41(10), 2925–2940.

Zhu, J.-Y., Park, T., Isola, P., Efros, A.A. (2017). Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232.

Biographies

Han Le Hoang Ngoc

hanlehngoc080601@gmail.com

L.H.N. Han is a junior researcher at the Department of Software Engineering, The University of Danang – University of Science and Technology. Her research interests are data science and computer vision, with a special focus on MRI image processing. She obtained her BSc from the University of Danang – University of Science and Technology in Vietnam with a specialization in data science and artificial intelligence.

Hien Ngo Le Huy

ngolehuyhien@gmail.com

N.L.H. Hien is a postgraduate researcher at the School of Built Environment, Engineering, and Computing, Leeds Beckett University, United Kingdom. He obtained an MSc in information and technology at Leeds Beckett University under the Erasmus Mundus in Green Networking and Cloud Computing program. During the last six years, he has published research papers mainly related to the development of machine learning and deep learning models and their applications. His research interests include artificial intelligence, green ICT, and sustainable computing.

Huy Luu Van

lvhuy@dut.udn.com

L.V. Huy is a teaching assistant at the University of Danang – University of Science and Technology. His research interests include computer vision and natural language processing. He obtained his MSc and BSc degrees from the University of Danang – University of Science and Technology in Vietnam.

Hieu Nguyen Van

nvhieuqt@dut.udn.com

N.V. Hieu is a full senior lecturer at the University of Danang – University of Science and Technology. His research interests include deep learning, modelling, optimization, decision making and computational impacts. He obtained his PhD from Saint Petersburg State Institute of Technology and his MSc and BSc degrees from Saint Petersburg State University in Russia.

Full article Related articles Cited by

Open access article under the CC BY license.

Keywords

medical imaging MRI synthesis deep learning GANs

Funding

This work was funded by The University of Danang, University of Science and Technology and the Master, PhD Scholarship Programme of Vingroup Innovation Foundation (VINIF), the code number of Project: VINIF.2023.Ths.045.

Metrics

since January 2020

475 Article info views	192 Full article views
123 PDF downloads	42 XML downloads

RSS

Authors

Abstract

References

Biographies

Export citation

Copy and paste formatted citation

Download citation in file