1. Home
  2. Issues
  3. Volume 35, Issue 4 (2024)
  4. MRI Brain Tumour Segmentation Using Mult ...

Informatica


MRI Brain Tumour Segmentation Using Multiscale Attention U-Net
Volume 35, Issue 4 (2024), pp. 751–774
Pub. online: 30 October 2024      Type: Research Article      Open accessOpen Access
Received
1 March 2024
Accepted
1 October 2024
Published
30 October 2024

Abstract

Focusing on the problems of failing to make full use of spatial context information and limited local receptive field when U-Net is utilized to solve MRI brain tumour segmentation, a novel 3D multi-scale attention U-Net method, i.e. MAU-Net, is proposed in this paper. Firstly, a Mixed Depth-wise Convolution (MDConv) module is introduced in the encoder and decoder, which leverages various convolution kernels to extract the multi-scale features of brain tumour images, and effectively strengthens the feature expression of the brain tumour lesion region in the up and down sampling. Secondly, a Context Pyramid Module (CPM) combining multi-scale and attention is embedded in the skip connection position to achieve the combination of local feature enhancement at multi-scale with global feature correlation. Finally, MAU-Net adopts Self-ensemble in the decoding process to achieve complementary detailed features of sampled brain tumour images at different scales, thereby further improving segmentation performance. Ablation and comparison experiment results on the publicly available BraTS 2019/2020 datasets well validate its effectiveness. It respectively achieves the Dice Similarity Coefficients (DSC) of 90.6%/90.2%, 82.7%/82.8%, and 77.9%/78.5% on the whole tumour (WT), tumour core (TC) and enhanced tumour (ET) segmentation. Additionally, on the BraTS 2021 training set, the DSC for WT, TC, and ET reached 93.7%, 93.2%, and 88.9%, respectively.

References

 
Akbar, A.S., Fatichah, C., Suciati, N. (2022). Single level UNet3D with multipath residual attention block for brain tumor segmentation. Journal of King Saud University-Computer and Information Sciences, 34(6), 3247–3258.
 
Baid, U., Ghodasara, S., Mohan, S., Bilello, M., Calabrese, E., Colak, E., Farahani, K., Kalpathy-Cramer, J., Kitamura, F.C., Pati, S., Prevedello, L.M., Rudie, J.D., Sako, C., Shinohara, R.T., Bergquist, T., Chai, R., Eddy, J., Elliott, J., Reade, W., Schaffter, T., Yu, T., Zheng, J., Moawad, A.W., Otavio, C.L., McDonnell, O., Miller, E., Moron, F.E., Oswood, M.C., Shih, R.Y., Siakallis, L., Bronstein, Y., Mason, J.R., Miller, A.F., Choudhary, G., Agarwal, A., Besada, C.H., Derakhshan, J.J., Diogo, M.C., Do-Dai, D.D., Farage, L., Go, J.L., Hadi, M., Hill, V.B., Michael, I., Joyner, D., Lincoln, C., Lotan, E., Miyakoshi, A., Sanchez-Montano, M., Nath, J., Nguyen, X.V., Nicolas-Jilwan, M., Ortiz, J.J., Ozturk, K., Petrovic, B.D., Shah, C., Shah, L.M., Sharma, M., Simsek, O., Singh, A.K., Soman, S., Statsevych, V., Weinberg, B.D., Young, R.J., Ikuta, I., Agarwal, A.K., Cambron, S.C., Silbergleit, R., Dusoi, A., Postma, A.A., et al. (2021). The RSNA-ASNR-MICCAI BraTS 2021 benchmark on brain tumor segmentation and radiogenomic classification. arXiv preprint arXiv:2107.02314.
 
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C. (2017). Advancing the Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data, 4, 170117.
 
Bao, S., Chung, A.C. (2018). Multi-scale structured CNN with label consistency for brain MR image segmentation. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, 6(1), 113–117.
 
Chang, Y., Zheng, Z., Sun, Y., Zhao, M., Lu, Y., Zhang, Y. (2023). DPAFNet: a residual dual-path attention-fusion convolutional neural network for multimodal brain tumor segmentation. Biomedical Signal Processing and Control, 79, 104037.
 
Chen, H., Dou, Q., Yu, L., Qin, J., Heng, P.-A. (2018). VoxResNet: deep voxelwise residual networks for brain segmentation from 3D MR images. NeuroImage, 170, 446–455.
 
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O. (2016). 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016, Lecture Notes in Computer Science, vol. 9901. Springer, Cham, pp. 424–432.
 
Cirillo, M.D., Abramian, D., Eklund, A. (2021). Vox2Vox: 3D-GAN for brain tumour segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, vol. 12658. Springer, Cham, pp. 274–284.
 
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N. et al. (2020). An image is worth 16 x 16 words: transformers for image recognition at scale. In: International Conference on Learning Representations.
 
González, S.R., Zemmoura, I., Tauber, C. (2021). 3D brain tumor segmentation and survival prediction using ensembles of convolutional neural networks. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, vol. 12659. Springer, Cham, pp. 241–254.
 
Guo, D., Wang, L., Song, T., Wang, G. (2020). Cascaded global context convolutional neural network for brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, vol. 11992. Springer, Cham, pp. 315–326.
 
Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H.R., Xu, D. (2022). UNETR: transformers for 3D medical image segmentation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 574–584.
 
Herholz, K. (2017). Brain tumors: an update on clinical PET research in gliomas. Seminars in Nuclear Medicine, 47(1), 5–17.
 
Hu, J., Shen, L., Sun, G. (2018). Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141.
 
Hussain, S., Anwar, S.M., Majid, M. (2018). Segmentation of glioma tumors in brain using deep convolutional neural network. Neurocomputing, 282, 248–261.
 
Işın, A., Direkoğlu, C., Şah, M. (2016). Review of MRI-based brain tumor image segmentation using deep learning methods. Procedia Computer Science, 102, 317–324.
 
Jia, Z., Zhu, H., Zhu, J., Ma, P. (2023). Two-branch network for brain tumor segmentation using attention mechanism and super-resolution reconstruction. Computers in Biology and Medicine, 157, 106751.
 
Jiang, Y., Zhang, Y., Lin, X., Dong, J., Cheng, T., Liang, J. (2022). SwinBTS: a method for 3D multimodal brain tumor segmentation using swin transformer. Brain Sciences, 12(6), 797.
 
Jun, W., Haoxiang, X., Wang, Z. (2021). Brain tumor segmentation using dual-path attention U-Net in 3D MRI images, Lecture Notes in Computer Science, vol. 12658. Springer, Cham, pp. 183–193.
 
Kamnitsas, K., Ledig, C., Newcombe, V.F., Simpson, J.P., Kane, A.D., Menon, D.K., Rueckert, D., Glocker, B. (2017). Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis, 36, 61–78.
 
Kirillov, A., Girshick, R., He, K., Dollár, P. (2019). Panoptic feature pyramid networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6399–6408.
 
Li, P., Li, Z., Wang, Z., Li, C., Wang, M. (2024a). mResU-Net: multi-scale residual U-Net-based brain tumor segmentation from multimodal MRI. Medical & Biological Engineering & Computing, 62(3), 641–651.
 
Li, Z., Chen, Z., Huang, H., Chen, C. (2024b). Multi-level fusion in a hybrid architecture for 3D image segmentation. In: 2024 39th Youth Academic Annual Conference of Chinese Association of Automation (YAC), Dalian, China, 2024, pp. 373–380.
 
Liu, Z., Tong, L., Chen, L., Zhou, F., Jiang, Z., Zhang, Q., Wang, Y., Shan, C., Li, L., Zhou, H. (2021a). CANet: context aware network for brain glioma segmentation. IEEE Transactions on Medical Imaging, 40(7), 1763–1777.
 
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B. (2021b). Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022.
 
Long, J., Shelhamer, E., Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440.
 
Menze, B.H., Jakab, A., Bauer, S., Kalpathy-Cramer, J., Farahani, K., Kirby, J., Burren, Y., Porz, N., Slotboom, J., Wiest, R., et al. (2014). The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Transactions on Medical Imaging, 34(10), 1993–2024.
 
Milletari, F., Navab, N., Ahmadi, S.-A. (2016). V-Net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 2016, pp. 565–571.
 
Peiris, H., Hayat, M., Chen, Z., Egan, G., Harandi, M. (2022). A robust volumetric transformer for accurate 3D tumor segmentation. In: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022, Lecture Notes in Computer Science, vol. 13435. Springer, Cham, pp. 162–172.
 
Rao, V., Sarabi, M.S., Jaiswal, A. (2015). Brain tumor segmentation with deep learning. In: MICCAI Multimodal Brain Tumor Segmentation Challenge (BraTS), pp. 56–59.
 
Ronneberger, O., Fischer, P., Brox, T. (2015). U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015, Lecture Notes in Computer Science, vol. 9351. Springer, Cham, pp. 234–241.
 
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L. (2015). Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115, 211–252.
 
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C. (2018). MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4510–4520.
 
Tan, M., Le, Q.V. (2019). MixConv: mixed depthwise convolutional kernels. arXiv preprint arXiv:1907.09595.
 
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., Le, Q.V. (2019). MnasNet: platform-aware neural architecture search for mobile. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2820–2828.
 
Urban, G., Bendszus, M., Hamprecht, F., Kleesiek, J. (2014). Multi-modal brain tumor segmentation using deep convolutional neural networks. In: MICCAI BraTS (Brain Tumor Segmentation) Challenge. Proceedings, Winning Contribution, pp. 31–35.
 
van Dijken, B.R., van Laar, P.J., Holtman, G.A., van der Hoorn, A. (2017). Diagnostic accuracy of magnetic resonance imaging techniques for treatment response evaluation in patients with high-grade glioma, a systematic review and meta-analysis. European Radiology, 27(10), 4129–4144.
 
Vijay, S., Guhan, T., Srinivasan, K., Vincent, P.D.R., Chang, C.-Y. (2023). MRI brain tumor segmentation using residual Spatial Pyramid Pooling-powered 3D U-Net. Frontiers in Public Health, 11, 1091850.
 
Vu, M.H., Nyholm, T., Löfstedt, T. (2021). Multi-decoder networks with multi-denoising inputs for tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, vol. 12658. Springer, Cham, pp. 412–423.
 
Wang, G., Li, W., Aertsen, M., Deprest, J., Ourselin, S., Vercauteren, T. (2019). Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks. Neurocomputing, 338, 34–45.
 
Wang, W., Chen, C., Ding, M., Yu, H., Zha, S., Li, J. (2021). TransBTS: multimodal brain tumor segmentation using transformer. In: Medical Image Computing and Computer Assisted Intervention–MICCAI 2021: 24th International Conference, Strasbourg, France, September 27–October 1, 2021, Proceedings, Part I 24. Springer, pp. 109–119.
 
Wang, X., Girshick, R., Gupta, A., He, K. (2018). Non-local neural networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 7794–7803.
 
Wang, Z., Zou, N., Shen, D., Ji, S. (2020). Non-local U-Nets for biomedical image segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 34(4), 6315–6322.
 
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S. (2018). CBAM: convolutional block attention module. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–19.
 
Zhang, T., Cao, S., Pu, T., Peng, Z. (2021). AGPCNet: attention-guided pyramid context networks for infrared small target detection. arXiv preprint arXiv:2111.03580.
 
Zhang, X., Zhou, X., Lin, M., Sun, J. (2018). ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6848–6856.
 
Zhao, Y.-X., Zhang, Y.-M., Liu, C.-L. (2020). Bag of tricks for 3D MRI brain tumor segmentation. In: Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Lecture Notes in Computer Science, vol. 11992. Springer, Cham, pp. 210–220.
 
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J. (2019). UNet++: redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transactions on Medical Imaging, 39(6), 1856–1867.
 
Zikic, D., Ioannou, Y., Brown, M., Criminisi, A. (2014). Segmentation of brain tumor tissues with convolutional neural networks. Proceedings MICCAI-BRATS, 36(2014), 36–39.

Biographies

Chen Bonian

B. Chen is currently a graduate student at College of Computer Science and Engineering, Dalian Minzu University, Dalian, China. His main research interests include computer vision and medical image analysis.

He Tao

T. He received his master’s degree from College of Computer Science and Engineering, Dalian Minzu University, Dalian, China. His main research interests include computer vision and medical image analysis.

Wang Weizhuo

W.Wang is currently a lecturer at college of International Business, Dalian Minzu University, Dalian, China. She received her PhD degree from Lincoln University, New Zealand. Her main research interests include sustainable finance and financial risk management.

Han Yutong
hanyt@dlnu.edu.cn

Y. Han is currently a lecturer at College of Computer Science and Engineering, Dalian Minzu University, Dalian, China. Her main research interests are database technology and medical image analysis.

Zhang Jianxin
jxzhang@dlnu.edu.cn

J. Zhang is currently a professor at College of Computer Science and Engineering, Dalian Minzu University, Dalian, China. His main research interests include computer vision and intelligent medical data processing.

Bobek Samo

S. Bobek is currently a professor at the Faculty of Economics and Business, Maribor University, Maribor, Slovenia. His main research interests are in E-commerce, business and information systems.

Zabukovsek Simona Sternad

S.S. Zabukovsek is currently a professor at the Faculty of Economics and Business, Maribor University, Maribor, Slovenia. Her main research interests are in financial management and qualitative modelling.


Full article Related articles PDF XML
Full article Related articles PDF XML

Copyright
© 2024 Vilnius University
Open access article under the CC BY license.

Keywords
brain tumour segmentation deep learning 3D U-Net multi-scale feature attention mechanism

Funding
This research was funded by the National Natural Science Foundation of China under Grant 61972062, the Applied Basic Research Project of Liaoning under grants 2023JH2/101300191 and 2023JH2/101300193, the Major Open Project of Key Laboratory for Advanced Design and Intelligent Computing of the Ministry of Education under grant ADIC2023ZD003.

Metrics
since January 2020
253

Article info
views

279

Full article
views

154

PDF
downloads

33

XML
downloads


Share


RSS