1. Home
  2. Issues
  3. Volume 31, Issue 3 (2020)
  4. Perceptual Autoencoder for Compressive S ...

Informatica


Perceptual Autoencoder for Compressive Sensing Image Reconstruction
Volume 31, Issue 3 (2020), pp. 561–578
Pub. online: 17 June 2020      Type: Research Article      Open accessOpen Access
Received
1 August 2019
Accepted
1 May 2020
Published
17 June 2020

Abstract

This paper presents a non-iterative deep learning approach to compressive sensing (CS) image reconstruction using a convolutional autoencoder and a residual learning network. An efficient measurement design is proposed in order to enable training of the compressive sensing models on normalized and mean-centred measurements, along with a practical network initialization method based on principal component analysis (PCA). Finally, perceptual residual learning is proposed in order to obtain semantically informative image reconstructions along with high pixel-wise reconstruction accuracy at low measurement rates.

References

 
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G.S., Davis, A., Dean, J., Devin, M., Ghemawat I, S., Goodfellow, Harp, A., Irving, G., Isard, M., Jia, Y., Jozefowicz, R., Kaiser, L., Kudlur, M., Levenberg, J., Mané, D., Monga, R., Moore, S., Murray, D., Olah, C., Schuster, M., Shlens, J., Steiner, B., Sutskever, I., Talwar, K., Tucker, P., Vanhoucke, V., Vasudevan, V., Viégas, F., Vinyals, O., Warden, P., Wattenberg, M., Wicke, M., Yu, Y., Zheng, X. (2015). TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Online: https://www.tensorflow.org/.
 
Baldi, P., Hornik, K. (1989). Neural networks and principal component analysis: learning from examples without local minima. Neural Networks, 2, 53–58.
 
Baraniuk, R.G. (2007). Compressive sensing [lecture notes]. IEEE Signal Processing Magazine, 24, 118–121.
 
Beck, A., Teboulle, M. (2009). A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2, 183–202.
 
Becker, S., Bobin, J., Candès, E.J. (2011). NESTA: a fast and accurate first-order method for sparse recovery. SIAM Journal on Imaging Sciences, 4, 1–39.
 
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C. (2000). Image inpainting. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniqus. ACM Press/Addison-Wesley Publishing Co., pp. 417–424.
 
Bugeau, A., Bertalmio, M., Caselles, V., Sapiro, G. (2010). A comprehensive framework for image inpainting. IEEE Transactions on Image Processing, 19, 2634–2645.
 
Candes, E.J., Tao, T. (2006). Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Transactions on Information Theory, 52, 5406–5425.
 
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K. (2009). BM3D image denoising with shape-adaptive principal component analysis. In: SPARS’09-Signal Processing with Adaptive Sparse Structured Representations.
 
Dong, C., Loy, C.C., He, K., Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295–307.
 
Du, J., Xie, X., Wang, C., Shi, G. (2012). Block-based compressed sensing of images and video. Foundations and Trends in Signal Processing, 4, 297–416.
 
Du, J., Xie, X., Wang, C., Shi, G. (2018). Perceptual compressive sensing. Elsevier Neurocomputing, 328, 105–112.
 
Du, J., Xie, X., Wang, C., Shi, G., Xu, X., Wang, Y. (2019). Fully convolutional measurement network for compressive sensing image reconstruction. Elsevier Neurocomputing, 328, 105–112.
 
Duarte, M.F., Eldar, Y.C. (2011). Structured compressed sensing: from theory to applications. IEEE Transactions on Signal Processing, 59, 4053–4085.
 
Duarte, M.F., Baraniuk, R.G. (2012). Kronecker compressive sensing. IEEE Transactions on Image Processing, 21, 494–504.
 
Dumoulin, V., Visin, F. (2016). A guide to convolution arithmetic for deep learning. ArXiv preprint arXiv:1603.07285, pp. 1–13.
 
Elad, M., Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15, 3736–3745.
 
Hantao, Y., Feng, D., Shiliang, Z., Yongdong, Z., Tian, Q., Xu, C. (2019). DR2-Net: Deep Residual Reconstruction Network for Image Compressive Sensing. Neurocomputing. abs/1702.05743.
 
He, K., Zhang, X., Ren, S., Sun, J. (2015). Deep residual learning for image recognition. Multimedia Tools and Applications, 1–17. arXiv:1512.03385.
 
Ioffe, S., Szegedy, C. (2015). Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, 2015, pp. 448–456.
 
Johnson, J., Alahi, A., Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In: Springer Chinese Conference on Pattern Recognition and Computer Vision (PRCV), 2018, pp. 268–279.
 
Karpathy, A. (2017). CS231n: Convolutional Neural Networks for Visual Recognition, Spring 2017. Online: http://cs231n.github.io/neural-networks-1/, accessed: June 2019.
 
Kingma, D.P., Ba, J. (2015). Adam: a method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015).
 
Kristiadi, A. (2019). Why does L2 reconstruction loss yield blurry images? Online: https://wiseodd.github.io/techblog/2017/02/09/why-l2-blurry/, accessed: June 2019.
 
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A. (2016). ReconNet: non-iterative reconstruction of images from compressively sensed measurements. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
 
Lohit, S., Kulkarni, K., Kerviche, R., Turaga, P., Ashok, A. (2018). Convolutional neural networks for non-iterative reconstruction of compressively sensed images. IEEE Transactions on Computational Imaging, 4, 326–340.
 
Mallat, S.G., Zhifeng, Z. (2006). Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41, 3397–3415.
 
Mousavi, A., Baraniuk, R.G. (2017). Learning to invert: signal recovery via deep convolutional networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2017, pp. 2272–2276.
 
Mousavi, A., Patel, A.B., Baraniuk, R.G. (2015). A deep learning approach to structured signal recovery. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1336–1343.
 
Mousavi, A., Dasarathy, G., Baraniuk, R.G. (2017). DeepCodec: adaptive sensing and recovery via deep convolutional neural networks. In: 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 744–744.
 
Needell, D., Tropp, J.A. (2009). CoSaMP: iterative signal recovery from incomplete and inaccurate samples. Elsevier Applied and Computational Harmonic Analysis, 26, 301–321.
 
Párraga, C.A., Baldrich, R., Vanrell, M. (2010). Accurate mapping of natural scenes radiance to cone activation space: a new image dataset. In: Conference on Colour in Graphics, Imaging, and Vision, 2010. Society for Imaging Science and Technology, pp. 50–57.
 
Pati, Y.C., Yagyensh, C., Rezaiifar, R., Krishnaprasad, P.S. (1993). Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: Conference Record of The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, 1993. IEEE, pp. 40–44.
 
Ralašić, I., Seršić, D. (2019). Real-time motion detection in extremely subsampled compressive sensing video. In: 2019 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pp. 198–203.
 
Ralašić, I., Seršić, D., Petrinović, D. (2018). Off-the-shelf measurement setup for compressive imaging. IEEE Transactions on Instrumentation and Measurement, 68, 502–512.
 
Simonyan, K., Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
 
Sparse Modeling Software – optimization toolbox (2010). Online: http://spams-devel.gforge.inria.fr/index.html.
 
Takhar, D., Laska, J.N., Wakin, M.B., Duarte, M.F., Baron, D., Sarvotham, S., Kelly, K.F., Baraniuk, R.G. (2006). A new compressive imaging camera architecture using optical-domain compression. In: Computational Imaging IV, 2006. International Society for Optics and Photonics.
 
Testing dataset for learning-based compressive sensing reconstruction (2019). Available on: https://github.com/KuldeepKulkarni/ReconNet/tree/master/test/test_images.
 
Wright, S.J., Nowak, R.D., Figueiredo, M.A.T. (2009). Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 57, 2479–2493.
 
Xie, X., Wang, Y., Shi, G., Wang, C., Du, J., Han, X. (2017). Adaptive measurement network for CS image reconstruction. In: CCF Chinese Conference on Computer Vision 2017. Springer.
 
Yang, J., Wright, J., Huang, T.S., Ma, Y. (2010). Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 19, 2861–2873.

Biographies

Ralašić Ivan
ivan.ralasic@fer.hr

I. Ralasic received his BSc degree in computing and MSc degree in information and communication technology from the University of Zagreb, in 2014 and 2016, respectively. He is currently pursuing the PhD degree at the University of Zagreb, Faculty of Electrical Engineering and Computing. His current research interests include signal processing, compressive sensing, sparse modelling and machine learning. He is a Student Member of IEEE.

Seršić Damir

D. Seršić received the diploma degree and the MS and PhD degrees in electrical engineering from the University of Zagreb, Zagreb, Croatia, in 1986, 1993, and 1999, respectively. Since 1987, he has been with the Faculty of Electrical Engineering and Computing, University of Zagreb, where he is currently a full professor. His current research interests include theory and applications of wavelets, advanced signal and image processing, adaptive systems, blind source separation, and compressive sensing. Dr. Seršić is a member of the European Association for Signal Processing. From 2006 to 2008, he served as the chair for the Croatian IEEE Signal Processing Chapter.

Šegvić Siniša

S. Šegvić received his PhD degree in computer science, in 2004. He spent one year as a post-doctoral researcher at IRISA/INRIA, Rennes, France, and also at TU Graz, Austria. He is currently a full professor at UniZg-FER. His research and professional interests focus on lightweight convolutional architectures for semantic segmentation, detection, re-identification, outlier detection, and semantic forecasting.


Full article Related articles Cited by PDF XML
Full article Related articles Cited by PDF XML

Copyright
© 2020 Vilnius University
Open access article under the CC BY license.

Keywords
compressive sensing convolutional autoencoder deep learning image reconstruction perceptual loss principal component analysis

Funding
This work was supported in part by the Croatian Science Foundation under Projects IP-2014-09-2625 and IP-2019-04-6703, and in part by the European Regional Development Fund under Grant KK.01.1.1.01.0009 (DATACROSS).

Metrics
since January 2020
1722

Article info
views

1015

Full article
views

1204

PDF
downloads

287

XML
downloads


Share


RSS