1. Home
  2. Issues
  3. Volume 30, Issue 1 (2019)
  4. Video Saliency Detection Using Motion Di ...

Informatica


Video Saliency Detection Using Motion Distinctiveness and Uniform Contrast Measure
Volume 30, Issue 1 (2019), pp. 53–72
Pub. online: 1 January 2019      Type: Research Article      Open accessOpen Access
Received
1 May 2018
Accepted
1 October 2018
Published
1 January 2019

Abstract

Saliency detection has been deeply studied in the last few years and the number of the designed computational models is increasing. Starting from the assumption that spatial and temporal information of an input video frame can provide better saliency results than using each information alone, we propose a spatio-temporal saliency model for detecting salient objects in videos. First, spatial saliency is measured at patch-level by fusing local contrasts with spatial priors to label each patch as a foreground or a background one. Then, the newly proposed motion distinctiveness feature and gradient flow field measure are used to obtain the temporal saliency maps. Finally, spatial and temporal saliency maps are fused together into one final saliency map.
On the challenging SegTrack v2 and Fukuchi benchmark datasets we significantly outperform the state-of-the-art methods.

References

 
Annum, R., Riaz, M.M., Ghafoor, A. (2018). Saliency detection using contrast enhancement and texture smoothing operations. Signal, Image and Video Processing, 12(3), 505–511.
 
Bhattacharya, S., Venkatsh, K., Gupta, S. (2017). Background estimation and motion saliency detection using total variation-based video decomposition. Signal, Image and Video Processing, 11(1), 113–121.
 
Brox, T., Malik, J. (2011). Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(3), 500–513.
 
Borji, A., Itti, L. (2013). State-of-the-art in visual attention modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(1), 185–207.
 
Borji, A., Cheng, M.-M., Jiang, H., Li, J. (2014). Salient object detection: a survey. Preprint arXiv:1411.5878.
 
Borji, A., Cheng, M.-M., Jiang, H., Li, J. (2015). Salient object detection: a benchmark. IEEE Transactions on Image Processing, 25(12), 5706–5722.
 
Chen, T., Cheng, M.-M., Tan, P., Shamir, A., Hu, S.-M. (2009). Sketch2photo: internet image montage. ACM Transactions on Graphics (TOG), 124–129.
 
Cheng, M.-M., Zhang, F.-L., Mitra, N.J., Huang, X., Hu, S.-M. (2010). Repfinder: finding approximately repeated scene elements for image editing. ACM Transactions on Graphics (TOG), 29(4), 84.
 
Cheng, M.-M., Mitra, N.J., Huang, X., Torr, P.H.S., Hu, S.-M.G. (2015). Global contrast based salient region detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3), 569–582.
 
Christopoulos, C., Skodras, A., Ebrahimi, T. (2000). The JPEG2000 still image coding system: an overview. IEEE Transactions on Consumer Electronics, 46(4), 1103–1127.
 
Cui, X., Liu, Q., Metaxas, D. (2009). Temporal spectral residual: fast motion saliency detection. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 617–620.
 
Donoser, M., Urschler, M., Hirzer, M., Bischof, H. (2009). Saliency driven total variation segmentation. In: IEEE International Conference on Computer Vision, pp. 817–824.
 
Fukuchi, K., Miyazato, K., Kimura, A., Takagi, S., Yamato, J. (2009). Saliency-based video segmentation with graph cuts and sequentially updated priors. In: IEEE International Conference on Multimedia and Expo, pp. 638–641.
 
Gao, D., Mahadevan, V., Vasconcelos, N. (2008). The discriminant center-surround hypothesis for bottom-up saliency. Advances in Neural Information Processing Systems, 497–504.
 
Goferman, S., Zelnik-Manor, L., Tal, A. (2012). Context-aware saliency detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 1915–1926.
 
Gopalakrishnan, V., Rajan, D., Hu, Y. (2012). A linear dynamical system framework for salient motion detection. IEEE Transactions on Circuits and Systems for Video Technology, 683–692.
 
Gu, G., Zhu, J., Liu, Z., Zhao, Y. (2015). Visual saliency detection based object recognition. Journal of Information Hiding and Multimedia Signal Processing, 6, 1250–1263.
 
Hamel, S., Guyader, N., Pellerin, D., Houzet, D. (2016). Contribution of color in saliency model for videos. Signal, Image and Video Processing, 10(3), 423–429.
 
Harel, J., Koch, C., Perona, P. (2006). Graph-based visual saliency. Advances in Neural Information Processing Systes, 545–5524.
 
Imamoglu, N., Shimoda, W., Zhang, C., Fang, Y., Kanezaki, A., Yanai, K., Nishida, Y. (2017). An integration of bottom-up and top-down salient cues on RGB-d data: saliency from objectness versus non-objectness. Signal, Image and Video Processing, 12(2), 1–8.
 
Itti, L., Baldi, P. (2005). A principled approach to detecting surprising events in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 631–637.
 
Itti, L., Koch, C., Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.
 
Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S. (2011). Automatic salient object segmentation based on context and shape prior. In: British Machine Vision Conference, pp. 2083–2090.
 
Kim, W., Jung, C., Kim, C. (2011). Spatiotemporal saliency detection and its applications in static and dynamic scenes. IEEE Transactions on Circuits and Systems for Video Technology, 21(4), 446–456.
 
Kim, J., Han, D., Tai, Y.-W., Kim, J. (2014). Salient region detection via high-dimensional color transform. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 883–890.
 
Li, F., Kim, T., Humayun, A., Tsai, D., Rehg, J.M. (2013). Video segmentation by tracking many figure-ground segments. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2192–2199.
 
Li, Y., Hou, X., Koch, C., Rehg, J.M., Yuille, A.L. (2014). The secrets of salient object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 280–287.
 
Luo, Y., Tian, Q. (2012). Spatio-temporal enhanced sparse feature selection for video saliency estimation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–38.
 
Mahadevan, V., Vasconcelos, N. (2010). Spatiotemporal saliency in dynamic scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1), 171–177.
 
Mancas, M., Riche, N., Leroy, J., Gosselin, B. (2011). Abnormal motion selection in crowds using bottom-up saliency. In: IEEE International Conference on Image Processing, pp. 229–232.
 
Mannan, S.K., Kennard, C., Husain, M. (2009). The role of visual salience in directing eye movements in visual object agnosia. Current Biology. R247–R248.
 
Mao, Y., Guo, B.-L., Yan, Y., Sun, W. (2015). Multiple structure based saliency detection and its application in image retrieval. Journal of Information Hiding and Multimedia Signal Processing, 6(4), 771–782.
 
Mauthner, T., Possegger, H., Waltner, G., Bischof, H. (2015). Encoding based saliency detection for videos and images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2494–2502.
 
Rahtu, E., Kannala, J., Salo, M., Heikkilä, J. (2010). Segmenting salient objects from images and videos. In: European Conference on Computer Vision, pp. 366–379.
 
Rudoy, D., Goldman, D.B., Shechtman, E., Zelnik-Manor, L. (2013). Learning video saliency from human gaze using candidate selection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1147–1154.
 
Shanableh, T. (2016). Saliency detection in mpeg and hevc video using intra-frame and inter-frame distances. Signal, Image and Video Processing, 10(4), 703–709.
 
Shi, J., Malik, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(8), 888–905.
 
Singh, A., Chu Chee-Hung, H., Pratt, M. (2015). Learning to predict video saliency using temporal superpixels. In: International Conference on Pattern Recognition Applications and Methods, pp. 201–209.
 
Wang, W., Shen, J., Porikli, F. (2015). Saliency-aware geodesic video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3395–3402.
 
Wolf, J.M. (2004). What attributes guide the deployment of visual attention and how do they do it? Nature Reviews Neuroscience, 5(6), 1–7.
 
Xue, Y., Guo, X., Cao, X. (2012). Motion saliency detection using low-rank and sparse decomposition. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1485–1488.
 
Yeh, H.-H., Liu, K.-H., Chen, C.-S. (2014). Salient object detection via local saliency estimation and global homogeneity refinement. Pattern Recognition, 47(4), 1740–1750.
 
Zhang, G.-X., Cheng, M.-M., Hu, S.-M., Martin, R.R. (2009). A shape-preserving approach to image resizing. In: Computer Graphics Forum, pp. 1897–1906.
 
Zhong, S.-H., Liu, Y., Ren, F., Zhang, J., Ren, T. (2013). Video saliency detection via dynamic consistent spatio-temporal attention modelling. In: National Conference of the American Association for Artificial Intelligence, pp. 1063–1069.

Biographies

Kalboussi Rahma
rahma.kalboussi@gmail.com

R. Kalboussi received her bachelor and engineer degree from the Higher Institute Higher Institute of Computer Science and Communication Techniques. She is currently a PhD candidate in computer sciences. She is also a researcher at the Networked Objects, Control and Communication Systems (NOCCS) Research Laboratory, National School of Engineering of Sousse, Tunisia. Her research interests include image and video processing, pattern recognition, computer vision and machine learning.

Abdellaoui Mehrez

M. Abdellaoui received his engineer, MS and PhD degrees from the National School of Engineering of Monastir, Tunisia, in 2003, 2005 and 2012, respectively. He is currently an assistant professor in Signal and Image Processing at the High Institute of Applied Technologies, University of Kairouan. He is also a researcher at the Networked Objects, Control and Communication Systems (NOCCS) Research Laboratory, National School of Engineering of Sousse, Tunisia. His research interests include image and video processing, computer vision and machine learning.

Douik Ali

A. Douik received his MSEE degree in 1990 and PhD degree in 1996, both from ENSET, University of Tunis. He is currently a full professor in signal and image processing at the National Engineering School of Sousse. He is also a researcher at the Networked Objects, Control and Communication Systems (NOCCS) Research Laboratory, National School of Engineering of Sousse, Tunisia. His research interests include image and video processing, machine learning and control.


Full article Cited by PDF XML
Full article Cited by PDF XML

Copyright
© 2019 Vilnius University
Open access article under the CC BY license.

Keywords
saliency detection motion estimation object of interest optical flow contrast measure

Metrics
since January 2020
1129

Article info
views

1159

Full article
views

906

PDF
downloads

249

XML
downloads


Share


RSS