1. Home
  2. Issues
  3. Volume 31, Issue 4 (2020)
  4. Long Short-Term Memory Networks for Traf ...

Informatica


Long Short-Term Memory Networks for Traffic Flow Forecasting: Exploring Input Variables, Time Frames and Multi-Step Approaches
Volume 31, Issue 4 (2020), pp. 723–749
Pub. online: 6 October 2020      Type: Research Article      Open accessOpen Access
Received
1 June 2019
Accepted
1 September 2020
Published
6 October 2020

Abstract

Traffic flow forecasting is an acknowledged time series problem whose solutions have been essentially grounded on statistical-based models. Recent times came, however, with promising results regarding the use of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory networks (LSTMs), to accurately address time series problems. Literature is, however, evasive in regard to several aspects of the conceived models and often exhibits misconceptions that may lead to important pitfalls. This study aims to conceive and find the best possible LSTM model for traffic flow forecasting while addressing several important aspects of such models such as the multitude of input features, the time frames used by the model and the employed approach for multi-step forecasting. To overcome the spatial problem of open source datasets, this study presents and describes a new dataset collected by the authors of this work. After several weeks of model fitting, Recursive Multi-Step Multi-Variate models were the ones showing better performance, strengthening the perception that LSTMs can be used to accurately forecast the traffic flow for several future timesteps.

References

 
Babu, C., Reddy, B. (2012). Predictive data mining on Average Global Temperature using variants of ARIMA models. In: IEEE International Conference On Advances In Engineering, Science And Management (ICAESM 2012), pp. 256–260. 978-81-909042-2-3.
 
Bahdanau, D., Cho, K., Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In: 6th International Conference on Learning Representations (ICLR).
 
Bayer, J., Wierstra, D., Togelius, J., Schmidhuber, J. (2009). Evolving memory cell structures for sequence learning. In: International Conference on Artificial Neural Networks, pp. 755–764. https://doi.org/10.1007/978-3-642-04277-5_76.
 
Box, G., Jenkins, G. (1976). Time Series Analysis: Forecasting and Control. Holden-Day, Minnesota. 9780816211043.
 
Breuel, T. (2017). High Performance text recognition using a hybrid convolutional LSTM implementation. In: 14th IAPR International Conference on Document Analysis and Recognition (ICDAR), pp. 11–16. https://doi.org/10.1109/ICDAR.2017.12.
 
Cai, M., Pipattanasomporn, M., Rahman, S. (2019). Day-ahead building-level load forecasts using deep learning vs. traditional time-series techniques. Applied Energy, 236, 1078–1088. https://doi.org/10.1016/j.apenergy.2018.12.042.
 
Chenbin, L., Guohua, Z., Zhihua, L. (2018). News text classification based on improved Bi-LSTM-CNN. In: 9th International Conference on Information Technology in Medicine and Education (ITME), pp. 890–893. https://doi.org/10.1109/ITME.2018.00199.
 
Choi, K., Fazekas, G., Sandler, M. (2016). Text-based LSTM networks for automatic music composition. In: 1st Conference on Computer Simulation of Musical Creativity.
 
Coca, A., Correa, D., Zhao, L. (2013). Computer-aided music composition with LSTM neural network and chaotic inspiration. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–7. https://doi.org/10.1109/IJCNN.2013.6706747.
 
Cortez, P., Rocha, M., Neves, J. (2004). Evolving time series forecasting ARMA models. Journal of Heuristics, 10(4), 415–429. https://doi.org/10.1023/B:HEUR.0000034714.09838.1e.
 
Cui, Z., Ke, R., Wang, Z.P.Y. (2018). Deep bidirectional and unidirectional LSTM recurrent neural network for network-wide traffic speed prediction. arXiv e-print 1801.02143 [cs.LG].
 
Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179–211. https://doi.org/10.1016/0364-0213(90)90002-E.
 
Fernandes, B., Silva, F., Alaiz-Moretn, H., Novais, P., Analide, C., Neves, J. (2019). Traffic flow forecasting on data-scarce environments using ARIMA and LSTM networks. Advances in Intelligent Systems and Computing, 930, 273–282. https://doi.org/10.1007/978-3-030-16181-1_26.
 
Fu, R., Zhang, Z., Li, L. (2016). Using LSTM and GRU neural network methods for traffic flow prediction. In: 31st Youth Academic Annual Conference of Chinese Association of Automation (YAC), pp. 324–328. https://doi.org/10.1109/YAC.2016.7804912.
 
Gers, F., Schmidhuber, J., Cummins, F. (2000). Learning to forget: continual prediction with LSTM. Neural computation, 12(10), 2451–2471. https://doi.org/10.1162/089976600300015015.
 
Gers, F., Eck, D., Schmidhuber, J. (2002). Applying LSTM to time series predictable through time-window approaches. Perspectives in Neural Computing, 193–200. https://doi.org/10.1007/978-1-4471-0219-9_20.
 
Graves, A., Mohamed, A., Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In: IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 6645–6649. https://doi.org/10.1109/ICASSP.2013.6638947.
 
Greff, K., Srivastava, R.K., Koutnik, J., Steunebrink, B.R., Schmidhuber, J. (2017). LSTM: a search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2222–2232. https://doi.org/10.1109/TNNLS.2016.2582924.
 
Hochreiter, S. (1998). Recurrent neural net learning and vanishing gradient. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 6(2), 107–116.
 
Hochreiter, S., Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735.
 
Huang, X., Tan, H., Lin, G., Tian, Y. (2018). A LSTM-based bidirectional translation model for optimizing rare words and terminologies. In: International Conference on Artificial Intelligence and Big Data (ICAIBD), pp. 185–189. https://doi.org/10.1109/ICAIBD.2018.8396191.
 
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., Fei-Fei, L. (2014). Large-scale video classification with convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1725–1732. https://doi.org/10.1109/CVPR.2014.223.
 
Li, K., Zhai, C., Xu, J. (2017). Short-term traffic flow prediction using a methodology based on ARIMA and RBF-ANN. In: Chinese Automation Congress (CAC), pp. 2804–2807. https://doi.org/10.1109/CAC.2017.8243253.
 
Ma, X., Tao, Z., Wang, Y., Yu, H., Wang, Y. (2015). Long short-term memory neural network for traffic speed prediction using remote microwave sensor data. Transportation Research Part C: Emerging Technologies, 54, 187–197. https://doi.org/10.1016/j.trc.2015.03.014.
 
Messina, R., Louradour, J. (2015). Segmentation-free handwritten Chinese text recognition with LSTM-RNN. In: 13th International Conference on Document Analysis and Recognition (ICDAR), pp. 171–175. https://doi.org/10.1109/ICDAR.2015.7333746.
 
Pham, V., Bluche, T., Kermorvant, C., Louradour, J. (2014). Dropout improves recurrent neural networks for handwriting recognition. In: 14th International Conference on Frontiers in Handwriting Recognition, pp. 285–290. https://doi.org/10.1109/ICFHR.2014.55.
 
Rahimilarki, R., Gao, Z., Jin, N., Zhang, A. (2019). Time-series deep learning fault detection with the application of wind turbine benchmark. In: IEEE 17th International Conference on Industrial Informatics (INDIN), pp. 1337–1342. https://doi.org/10.1109/INDIN41052.2019.8972237.
 
Sak, H., Senior, A., Beaufays, F. (2014). Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition. arXiv e-print 1402.1128 [cs.NE].
 
Serra, J., Pascual, S., Karatzoglou, A. (2018). Towards a universal neural network encoder for time series. arXiv e-print 1805.03908 [cs.LG].
 
Tian, Y., Pan, L. (2015). Predicting short-term traffic flow by long short-term memory recurrent neural network. In: IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity), pp. 153–158. https://doi.org/10.1109/SmartCity.2015.63.
 
Trianto, R., Tai, T., Wang, J. (2018). Fast-LSTM acoustic model for distant speech recognition. In: IEEE International Conference on Consumer Electronics (ICCE), pp. 1–4. https://doi.org/10.1109/ICCE.2018.8326195.
 
Van Der Voort, M., Dougherty, M., Watson, S. (1996). Combining kohonen maps with arima time series models to forecast traffic flow. Transportation Research Part C: Emerging Technologies, 4(5), 307–318. https://doi.org/10.1016/S0968-090X(97)82903-8.
 
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A., Kaiser, L., Polosukhin, I. (2017). Attention is all you need. In: 31st Conference on Neural Information Processing Systems (NIPS), pp. 5998–6008.
 
Werbos, P. (1990). Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10), 1550–1560. https://doi.org/10.1109/5.58337.
 
Williams, B. (2001). Multivariate vehicular traffic flow prediction: evaluation of ARIMAX modeling. Transportation Research Record, 1776(1), 194–200. https://doi.org/10.3141/1776-25.
 
Yao, S., Hu, S., Zhao, Y., Zhang, A., Abdelzaher, T. (2017). DeepSense: a unified deep learning framework for time-series mobile sensing data processing. In: International World Wide Web Conference Committee (IW3C2), pp. 351–360. https://doi.org/10.1145/3038912.3052577.
 
Zhang, K., Zhang, Z., Li, Z., Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499–1503. https://doi.org/10.1109/LSP.2016.2603342.
 
Zhang, S., Liu, S., Liu, M. (2017). Natural language inference using LSTM model with sentence fusion. In: 36th Chinese Control Conference (CCC), pp. 11081–11085. https://doi.org/10.23919/ChiCC.2017.8029126.
 
Zhao, Z., Chen, W., Wu, X., Chen, P., Liu, J. (2017). LSTM network: a deep learning approach for short-term traffic forecast. Intelligent Transport Systems, 11(2), 68–75. https://doi.org/10.1049/iet-its.2016.0208.
 
Zheng, J., Xu, C., Zhang, Z., Li, X. (2017). Electric load forecasting in smart grids using long-short-term-memory based recurrent neural network. In: 51st Annual Conference on Information Sciences and Systems (CISS), pp. 1–6. https://doi.org/10.1109/CISS.2017.7926112.

Biographies

Fernandes Bruno
bruno.fmf.8@gmail.com

B. Fernandes holds a Master’s degree in informatics engineering from the University of Minho, in Braga, Portugal. At this same university he is now concluding his PhD in informatics. He currently holds a doctoral grant, which allows him to be fully dedicated to his research at the ALGORITMI Centre, a research unit of the School of Engineering of the University of Minho. He is also an invited assistant professor at the same university, lecturing machine learning and intelligent systems. His current research interests include smart cities, internet of people, machine learning, multi-agent systems, blockchain, and road safety.

Silva Fabio
fabiosilva@di.uminho.pt

F. Silva obtained a PhD in informatics, in 2016, from the University of Minho in Braga, Portugal. Currently, he is a post-doc researcher at the ALGORITMI Centre at the same university. His current research interests include computational sustainability, smart cities, multi-agent support systems, and urban transportation.

Alaiz-Moreton Hector
hector.moreton@unileon.es

H. Alaiz-Moreton received his degree in computer science, performing the final project at Dublin Institute of Technology, in 2003. He received his PhD in information technologies in 2008 (University of Leon). He has worked as a lecturer since 2005 at the school of engineering at the University of Leon. His research interests include knowledge engineering, machine and deep learning, networks communication, and security. He has several works published in international conferences, as well as books and scientific papers in peer reviewed journals. He has been a member of scientific committees in conferences. He has headed several PhD thesis and research projects.

Novais Paulo
pjon@di.uminho.pt

P. Novais is a full professor of computer science at the Department of Informatics, in the University of Minho, Braga, Portugal, and a researcher at the ALGORITMI Centre. He received a PhD in computer science from the same university, in 2003. He develops scientific research in the field of artificial intelligence, namely knowledge representation and reasoning, machine learning and multi-agent systems, with applications to the areas of law and ambient intelligence.

Neves Jose
jneves@di.uminho.pt

J. Neves is an emeritus professor at the Department of Informatics at the School of Engineering at the University of Minho and is a researcher at the ALGORITMI Centre. He has his graduation in chemical engineering, MSc, PhD and habilitation degrees, respectively, from the universities of University of Coimbra (1976), Portugal, Heriot Watt (1981, 1983), Edinburgh, Scotland, and the University of Minho, Portugal (1988). He was the founder of the artificial intelligence area at the University of Minho. His research interests include, among others, artificial intelligence, machine learning, knowledge representation and reasoning, and evolutionary computing.

Analide Cesar
analide@di.uminho.pt

C. Analide is a professor at the Department of Informatics of the University of Minho and a researcher and founder member of ISLab – Synthetic Intelligence Laboratory, a branch of the ALGORITMI Centre at University of Minho. His main interests are in the areas of knowledge representation, intelligent agents and multi-agent systems, and sensorization.


Full article Cited by PDF XML
Full article Cited by PDF XML

Copyright
© 2020 Vilnius University
Open access article under the CC BY license.

Funding
This work has been supported by FCT – Fundacao para a Ciencia e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. It was also partially supported by a Portuguese doctoral grant, SFRH/BD/130125/2017, issued by FCT in Portugal.

Metrics
since January 2020
2595

Article info
views

882

Full article
views

1271

PDF
downloads

267

XML
downloads


Share


RSS