A New Decision Making Method for Selection of Optimal Data Using the Von Neumann-Morgenstern Theorem
Volume 34, Issue 4 (2023), pp. 771–794
Pub. online: 14 September 2023
Type: Research Article
Open Access
Open Access
Received
1 April 2023
1 April 2023
Accepted
1 September 2023
1 September 2023
Published
14 September 2023
14 September 2023
Abstract
The quality of the input data is amongst the decisive factors affecting the speed and effectiveness of recurrent neural network (RNN) learning. We present here a novel methodology to select optimal training data (those with the highest learning capacity) by approaching the problem from a decision making point of view. The key idea, which underpins the design of the mathematical structure that supports the selection, is to define first a binary relation that gives preference to inputs with higher estimator abilities. The Von Newman Morgenstern theorem (VNM), a cornerstone of decision theory, is then applied to determine the level of efficiency of the training dataset based on the probability of success derived from a purpose-designed framework based on Markov networks. To the best of the author’s knowledge, this is the first time that this result has been applied to data selection tasks. Hence, it is shown that Markov Networks, mainly known as generative models, can successfully participate in discriminative tasks when used in conjunction with the VNM theorem.
The simplicity of our design allows the selection to be carried out alongside the training. Hence, since learning progresses with only the optimal inputs, the data noise gradually disappears: the result is an improvement in the performance while minimising the likelihood of overfitting.
References
Chen, A.N. (2006). Robust optimization for performance tuning of modern database systems. European Journal of Operational Research, 171, 412–429. https://doi.org/10.1016/j.ejor.2011.03.043.
Chou, P., Chuang, H.H., Chou, Y., Liang, T. (2022). Predictive analytics for customer repurchase: Interdisciplinary integration of buy till you die modeling and machine learning. European Journal of Operational Research, 296, 635–651. https://doi.org/10.1016/j.ejor.2021.04.021.
Delbaen, F., Drapeau, S., Kupper, M. (2011). A von neumann morgenstern representation result without weak continuity assumption. Journal of Mathematical Economics, 47, 401–408. https://doi.org/10.1016/j.jmateco.2011.04.002.
Dynkin, E. (1984). Gaussian and nongaussian random fields associated with markov processes. Journal of Functional Analysis, 55, 344–376. https://doi.org/10.1016/0022-1236(84)90004-1.
Fernandez Anitzine, I., Romo Argota, J.A., Fontan, F.P. (2012). Influence of training set selection in artificial neural network based propagation path loss predictions. International Journal of Antennas and Propagation, 2012. https://doi.org/10.1155/2012/351487.
García Cabello, J. (2021). A novel intelligent system for securing cash levels using markov random fields. International Journal of Intelligent Systems, 36, 4468–4490. https://doi.org/10.1002/int.22467.
García Cabello, J. (2023). Improved deep neural network performance under dynamic programming mode. Preprint. https://doi.org/10.2139/ssrn.4410415.
Gordon, J., Hernandez-Lobato, J.M. (2020). Combining deep generative and discriminative models for bayesian semi-supervised learning. Pattern Recognition, 100, 107156. https://doi.org/10.1016/j.patcog.2019.107156.
Higham, C.F., Higham, D.J. (2019). Deep learning: an introduction for applied mathematicians. Siam Review, 61, 860–891. https://doi.org/10.1137/18M1165748.
Jiang, L., Liao, H. (2022). Bounded rational reciprocal preference relation for decision making. Informatica, 33, 731–748. https://doi.org/10.15388/23-INFOR511.
Kim, K. (2006). Artificial neural networks with evolutionary instance selection for financial forecasting. Expert Systems with Applications, 30, 519–526. https://doi.org/10.1016/j.eswa.2005.10.007.
Machina, M.J. (1982). Expected utility analysis without the independence axiom. Econometrica: Journal of the Econometric Society, 50(2), 277–323. https://doi.org/10.2307/1912631.
Mirjalili, S., Hashim, S.Z.M., Sardroudi, H.M. (2012). Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Applied Mathematics and Computation, 218, 11125–11137. https://doi.org/10.1016/j.amc.2012.04.069.
Nalepa, J., Myller, M., Piechaczek, S., Hrynczenko, K., Kawulok, M. (2018). Genetic selection of training sets for (not only) artificial neural networks. In: Kozielski, S., Mrozek, D., Kasprowski, P., Małysiak-Mrozek, B., Kostrzewa, D. (Eds.), Beyond Databases, Architectures and Structures. Facing the Challenges of Data Proliferation and Growing Variety, BDAS 2018, Communications in Computer and Information Science, Vol. 928. Springer, Cham. https://doi.org/10.1007/978-3-319-99987-6_15.
Pollak, R.A. (1967). Additive von neumann-morgenstern utility functions. Econometrica, Journal of the Econometric Society, 353–4, 485–494. https://doi.org/10.2307/1905650.
Reeves, C.R., Bush, D.R. (2001). Using genetic algorithms for training data selection in RBF networks. In: Liu, H., Motoda, H. (Eds.), Instance Selection and Construction for Data Mining, The Springer International Series in Engineering and Computer Science, 608. Springer, Boston, MA, pp. 339–356. https://doi.org/10.1007/978-1-4757-3359-4_19.
Reeves, C.R., Taylor, S.J. (1998). Selection of training data for neural networks by a genetic algorithm. In: Parallel Problem Solving from Nature-PPSN V: 5th International Conference Amsterdam, The Netherlands September 27–30, 1998 Proceedings 5. Springer, pp. 633–642. https://doi.org/10.1007/BFb0056905.
Smale, S., Rosasco, L., Bouvrie, J., Caponnetto, A., Poggio, T. (2010). Mathematics of the neural response. Foundations of Computational Mathematics, 10, 67–91. https://doi.org/10.1007/s10208-009-9049-1.
Van Den Brink, R., Rusinowska, A. (2022). The degree measure as utility function over positions in graphs and digraphs. European Journal of Operational Research, 299, 1033–1044. https://doi.org/10.1016/j.ejor.2021.10.017.
Wang, L., Zhou, Y., Li, R., Ding, L. (2022). A fusion of a deep neural network and a hidden markov model to recognize the multiclass abnormal behavior of elderly people. Knowledge-Based Systems, 252, 109351. https://doi.org/10.1016/j.knosys.2022.109351.
Wong, D.F., Lu, Y., Chao, L.S. (2016). Bilingual recursive neural network based data selection for statistical machine translation. Knowledge-Based Systems, 108, 15–24. https://doi.org/10.1016/j.knosys.2016.05.003.
Yang, J., Qiu, W. (2005). A measure of risk and a decision-making model based on expected utility and entropy. European Journal of Operational Research, 164, 792–799. https://doi.org/10.1016/j.ejor.2004.01.031.
Zapf, F., Wallek, T. (2021). Comparison of data selection methods for modeling chemical processes with artificial neural networks. Applied Soft Computing, 113, 107938. https://doi.org/10.1016/j.asoc.2021.107938.
Zhang, H., Wang, Z., Liu, D. (2014). A comprehensive review of stability analysis of continuous-time recurrent neural networks. IEEE Transactions on Neural Networks and Learning Systems, 25, 1229–1262. https://doi.org/10.1109/TNNLS.2014.2317880.
Zhang, L., Suganthan, P.N. (2016). A survey of randomized algorithms for training neural networks. Information Sciences, 364, 146–155. https://doi.org/10.1016/j.ins.2016.01.039.
Biographies
GarcÍa Cabello Julia
J. García Cabello was born in Andalusia (Spain). She received the PhD degree in pure and applied mathematics from the University of Granada where she has been teaching since 1990. Prior to getting to know at the world of applied mathematics, she developed a successful career in pure algebra (known as JG Cabello). Today, she is a fully tenured professor and a full researcher at the Applied Mathematics Department of the University of Granada (Spain), where she teaches undergraduate, MBA and Executive MBA courses and conducts seminars on a wide range of mathematical business-related topics.
She is a full researcher at the Andalusian Research Institute in Data Science and Computational Intelligence. Her current research interests include the application of applied mathematics to the resolution of real problems, decision making, theoretical computer science and operational research. To this regard, her mathematical baggage (from pure algebra to applied mathematics) makes Dr. García Cabello’s research characterized by using a wide range of mathematical tools, from stochastic processes to dynamic systems. Dr. Julia García Cabello is also a regular reviewer of journals Applied Mathematics and Intelligent and Information Systems.
