Kriging Model with Fractional Euclidean Distance Matrices

. The multidimensional data model for kriging is developed using fractional Euclidean distance matrices (FEDM). The properties of FEDM are studied by means of the kernel matrix mehod. It has been shown that the factorization of kernel matrix enables us to create the embedded set being a nonsingular simplex. Using the properties of FEDM the Gaussian random ﬁeld (GRF) is constructed doing it without positive deﬁnite correlation functions usually applied for such a purpose. Created GRF can be considered as a multidimensional analogue of the Wiener process, for instance, line realizations of this GRF are namely Wiener processes. Next, the kriging method is developed based on FEDM. The method is rather simple and depends on parameters that are simply estimated by the maximum likelihood method. Computer simulation of the developed kriging extrapolator has shown that it outperforms the well known Shepard inverse distance extrapolator. Practical application of the developed approach to surrogate modelling of wastewater treatment is discussed. Theoretical investigation, computer simulation, and a practical example demonstrate that the proposed kriging model, using FEDM, can be eﬃciently applied to multidimensional data modelling and processing.


Introduction
A lot of computational models applied in the business and industry, using multidimensional data, obtained by measuring, sometimes require to fulfil expensive simulations.In these cases, the kriging method is proposed to increase the efficiency of solving modelling, prediction and optimization problems (Jones, 2001;Forrester and Keane, 2009;Xiao et al., 2018).Since the kriging method was developed as a consequence of its accurate predictions (Krige, 1951), it became one of the most promising numerical approaches in the various fields of engineering design, spatial statistics, experimental design, and so on (Kwon and Choi, 2015;Bhosekar and Ierapetritou, 2018;Carpio et al., 2017).In general, kriging is a method of interpolation for which the interpolated values are modelled by a multivariate Gaussian process, governed by prior covariances.The basic idea of kriging is to predict the value of a response function at a given point by computing a weighted average of the known values of this function in the neighbourhood of the point.Thus, kriging gives a way of anticipating, with some probability, a result associated with values of the parameters that have never been met before using the existing information (e.g. the experimental measurements).
The first step in kriging is to create a stochastic model that likely best describes the set of observed data.The assumption is usually made that the sample values are sufficiently homogeneous, as well as the correlation between two random points solely depends on the distance between them, but it is independent of their location.In this way, the stochastic homogeneous and isotropic model for kriging is constructed using the correlation functions that have to obey the conditions of positive definiteness according to Bochner's theorem (Abrahamsen, 1997).Following this approach, many models in spatial statistical research and applications are developed using the Whittle-Matern correlation family (Abrahamsen, 1997; Guttorp and Gneiting, 2005), Gneiting correlation class (Guttorp and Gneiting, 2005), etc.In this paper, the application of Euclidean distance matrices (EDM) to multivariate data modelling and kriging is considered, doing it without the standard way based on positive definite correlation functions.
EDM's have received increased attention due to their applications in recently active fields of research, such as molecular conformation in bioinformatics, dimensionality reduction in machine learning and statistics, semidefinite programming, wireless sensor network localization (Pham et al., 2008;Ghiasi et al., 2018;Koo et al., 2018;Qian et al., 2008).However, in recent years square EDM's are mostly studied in literature (Schoenberg, 1935;Gower, 1984;Weinberger et al., 2004), although the Euclidean geometry in the real world is dealing with a square root, i.e. the fractional degree of distance squares.Moreover, fractional degrees of distance squares appear in some applications as well, however many fundamental issues about their properties remain unstudied.Therefore, this work focuses on the geometry of Fractional Euclidean Distance Matrices (FEDM) and their application in the multidimensional data analysis using kriging in the context of computer modelling and solving of practical examples.Sakalauskas (2013) considered a random field with a positive definite correlation function tending its scale parameter to zero as well as infinitely increasing the variance of field, and in this way has derived that the limiting kriging method relates with inverse FEDM's (see, also Remark 1 below).Usually EDM's are studied using the Gram matrix of the observed data set, called a kernel matrix (Gower, 1984, etc.).Using the kernel matrix method (Pozniak and Sakalauskas, 2017) has shown FEDM with a strongly fractional degree of distance squares to be nonsingular, and, consequently, the derived kriging method in can be used correctly.In this paper, the multinormal data model and the kriging method are developed taking covariances proportional to the elements of kernel matrix, and, thus, avoiding the standard way, based on positive definite correlation functions.Once constructed, the kriging model can be used for data modelling, prediction, and optimization.The objective is to discuss and demonstrate the applicability of the kriging technique, based on FEDM, and to verify the computational efficiency of the developed approach.

Geometrical Properties of FEDM
Let us assume that a data set X = (x 1 , x 2 , . . ., x K ) of K vectors x i ∈ R d , 1 i K, d 1 be given, at which the values of some response function Y = (y 1 , y 2 , . . ., y K ) T are used, obtained by physical measurement, computer simulation, etc. Denote the K × K matrix of fractional degrees δ of Euclidean distance squares among pairs of vectors by where Recently the properties of matrices of Euclidean distance squares are studied most, i.e. as δ = 1 (Schoenberg, 1935;Gower, 1984;Weinberger et al., 2004).Usually the main tool for EDM study is the Gram matrix of data set X, called a kernel matrix.Thus, following this approach, the matrix is introduced, called hereinafter a kernel matrix, where The proof can be seen in Pozniak and Sakalauskas (2017), Theorem 1.Note that, if δ = 1, then the rank of kernel matrix can be less K − 1. where The proposition implied by an easily verified equality of idempotence as Let us consider FEDM geometrical properties more.
Proposition 3.Under the conditions of Proposition 1, the embedded set Z = (z 1 , z 2 , . . ., z K ) T exists such that z K ∈ R K−1 , Z is a full rank matrix, and Proof.Meanwhile, according to Proposition 1, one can get F = Z • ZT by factorization of the kernel matrix.Then, the embedded set is obtained taking first K − 1 columns of set Z, the rank of which strictly K − 1.
Note that (1) implies Next, by means of the latter formula after simple manipulations one can make sure that the proposition is true, because: Hence, the embedded set presents itself as a nonsingular K − 1-dimensional simplex.
Corollary 1.Let us consider the factorization F = Z • Z T of the kernel matrix F , constructed according to assumptions of Proposition 1, where Z is the embedded set.Assume s T c • E = 1, s c ∈ R K , and denote z c = Z T • s c .Then, according to Proposition 2: Hence, Corollary 1 shows us the geometrical sense of vector s, that displays the origin of coordinates to the point z c in the embedded space.Therefore, if s T • E = 1, it is reasonable to call s a centering vector.
The next corollary follows from (1).
Corollary 2. If under the conditions of Proposition 1, the origin in the embedded space is displaced at the point z 1 , namely, s = (1, 0, 0, . .., 0), then the elements of the kernel matrix are as follows: Corollary 3.Under the conditions of Proposition 1, the centering vector s = E/K minimizes the trace of kernel matrix.
Proof.Indeed, after simple manipulations it follows: By differentiating the latter expression with respect to s and equating the obtained derivative to zero, one can be sure of the truth of corollary.
Corollary 4. If the conditions of Proposition 1 are valid, then displays the origin of coordinates in the centre of the circumscribed sphere of the embedded simplex.
Proof.Note that the diagonal of F consists of squares of distances from vertices of the embedded simplex to the origin in the embedded space.By means of (1), one can easily verify that diag(F Since the distances from vertices to the centre of the circumscribed sphere should be the same, find the centering vector s, i.e. which obeys the condition 2 where r is the radius of the circumscribed sphere.Thus, the latter condition implies that because the inverse of matrix A exists, as proved in Pozniak and Sakalauskas (2017), Theorem 2, and, therefore, it can be used correctly.Then, using the normalization condition one can derive The latter equality means 1) that and finally, Let us consider the inverse and determinant of FEDM more in detail.Denote by (x 0 , X) the set obtained adding some point x 0 ∈ R d to the observation set X, as well as the respective vector of fractional degrees of Euclidean distance squares between x 0 and X by Now, define the extended kernel matrix, respective to FEDM Ã = 0 a T a A of the set (x 0 , X), calculated at the centering vector sT = (s ′ , s T ) and presented in the block matrix: where, due to (1): Then next theorem relates the inverse and determinant of FEDM with that of the kernel matrix.

Theorem 1. Assume FEDM
as well as the centering vector (s ′ , s) to be given, where Then the inverse of K × K lower-right submatrix of kernel matrix (2) is as follows: where Besides, the determinants of matrices F and A are related Corollary 5. Suppose the conditions of Theorem 1 to be satisfied.Then, The proof of Theorem 1 and Corollary 5 is given in Appendix.It is of interest that the right part of conclusion of Corollary 5 is invariant to the choice of the centering vector s and point x 0 .

Creation of the Gaussian Random Field Using FEDM
Statistical approach for modelling the response function by a random field has the origin in the works of Kushner (1964), Žilinskas (1985), and Mockus (1989).In this section, a probabilistic model of response surface (scalar function) is developed, the values of which are collected by observation, physical measurements or computer simulation, etc. Due to the fact that there are no further data except the measurement performance at the experimental points, the surface which represents the objective response function can be designed as a homogeneous Gaussian random field (GRF) Z(x, ω), which for each point in the variable space x ∈ R d , is a measurable function of random event ω ∈ ( , , P ) in some probability space (Mockus, 1989;Jones, 2001;Sakalauskas, 2013).To model the observed data in a probabilistic way, one has to define the probability distribution of response surface values Z(x, ω) = (Z(x 1 , ω), Z(x 2 , ω), . . ., Z(x K , ω)) in the given set of points X = (x 1 , x 2 , . . ., x K ).Consider this probability distribution with the constant mean vector: and covariance matrix where µ and β are parameters, β > 0, and F is a positively defined matrix.
Taking into account geometrical properties of FEDM, explored in the previous section, choose matrix F as a lower-right submatrix of kernel matrix (2), calculated using some point x 0 and centering vector (s ′ , s) such that s ′ + E T • s = 1, s ′ = 0. Note that matrix F is positively defined according to Proposition 1.
Thus, the multivariate Gaussian density is written down as well: However, as one can easily see that this density, using denotations of the previous section and ( 7)-( 11), can be rewritten in an equivalent manner: In order to describe the random field by its finite-dimensional (cumulative) distributions, latter ones have to obey the consistency conditions of symmetry and compatibility (Abrahamsen, 1997).This is done by the next theorem under the appropriate choice of centering vector.
Next, the symmetry of distributions ( 14) and, consequently, that of ( 15) and ( 16), is shown in the standard way for multinormal distributions using the symmetry of kernel matrix (see Section 1.4.1,Abrahamsen, 1997).
Proving the compatibility, denote the covariance matrices respective to X K and X K+1 by F K and F K+1 , that are lower-right submatrices in the respective kernel matrices.Now, one can easily notice that, due to Corollary 2, F K is the upper-left submatrix in the decomposition Thus, using the standard block matrix operations, one can make sure that: where are, respectively, a posteriori mean and variance of GRF at the point x k+1 under known Y K (Casella and Berger, 2002).Then the compatibility follows as well: However, according to Kolmogorov's consistency theorem, GRF exists having ( 16) as its finite-dimensional distributions (Khoshnevisan, 2002).

E
. Assume that the observation set is displayed on a line: then GRF is distributed on this line as a Wiener process.
Actually, FEDM of the considered set on the line is as follows: One can easily see that, according to Corollary 2, the lower-right submatrix of the kernel matrix, being the covariance matrix of the considered set, is the same as that of the well-known Wiener process:

Kriging by FEDM
The stochastic model of kriging should incorporate uncertainty about quantities in unobserved points and to quantify the uncertainty associated with the kriging estimator.Kriging gives us a way of anticipating, with some probability, a result associated with values of the parameters that have never been met before, or have been lost, to "store" the existing information (the experimental measurements), and propagate it to any situation where no measurement has been made.As it is unknown which of all function variables will be preponderant, consider them as equivalent, and thus calculate a distance between the measurement points, which now are symmetric with respect to the miscellaneous variables.Suppose that we have to predict the value of response surface y at some point fixed and the data of measurement Y = (y 1 , y 2 , . . ., y K ) T of the response surface at points of X be known.Let the data be modelled by the probabilistic Gaussian model with constant mean (12) and the covariance matrix of type (13).Then, decompose the covariance matrix β2 • F of vector Y = (Y, y) in to the covariance matrix β2 • F of the vector Y , to vector β2 • f of covariances between vector Y and y, and to variance of y, denoted by ν: According to the introduction of kriging (see Jones, 2001) it is defined by the kriging predictor a posteriori variance of the predictor and the MLE parameter of variance Let us consider the kriging method based on the probabilistic data model, derived in the previous section.
Theorem 3. Assume the set X = (x 1 , x 2 , . . ., x K ) of mutually disjoint points x i , x j ∈ R d , x i = x j , i = j , 1 i, j K, be given, at which the values Y = (y 1 , y 2 , . . ., y K ) T of some GRF realization are known, namely, Then the kriging predictor at x ∈ R d is as follows: its variance where MLE of the variance parameter Proof is given in Appendix.Thus, kriging predictor (21) turns out to a linear extrapolator y(x) = Y T • u(x), with the extrapolation weights: and variance ( 22) helps us the measure of accuracy of extrapolation.
The next property of the kriging extrapolator easily follows from (24).
Corollary 6.The weights in (24) satisfy the condition: Besides, it is easy to make sure that kriging predictor (13) coincides with the values of response at measured points.

R
1.The kriging model has also been derived by Sakalauskas (2013) studying the GRF with the positive defined covariance function of the shape and tending the parameter γ to infinity.R 2. Note that the degree δ is a perfect parameter of GRF as well, which can be estimated using the observation data.The least square estimate δ is estimated by the univariate minimization of variance parameter MLE ( 23): The properties of this estimate are explored by computer simulation in section 5.

Comparison of the Developed Kriging Extrapolator with the Shepard Extrapolator
The well-known approach for data interpolation is presented by the Shepard method (Shepard, 1968): where the weights are chosen in the following way: It is easy to see that the Shepard extrapolator also satisfies the condition A set of analytic test functions was chosen aiming to compare the developed kriging predictor with the Shepard extrapolator (Table 1).
At the beginning of the experiments, the behaviour of response of the system is generally unknown, so different analytic functions should be designed.In total six types of functions are considered (Kwon and Choi, 2015).The first test function is a Branin function (fourth-order polynomials) and it shows a dominant second-order trend.This function has an extremely complex and highly nonlinear behaviour.The second test function is a Linear function composed of polynomials and trigonometric functions, and it shows a strong first-order trend.The Rosenbrook function shows both the first and second-order trends.
Each test function is shown in Table 1.The nonlinear Haupt function is composed of trigonometric functions.The test function results indicate a second-order trend of both independent variables x and y.
Model parameters β 2 and δ have been estimated by the maximum likelihood method ( 23) and ( 25).The computer simulation experiment has been performed generating N = 200 samples of K = 20 randomly simulated points for each test function in its domain.
In the prediction problem, the value of the Euclidean fraction degree δ could be chosen so as to get the best model results.An elegant alternative is to calculate the optimal δ value according to (25).Mean and standard deviation of variance parameter β 2 and Euclidean fraction degree δ are given in Table 2, histograms are presented in Figs.7-18 (K = 20, N = 200).To compare the accuracy of the model the True Error Criterion is introduced (Kwon and Choi, 2015).It is defined as follows: The results are summarized in Table 3, and the best estimation of the true error are marked in bold for each test function.
True error comparisons results are as follows K = 20, N = 200.Therefore, the kriging predictor, being a posterior expected value of Gaussian random field, presents itself as an efficient extrapolator of scattered data, and, in turn, the variance of kriging predictor is an efficient measure of prediction or extrapolation error.

Optimization of the Filler Effectiveness of Surface Wastewater Treatment Using Mathematical Modelling
Inadequate treatment of surface wastewater may considerably impair the quality of water.With increasing urbanization, intensification of car traffic and increasing area of impervious surfaces, the pollution of surface water and a negative impact on the aquatic en-vironment are rising as well.One of surface wastewater treatment technologies, capable of reducing suspended solids, heavy metals and other pollutants, is surface wastewater treatment filters (Meškauskaitė and Marčiulaitienė, 2016;Meškauskaitė, 2017).The effectiveness of filters, filled with construction waste and biocarbon, was analysed using the kriging method with distance matrices.The developed method allows modelling filter characteristics with different filler ratios, based on the previous experimental studies of filters.Let x = (x 1 , x 2 , x 3 , x 4 ) T be a vector of filler ratios.Denote different filter characteristics: Y j i , 1 i m, 1 j K, where K = 4 is the number of experiments, m -is the number of filter characteristics, that describe capability to treat different wastes, depending on filler ratios of the filter.
These filters with different fillers are designed for treatment of the main pollutants of surface wastewater: zinc (Zn), copper (Cu).Then the experiment matrix of filler proportions was chosen as follows: T (see Table 4).
Filters data extrapolation, based on experiments by the kriging method with distance matrices (20), ( 21), has been perfomed, where FEDM  and the vector of distances from the selected extrapolation point x to the experimental points for i = 1, 2, 3, 4 is Visualization of the cleaning efficiency (in %) for the fillers proportion (BC=1-QS-SHAAC-SW) using the kriging approach proposed, is given in Figs.19-22.

Conclusions
Thus, the approach to multidimensional data modelling has been created using geometrical properties of FEDM, studied in the paper.It has been shown that the factorization of kernel matrix of FEDM enables us to create the embedded set being a nonsingular simplex.Using the properties of FEDM the Gaussian random field (GRF) is constructed doing it without positive definite correlation functions usually applied for such a purpose.Created GRF can be considered as a multidimensional analogue of the Wiener process, for instance, line realizations of this GRF are namely Wiener processes.Next, the multidimensional data kriging method is constructed that distinguishes by properties of homogeneity and isotropy.The developed model allows us to represent the information, obtained from any number of measurements of the objective function by a computational code or physical experiment.The resulting model is rather simple and depends on a small set of parameters (mean, variance, and parameter δ of FEDM), that are efficiently estimated by the ML method.The results of scattered data processing by the approach, considered for analytically computed surfaces and mathematical modelling of a wastewater filter design process, illustrate the applicability of GRF with FEDM as models for scattered multidimensional data kriging.
By analogous considerations used to get (44), one can show that the equality, similar to that in Corollary 5, takes the place of submatrices of extended kernel matrices and FEDM as well: Thus, ( 23) is a direct consequence of the latter relation.

.
After the experiment, the filtration characteristics have been presented by the measurement matrix Y = (Y 1 , Y 2 )

Table 1
Types of test functions and test function domains.

Table 2
Mean and standard deviation of variance parameter β 2 and Euclidean fraction degree δ.

Table 4
Filtration characteristics measurements.