Informatica

An estimation of the generalization performance of classifier is one of most important problems in pattern clasification and neural network training theory. In this paper we estimate the generalization error (mean expected probability of classification) for randomized linear zero empirical error (RLZEE) classifier which was considered by Raudys, Dičiūnas and Basalykas. Instead of “non-explicit” asymptotics of a generalization error of RLZEE classifier for centered multivariate spherically Gaussian classes proposed by Basalykas et al. (1996) we obtain an “explicit” and more simple asymptotics. We also present the numerical simulations illustrating our theoretical results and comparing them with each other and previously obtained results.

An analytical equation for a generalization error of minimum empirical error classifier is derived for a case when true classes are spherically Gaussian. It is compared with the generalization error of a mean squared error classifier – a standard Fisher linear discriminant function. In a case of spherically distributed classes the generalization error depends on a distance between the classes and a number of training samples. It depends on an intrinsic dimensionality of a data only via initialization of a weight vector. If initialization is successful the dimensionality does not effect the generalization error. It is concluded advantageous conditions to use artificial neural nets are to classify patterns in a changing environment, when intrinsic dimensionality of the data is low or when the number of training sample vectors is really large.

Small training sample effects common in statistical classification and artificial neural network classifier design are discussed. A review of known small sample results are presented, and peaking phenomena related to the increase in the number of features and the number of neurons is discussed.

The smoothing constant λ is the most important characteristic of the nonparametric Parzen window classifier (PWC). The PWC tends to a one-nearest neighbour classifier as λ tends to zero and to a parametric linear Eucliden distance classifier as λ tends to infinity. An asymptotic probability of misclassification of the PWC decreases with the decrease in λ. A sensitivity of the PWC to a finiteness of the training data depends on a true-intrinsic dimensionality of the data, and it increases with the decrease in the value of λ. It is proposed to determine an optimal value of the smoothing constant from a smoothed empirical graph of the dependence of an expected probability of misclassification on the value of λ. The graph can be estimated by means of leaving-one-out or hold-out methods simultaneously for a number of values of λ chosen from the interval (0.001–1000) in a logarithmic scale.