The problem of recursive estimation of a state of dynamic systems in the presence of time-varying outliers in observations to be processed has been considered. A learning phase used in the state estimation is investigated, assuming that the observations of a noisy output signal and that of a training one are given. A technique based on robust filtering by means of a bank of parallel Kalman filters and on the procedure of optimization of the state estimation itself is used, choosing, at each time moment, a current estimate, that ensures a minimal absolute deviation from the current value of the teaching signal. An approach, based on the relation between the mean squared deviation of state estimates from the true state and innovation sequence variance as well as on the fact that both variables achieve their minimum for the same filter from the respective Kalman filter bank, is proposed here for a working phase, where a training signal will be absent. The recursive technique based on an adaptive state estimation with optimization procedure is worked out. The results of numerical simulation of the linear discrete-time invariant (LTI) system (56) by computer using a bank, consisting of Kalman filters are given (Figs. 1–5).