PubMed Health⌕ Search
Home/Archive/PubMed

PubMed · 10976137

Bounds on error expectation for support vector machines.

V Vapnik·O Chapelle
View original source ↗Export metadata ↓

Abstract

We introduce the concept of span of support vectors (SV) and show that the generalization ability of support vector machines (SVM) depends on this new geometrical concept. We prove that the value of the span is always smaller (and can be much smaller) than the diameter of the smallest sphere containing the support vectors, used in previous bounds (Vapnik, 1998). We also demonstrate experimentally that the prediction of the test error given by the span is very accurate and has direct application in model selection (choice of the optimal parameters of the SVM).

Explore related subjects

LearningModels, NeurologicalNeural Networks, ComputerPattern Recognition, AutomatedJournal Article

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

V Vapnik, O Chapelle. 2000. Bounds on error expectation for support vector machines.. https://doi.org/10.1162/089976600300015042

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related citations

PubMed2001

Retarded learning: rigorous results from statistical mechanics.

We study learning of probability distributions characterized by an unknown symmetry direction. Based on an entropic performance measure and the variational method of statistical mechanics we develop exact upper and lower bounds on the scaled critical number of examples below which learning of the direction is impossible. The asymptotic tightness of the bounds suggests an asymptotically optimal method for learning nonsmooth distributions.

Learning↗
PubMed2001

Properties of incremental projection learning.

We proposed a method of incremental projection learning which provides exactly the same generalization capability as that obtained by batch projection learning in the previous paper. However, properties of the method have not yet been investigated. In this paper, we analyze its properties from the following aspects: First, it is shown that some of the training data which is regarded as redundant in most incremental learning methods have potential effectiveness, i.e. they will contribute to better generalization capability in the future learning process. Based on this fact, an improved criterion for redundancy of additional training data is derived. Second, the relationship between prior and posterior learning results is investigated where effective training data is classified into two categories from the viewpoint of improving generalization capability. Finally, a simpler representation of incremental projection learning under certain conditions is given. The size of memory required for storing prior results in the representation is fixed and independent of the total number of training data.

Learning↗
PubMed2001

Subspace information criterion for model selection.

The problem of model selection is considerably important for acquiring higher levels of generalization capability in supervised learning. In this article, we propose a new criterion for model selection, the subspace information criterion (SIC), which is a generalization of Mallows's C(L). It is assumed that the learning target function belongs to a specified functional Hilbert space and the generalization error is defined as the Hilbert space squared norm of the difference between the learning result function and target function. SIC gives an unbiased estimate of the generalization error so defined. SIC assumes the availability of an unbiased estimate of the target function and the noise covariance matrix, which are generally unknown. A practical calculation method of SIC for least-mean-squares learning is provided under the assumption that the dimension of the Hilbert space is less than the number of training examples. Finally, computer simulations in two examples show that SIC works well even when the number of training examples is small.

Learning↗