Methods of selecting informative variables.

Abstract

We propose a new method for selection of the most informative variables from the set of variables which can be measured directly. The information is measured by metrics similar to those used in experimental design theory, such as determinant of the dispersion matrix of prediction or various functions of its eigenvalues. The basic model admits both population variability and observational errors, which allows us to introduce algorithms based on ideas of optimal experimental design. Moreover, we can take into account cost of measuring various variables which makes the approach more practical. It is shown that the selection of optimal subsets of variables is invariant to scale transformations unlike other methods of dimension reduction, such as principal components analysis or methods based on direct selection of variables, for instance principal variables and battery reduction. The performance of different approaches is compared using the clinical data.