A linear feature extraction for multiclass classification problems based on class mean and covariance discriminant information.

Abstract

A parametric linear feature extraction method is proposed for multiclass classification. The skeleton of the proposed method consists of two types of schemes that are complementary to each other with regard to the discriminant information used. The approximate pairwise accuracy criterion (aPAC) and the common-mean feature extraction (CMFE) are chosen to exploit the discriminant information about class mean and about class covariance, respectively. Choosing aPAC rather than the linear discriminant analysis (LDA) can also resolve the problem of overemphasized large distances introduced by LDA, while maintaining other decent properties of LDA. To alleviate the suboptimum problem caused by a direct cascading of the two different types of schemes, there should be a mechanism for sorting and merging features based on their effectiveness. Usage of a sample-based classification error estimation for evaluation of effectiveness of features usually costs a lot of computational time. Therefore, we develop a fast spanning-tree-based parametric classification accuracy estimator as an intermediary for the aPAC and CMFE combination. The entire framework is parametric-based. This avoids paying a costly price in computation, which normally happens to the sample-based approach. Our experiments have shown that the proposed method can achieve a satisfactory performance on real data as well as simulated data.