Title:
A Comparison of Generalized Linear Discriminant Analysis Algorithms
A Comparison of Generalized Linear Discriminant Analysis Algorithms
Author(s)
Park, Cheong Hee
Park, Haesun
Park, Haesun
Advisor(s)
Editor(s)
Collections
Supplementary to
Permanent Link
Abstract
Linear Discriminant Analysis (LDA) is a dimension reduction method which finds
an optimal linear transformation that maximizes the class separability. However, in undersampled
problems where the number of data samples is smaller than the dimension
of data space, it is difficult to apply the LDA due to the singularity of scatter matrices
caused by high dimensionality. In order to make the LDA applicable, several generalizations
of the LDA have been proposed recently. In this paper, we present theoretical
and algorithmic relationships among several generalized LDA algorithms and compare
their computational complexities and performances in text classification and face recognition. Towards a practical dimension reduction method for high dimensional
data, an efficient algorithm is proposed, which reduces the computational complexity
greatly while achieving competitive prediction accuracies. We also present nonlinear
extensions of these LDA algorithms based on kernel methods. It is shown that a generalized
eigenvalue problem can be formulated in the kernel-based feature space, and
generalized LDA algorithms are applied to solve the generalized eigenvalue problem,
resulting in nonlinear discriminant analysis. Performances of these linear and nonlinear
discriminant analysis algorithms are compared extensively.
Sponsor
This work was supported in part by the National Science Foundation grants CCR-0204109 and ACI-
0305543. Any opinions, findings and conclusions or recommendations expressed in this material are those
of the authors and do not necessarily reflect the views of the National Science Foundation (NSF). The work
of Haesun Park has been performed while serving as a program director at the NSF and was partly supported
by IR/D from the NSF.
Date Issued
2006-01-28
Extent
Resource Type
Text
Resource Subtype
Technical Report