Title:
Fast Linear Discriminant Analysis using QR Decomposition and Regularization

Thumbnail Image
Files
GT-CSE-07-21.pdf (152 KB)
Authors
Park, Haesun
 Drake, Barry L.
 Lee, Sangmin
 Park, Cheong Hee
Authors
Person
Advisors
Advisors
Associated Organizations
Organizational Unit
Organizational Unit
Series
Collections
Research Publications
Supplementary to
Permanent Link
http://hdl.handle.net/1853/14462
Abstract
Linear Discriminant Analysis (LDA) is among the most optimal dimension reduction methods for classification, which provides a high degree of class separability for numerous applications from science and engineering. However, problems arise with this classical method when one or both of the scatter matrices is singular. Singular scatter matrices are not unusual in many applications, especially for high-dimensional data. For high-dimensional undersampled and oversampled problems, the classical LDA requires modification in order to solve a wider range of problems. In recent work the generalized singular value decomposition (GSVD) has been shown to mitigate the issue of singular scatter matrices, and a new algorithm, LDA/GSVD, has been shown to be very robust for many applications in machine learning. However, the GSVD inherently has a considerable computational overhead. In this paper, we propose fast algorithms based on the QR decomposition and regularization that solve the LDA/GSVD computational bottleneck. In addition, we present fast algorithms for classical LDA and regularized LDA utilizing the framework based on LDA/GSVD and preprocessing by the Cholesky decomposition. Experimental results are presented that demonstrate substantial speedup in all of classical LDA, regularized LDA, and LDA/GSVD algorithms without any sacrifice in classification performance for a wide range of machine learning applications.
Sponsor
The work of the first three authors was supported in part by the National Science Foundation grants ACI-0305543 and CCF-0621889. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
Date Issued
2007-03-23
Extent
Resource Type
Text
Resource Subtype
Technical Report
Rights Statement
Rights URI