Title:
On sparse representations and new meta-learning paradigms for representation learning

dc.contributor.advisor Isbell, Charles L.
dc.contributor.author Mehta, Nishant A.
dc.contributor.committeeMember Gray, Alexander G
dc.contributor.committeeMember Lebanon, Guy
dc.contributor.committeeMember Balcan, Maria-Florina
dc.contributor.committeeMember Zhang, Tong
dc.contributor.department Computer Science
dc.date.accessioned 2014-08-27T13:31:39Z
dc.date.available 2014-08-28T05:30:04Z
dc.date.created 2013-08
dc.date.issued 2013-05-15
dc.date.submitted August 2013
dc.date.updated 2014-08-27T13:31:39Z
dc.description.abstract Given the "right" representation, learning is easy. This thesis studies representation learning and meta-learning, with a special focus on sparse representations. Meta-learning is fundamental to machine learning, and it translates to learning to learn itself. The presentation unfolds in two parts. In the first part, we establish learning theoretic results for learning sparse representations. The second part introduces new multi-task and meta-learning paradigms for representation learning. On the sparse representations front, our main pursuits are generalization error bounds to support a supervised dictionary learning model for Lasso-style sparse coding. Such predictive sparse coding algorithms have been applied with much success in the literature; even more common have been applications of unsupervised sparse coding followed by supervised linear hypothesis learning. We present two generalization error bounds for predictive sparse coding, handling the overcomplete setting (more original dimensions than learned features) and the infinite-dimensional setting. Our analysis led to a fundamental stability result for the Lasso that shows the stability of the solution vector to design matrix perturbations. We also introduce and analyze new multi-task models for (unsupervised) sparse coding and predictive sparse coding, allowing for one dictionary per task but with sharing between the tasks' dictionaries. The second part introduces new meta-learning paradigms to realize unprecedented types of learning guarantees for meta-learning. Specifically sought are guarantees on a meta-learner's performance on new tasks encountered in an environment of tasks. Nearly all previous work produced bounds on the expected risk, whereas we produce tail bounds on the risk, thereby providing performance guarantees on the risk for a single new task drawn from the environment. The new paradigms include minimax multi-task learning (minimax MTL) and sample variance penalized meta-learning (SVP-ML). Regarding minimax MTL, we provide a high probability learning guarantee on its performance on individual tasks encountered in the future, the first of its kind. We also present two continua of meta-learning formulations, each interpolating between classical multi-task learning and minimax multi-task learning. The idea of SVP-ML is to minimize the task average of the training tasks' empirical risks plus a penalty on their sample variance. Controlling this sample variance can potentially yield a faster rate of decrease for upper bounds on the expected risk of new tasks, while also yielding high probability guarantees on the meta-learner's average performance over a draw of new test tasks. An algorithm is presented for SVP-ML with feature selection representations, as well as a quite natural convex relaxation of the SVP-ML objective.
dc.description.degree Ph.D.
dc.embargo.terms 2014-08-01
dc.format.mimetype application/pdf
dc.identifier.uri http://hdl.handle.net/1853/52159
dc.language.iso en_US
dc.publisher Georgia Institute of Technology
dc.subject Learning theory
dc.subject Data-dependent complexity
dc.subject Luckiness
dc.subject Dictionary learning
dc.subject Sparse coding
dc.subject Lasso
dc.subject Multi-task learning
dc.subject Meta-learning
dc.subject Learning to learn
dc.title On sparse representations and new meta-learning paradigms for representation learning
dc.type Text
dc.type.genre Dissertation
dspace.entity.type Publication
local.contributor.advisor Isbell, Charles L.
local.contributor.corporatename College of Computing
relation.isAdvisorOfPublication 3f357176-4c4b-402c-8b61-ec18ffb083a6
relation.isOrgUnitOfPublication c8892b3c-8db6-4b7b-a33a-1b67f7db2021
thesis.degree.level Doctoral
Files
Original bundle
Now showing 1 - 2 of 2
Thumbnail Image
Name:
MEHTA-DISSERTATION-2013.pdf
Size:
4.3 MB
Format:
Adobe Portable Document Format
Description:
No Thumbnail Available
Name:
source.tar.bz2
Size:
2.34 MB
Format:
Unknown data format
Description:
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
LICENSE_1.txt
Size:
3.87 KB
Format:
Plain Text
Description:
No Thumbnail Available
Name:
LICENSE.txt
Size:
3.87 KB
Format:
Plain Text
Description: