Optimization for Machine Learning: SMO-MKL and Smoothing Strategies

Vishwanathan, S. V. N.

Title:

Optimization for Machine Learning: SMO-MKL and Smoothing Strategies

dc.contributor.author	Vishwanathan, S. V. N.	en_US
dc.contributor.corporatename	Purdue University	en_US
dc.date.accessioned	2011-04-27T20:14:25Z
dc.date.available	2011-04-27T20:14:25Z
dc.date.issued	2011-04-15
dc.description	S V N Vishwanathan of Purdue University presented a lecture on April 15, 2011 from 2:00 pm - 3:00 pm in room 1447 of the Klaus Advanced Computing Building on the Georgia Tech campus.	en_US
dc.description	Runtime: 57:10 minutes	en_US
dc.description.abstract	Our objective is to train $p$-norm Multiple Kernel Learning (MKL) and, more generally, linear MKL regularised by the Bregman divergence, using the Sequential Minimal Optimization (SMO) algorithm. The SMO algorithm is simple, easy to implement and adapt, and efficiently scales to large problems. As a result, it has gained widespread acceptance and SVMs are routinely trained using SMO in diverse real world applications. Training using SMO has been a long standing goal in MKL for the very same reasons. Unfortunately, the standard MKL dual is not differentiable, and therefore can not be optimised using SMO style co-ordinate ascent. In this paper, we demonstrate that linear MKL regularised with the $p$-norm squared, or with certain Bregman divergences, can indeed be trained using SMO. The resulting algorithm retains both simplicity and efficiency and is significantly faster than the state-of-the-art specialised $p$-norm MKL solvers. We show that we can train on a hundred thousand kernels in less than fifteen minutes and on fifty thousand points in nearly an hour on a single core using standard hardware.	en_US
dc.format.extent	57:10 minutes
dc.identifier.uri	http://hdl.handle.net/1853/38699
dc.language.iso	en_US	en_US
dc.publisher	Georgia Institute of Technology	en_US
dc.relation.ispartofseries	Computational Science and Engineering Seminar Series	en_US
dc.subject	Machine learning	en_US
dc.subject	Optimization	en_US
dc.title	Optimization for Machine Learning: SMO-MKL and Smoothing Strategies	en_US
dc.type	Moving Image
dc.type.genre	Lecture
dspace.entity.type	Publication
local.contributor.corporatename	College of Computing
local.contributor.corporatename	School of Computational Science and Engineering
local.relation.ispartofseries	Computational Science and Engineering Seminar Series
relation.isOrgUnitOfPublication	c8892b3c-8db6-4b7b-a33a-1b67f7db2021
relation.isOrgUnitOfPublication	01ab2ef1-c6da-49c9-be98-fbd1d840d2b1
relation.isSeriesOfPublication	97f53edf-44c2-4e20-855a-72065461737d