Series
ML@GT Seminar Series

Permanent Link
https://hdl.handle.net/1853/70935
Series Type
Event Series
Description
Associated Organization(s)
Associated Organization(s)
Organizational Unit
Organizational Unit

Filters
2018 1
1
1
1
1
1
Reset filters

Settings
Start date: 2017 × End date: 2019 × Author: Aghasi, Alireza ×

Publication Search Results

Now showing 1 - 1 of 1
No Thumbnail Available
Item

Pruning Deep Neural Networks with Net-Trim: Deep Learning and Compressed Sensing Meet

2018-03-14 , Aghasi, Alireza

We introduce and analyze a new technique for model reduction in deep neural networks. Our algorithm prunes (sparsifies) a trained network layer-wise, removing connections at each layer by addressing a convex problem. We present both parallel and cascade versions of the algorithm along with the mathematical analysis of the consistency between the initial network and the retrained model. We also discuss an ADMM implementation of Net-Trim, easily applicable to large scale problems. In terms of the sample complexity, we present a general result that holds for any layer within a network using rectified linear units as the activation. If a layer taking inputs of size N can be described using a maximum number of s non-zero weights per node, under some mild assumptions on the input covariance matrix, we show that these weights can be learned from O(slog N/s) samples.

View