Scalable tensor decompositions in high performance computing environments

dc.contributor.advisor Vuduc, Richard
dc.contributor.author Li, Jiajia
dc.contributor.committeeMember Sun, Jimeng
dc.contributor.committeeMember Çatalyürek, Ümit V.
dc.contributor.committeeMember Kolda, Tamara G.
dc.contributor.committeeMember Ucar, Bora
dc.contributor.committeeMember Bader, David A.
dc.contributor.department Computational Science and Engineering
dc.date.accessioned 2018-08-20T15:37:08Z
dc.date.available 2018-08-20T15:37:08Z
dc.date.created 2018-08
dc.date.issued 2018-07-31
dc.date.submitted August 2018
dc.date.updated 2018-08-20T15:37:08Z
dc.description.abstract This dissertation presents novel algorithmic techniques and data structures to help build scalable tensor decompositions on a variety of high-performance computing (HPC) platforms, including multicore CPUs, graphics co-processors (GPUs), and Intel Xeon Phi processors. A tensor may be regarded as a multiway array, generalizing matrices to more than two dimensions. When used to represent multifactor data, tensor methods can help analysts discover latent structure; this capability has found numerous applications in data modeling and mining in such domains as healthcare analytics, social networks analytics, computer vision, signal processing, and neuroscience, to name a few. When attempting to implement tensor algorithms efficiently on HPC platforms, there are several obstacles: the curse of dimensionality, mode orientation, tensor transformation, irregularity, and arbitrary tensor dimensions (or orders). These challenges result in non-trivial computational and storage overheads. This dissertation considers these challenges in the specific context of the two of the most popular tensor decompositions, the CANDECOMP/PARAFAC (CP) and Tucker decompositions, which are, roughly speaking, the tensor analogues to low-rank approximations in standard linear algebra. Within that context, two of the critical computational bottlenecks are the operations known as Tensor-Times-Matrix (TTM) and Matricized Tensor Times Khatri-Rao Product (MTTKRP). We consider these operations in cases when the tensor is dense or sparse. Our contributions include: 1) applying memoization to overcome the curse of dimensionality challenge that exists in a sequence of tensor operations; 2) addressing the challenge of mode orientation through a novel tensor format HICOO and proposing a parallel scheduler to avoid the locks for write-conflict memory; 3) carrying out TTM and MTTKRP operations in-place, for dense and sparse cases, to avoid tensor-matrix conversions; 4) employing different optimization and parameter tuning techniques for CPU and GPU implementations to conquer the challenges of the irregularity and arbitrary tensor orders. To validate these ideas, we have implemented them in three prototype libraries, named AdaTM, InTensLi, and ParTI!, for arbitrary-order tensors. AdaTM is a model-driven framework to generate an adaptive tensor memoization algorithm with the optimal parameters for sparse CP decomposition. InTensLi produces fast single-node implementations of dense TTM of an arbitrary dimension. ParTI! is short for a Parallel Tensor Infrastructure which is written in C, OpenMP, MPI, and NVIDIA CUDA for sparse tensors and supports MATLAB interfaces for application-level users.
dc.description.degree Ph.D.
dc.format.mimetype application/pdf
dc.identifier.uri http://hdl.handle.net/1853/60274
dc.publisher Georgia Institute of Technology
dc.subject Tensor decompositions
dc.subject Sparse tensors
dc.subject High performance computing
dc.subject Multicore
dc.subject GPUs
dc.subject Performance tuning
dc.title Scalable tensor decompositions in high performance computing environments
dc.type Text
dc.type.genre Dissertation
dspace.entity.type Publication
local.contributor.advisor Vuduc, Richard
local.contributor.corporatename College of Computing
local.contributor.corporatename School of Computational Science and Engineering
relation.isAdvisorOfPublication e9a36794-e148-4304-8933-6ae0449c21d2
relation.isOrgUnitOfPublication c8892b3c-8db6-4b7b-a33a-1b67f7db2021
relation.isOrgUnitOfPublication 01ab2ef1-c6da-49c9-be98-fbd1d840d2b1
thesis.degree.level Doctoral
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
37.4 MB
Adobe Portable Document Format
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
3.86 KB
Plain Text