Title:
Extreme scale data management in high performance computing

Simple item page
dc.contributor.advisor Schwan, Karsten
dc.contributor.author Lofstead, Gerald Fredrick en_US
dc.contributor.committeeMember Liu, Ling
dc.contributor.committeeMember Wolf, Matthew
dc.contributor.committeeMember Oldfield, Ron
dc.contributor.committeeMember Klasky, Scott
dc.contributor.department Computing en_US
dc.date.accessioned 2011-03-04T20:59:24Z
dc.date.available 2011-03-04T20:59:24Z
dc.date.issued 2010-11-15 en_US
dc.description.abstract Extreme scale data management in high performance computing requires consideration of the end-to-end scientific workflow process. Of particular importance for runtime performance, the write-read cycle must be addressed as a complete unit. Any optimization made to enhance writing performance must consider the subsequent impact on reading performance. Only by addressing the full write-read cycle can scientific productivity be enhanced. The ADIOS middleware developed as part of this thesis provides an API nearly as simple as the standard POSIX interface, but with the flexibilty to choose what transport mechanism(s) to employ at or during runtime. The accompanying BP file format is designed for high performance parallel output with limited coordination overheads while incorporating features to accelerate subsequent use of the output for reading operations. This pair of optimizations of the output mechanism and the output format are done such that they either do not negatively impact or greatly improve subsequent reading performance when compared to popular self-describing file formats. This end-to-end advantage of the ADIOS architecture is further enhanced through techniques to better enable asychronous data transports affording the incorporation of 'in flight' data processing operations and pseudo-transport mechanisms that can trigger workflows or other operations. en_US
dc.description.degree Ph.D. en_US
dc.identifier.uri http://hdl.handle.net/1853/37232
dc.publisher Georgia Institute of Technology en_US
dc.subject Adaptive en_US
dc.subject File systems en_US
dc.subject HPC en_US
dc.subject IO en_US
dc.subject Storage en_US
dc.subject.lcsh File organization (Computer science)
dc.subject.lcsh File organization (Computer science) Computer programs
dc.subject.lcsh Information storage and retrieval systems
dc.title Extreme scale data management in high performance computing en_US
dc.type Text
dc.type.genre Dissertation
dspace.entity.type Publication
local.contributor.advisor Schwan, Karsten
local.contributor.corporatename College of Computing
local.contributor.corporatename School of Computer Science
relation.isAdvisorOfPublication a89a7e85-7f70-4eee-a49a-5090d7e88ce6
relation.isOrgUnitOfPublication c8892b3c-8db6-4b7b-a33a-1b67f7db2021
relation.isOrgUnitOfPublication 6b42174a-e0e1-40e3-a581-47bed0470a1e
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
lofstead_gerald_f_201012_phd.pdf
Size:
2.54 MB
Format:
Adobe Portable Document Format
Description:
Download
Collections
Theses and Dissertations