Native Data Representation: an Efficient Wire Format for High Performance Computing
Author(s)
Advisor(s)
Editor(s)
Collections
Supplementary to:
Permanent Link
Abstract
Flexible and high-performance data exchange is becoming increasingly important. This trend is due in part to the growing interest among high-performance researchers in tool- and component-based approaches to software development. In trying to reap the well-known benefits of these approaches, the question of what communications infrastructure should be used to link the various application components arises. Traditional HPC-style communication libraries such as MPI offer good performance, but are not intended for loosely-coupled systems. Object- and metadata-based approaches like XML offer the needed plug-and-play flexibility, but with significantly lower performance. We observe that the flexibility and baseline performance of data exchange systems are strongly determined by their "wire formants," or by how they represent data for transmission in the heterogeneous environments. Upon examining the performance implications of using a number of different wire formats, we propose an alternative approach to flexible high-performance data exchange, Native Data Representation, and evaluate its current implementation in the Portable Binary I/O library.
Sponsor
Date
2001
Extent
382704 bytes
Resource Type
Text
Resource Subtype
Technical Report