Efficient Data-Parallel Files via Automatic Mode Detection
Abstract:
Parallel languages rarely specify parallel I/O constructs, and
existing commercial systems provide the programmer with a low-level
I/O interface. We present design principles for integrating I/O into
languages and show how these principles are applied to a
virtual-processor-oriented language. We illustrate how
machine-independent modes are used to support both high performance
and generality. We describe an automatic mode detection technique
that saves the programmer from extra syntax and low-level file system
details. We show how virtual processor file operations, typically
small by themselves, are combined into efficient large-scale file
system calls. Finally, we present a variety of benchmark results
detailing design tradeoffs and the performance of various modes.
Parallel languages rarely specify parallel I/O constructs, and
existing commercial systems provide the programmer with a low-level
I/O interface. We present design principles for integrating I/O into
languages and show how these principles are applied to a
virtual-processor-oriented language. We illustrate how
machine-independent modes are used to support both high performance
and generality. We describe an automatic mode detection technique
that saves the programmer from extra syntax and low-level file system
details. We show how virtual processor file operations, typically
small by themselves, are combined into efficient large-scale file
system calls. Finally, we present a variety of benchmark results
detailing design tradeoffs and the performance of various modes.
David Kotz --
Last modified: Fri Jan 26 16:45:36 1996