Parallel I/O Examples and Benchmark Codes

This archive dates from 2011 (with most content from prior to 2005) and is no longer updated. It is here for archival reference.

This directory contains example programs that use the parallel I/O facilities of multiprocessor file systems. Please keep each source package intact; in particular, please respect any copying or use restrictions that the authors have specified.

See also the list of anecdotes.

Thanks,
Ron Oldfield

Collection of example codes and benchmarks

CFS3D

• By Steve Ryan at NASA Ames
• FORTRAN; runs on iPSC/x with CFS file system
• A version of ARC3D with output (only) of 3-d matrix
• compressed tar file
• [Paper] J.S. Ryan and S. K. Weeratunga. Parallel computation of 3-D Navier-Stokes flowfields for supersonic vehicles. In 31st Aerospace Sciences Meeting and Exhibit, Reno, NV, 1993. AIAA Paper 93-0064.

LUP

• by David Womble, et al, from Sandia Labs
• C; runs on nCUBE/2
• Does an external LUP factorization; lots of reads and writes
• compressed tar file
• [Paper] David Womble, David Greenberg, Rolf Riesen, and Stephen Wheat. Out of core, out of mind: Practical parallel I/O. In Proceedings of the Scalable Parallel Libraries Conference, pages 10-16, Mississippi State University, October 1993.

Effective I/O Bandwidth Benchmark

• by Rolf Rabenseifner et al.
• C; requires MPI-I/O
• Benchmark examines first write, rewrite and read access, strided and segmented collective patterns on one file per application and non-collective access to one file per process.
• [Paper] Rolf Rabenseifner, Alice E. Koniges, Jean-Pierre Prost, and Richard Hedges. The parallel effective I/O bandwidth benchmark: b_eff_io. In Christophe Cerin and Hai Jin, editors, Parallel I/O for Cluster Computing, chapter 4, pages 107-132. Kogan Page Ltd., February 2004.

PARKBENCH (PARallel Kernels and BENCHmarks)

• a collaboration involving Fujitsu European Centre for Information Technology (FECIT), the European Centre for Medium Range Weather Forecasting (ECMWF) and the Department of Electronics and Computer Science at the University of Southampton.
• C; requires MPI-I/O.
• The I/O benchmark suite includes a set of "Low-Level" and "Kernel-Level" tests. Low-level tests are simple and used to analyze MPI-I/O and file system performance. Kernel-level tests are more complex and meant to represent I/O access patterns typical of scientific applications.
• compressed tar file.
• [Paper] Tony Hey and David Lancaster. The development of Parkbench and performance prediction. The International Journal of High Performance Computing Applications, 14(3):205-215, Fall 2000.

NAS Parallel Benchmarks (NPB)

• NPB I/O version 2.4 includes the BT I/O benchmark, a code used to test I/O performance of the Block Tridiagonal problem of the NPB.
• C; requires MPI-I/O
• go here for download instructions (requires registration)
• [Paper] Parkson Wong and Rob F Van der Wijngaart. NAS parallel benchmarks I/O version 2.4. Technical Report NAS-03-002, Computer Sciences Corporation, NASA Advanced Supercomputing (NAS) Division, NASA Ames Research Center, Moffett Field, CA 94035-1000, January 2003.

Interleaved or Random (IOR) benchmarks

• I/O benchmark codes developed by the Scalable I/O Project (SIOP) at LLNL.
• C; MPI-I/O and POSIX I/O
• performs parallel reads and writes to/from a single file using MPI-IO.
• source code.

FLASH I/O Benchmark

• simulates the I/O employed by FLASH, a block-structured adaptive mesh hydrodynamics code.
• C and FORTRAN; I/O methods use parallel HDF 5 (using MPI-I/O) and serial f77 binary.
• tar file.

Hartree-Fock Calculations

• A quantum chemistry code by Rick Kendall of Pacific Northest Laboratory (PNL).
• FORTRAN; tuned for the Intel Pargon and the Pablo trace environment.
• go here for instructions on downloading, compiling, and executing the code.
• [Paper] Phyllis E. Crandall, Ruth A. Aydt, Andrew A. Chien, and Daniel A. Reed. Input/output characteristics of scalable parallel applications. In Proceedings of Supercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.

Other Resources

SIO project.

• maintains a semi-private sample applications suite.

Parallel I/O Benchmarking Consortium

• Created by Rob Ross and others at ANL with the primary goal of standardizing how we evaluate the I/O performance of systems.
• A secondary goal is to identify important workloads and create test applications that benchmark performance for these workloads.
• Test suite includes only one test (mpi-tile-io): an application that tests the performance of MPI-I/O for non-contiguous access workload.