The sesam package that we ported to the MasPar has its roots in code written years ago before anyone knew what a parallel computer was... So the I/O scheme we devised had to be compatible with that. Sesam has built into it an I/O system with asynch transfers, multiple buffers and overlapped IO/computation. The administration and some matrix operations are carried out by the Front end, and all the data structures in sesam are based on the sparse block oriented file structure. It is therefore quite impossible to change the file structure that is used by sesam. Our scheme for parallelising the 3 major routines in the code was to gather a number of blocks to make one large so called superblock. In simple terms, the front end code knows about matrix sub-blocks, the DPU code only knows about matrices (or submatrices). It has no knowledge of sesam or the data structures used by it (infact these routines are basically low level BLAS). The parallel I/O routine has the task of combining sub-blocks to make larger superblocks that can be processed efficiently by the DPU routines. When writing to disk again the parallel I/O code splits the superblocks back into sub-blocks and writes them back out as sesam expects to find them. ... So ... the disk system is being used extensively as intermediate storage, typically for several GIGABytes of data.
In this case a virtual memory system would not be of any help (we have developed a prototype virtual memory system at para//ab.... All the I/O is explicit, and sesam has an intimate knowledge of how the data is laid out and how to do this efficiently for various architectures. The application has such a long history that new routines to make the I/O side easier would have to be very flexible, to allow the DPU code to write the results back in the same format that the FE code expects to find them. The layout of blocks on the DA is governed by the FE code and cannot be easily changed. In this case however, the use of I/O ram buffers will help immensely as the I/O throughput will be increased significantly.
The most useful file system support in this case is routines that can efficiently read/write data from the physcal disk, and then permute the data so that it is on the right processor in the right location. The maspar routines (for example read2dcs/write2dcs) for doing this are not general enough for our purposes (both in this and other applications) so we have written our own routines for carrying this out.
Virtual memory (in particular the MPVMS system developed here) is very useful on the MasPar in applications that access 'memory' in a fairly well defined (and regular) way. Bjarne's thesis discusses this in more detail.
-- anonymous 1993