Site Tools


parallel-io-mpi

**This is an old revision of the document!**

Table of Contents

Parallel I/O (MPI)

POSIX pwrite(fd, buf, n, offset) lets multiple processes write to different offsets in the same file without interfering: the offset is passed explicitly rather than shared, so there is no race on the file position. The straightforward approach in a parallel program is for each process to compute its own offset and call pwrite directly. MPI-IO builds on the same idea but adds a collective interface: each process opens the same file with MPI_File_open and writes its portion with an explicit offset. The advantage of the collective call MPI_File_write_at_all is that the runtime can merge small contributions from all processes into fewer, larger I/O operations aligned to filesystem stripe boundaries.

MPI_File fh;
MPI_File_open(MPI_COMM_WORLD, "output.bin",
              MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);
 
MPI_Offset offset = (MPI_Offset)rank * N * sizeof(double);
MPI_File_write_at_all(fh, offset, local_data, N, MPI_DOUBLE, MPI_STATUS_IGNORE);
 
MPI_File_close(&fh);

File views (MPI_File_set_view) let each process define its portion of the file using a derived datatype so that subsequent read/write calls use a logical offset within the view rather than raw byte offsets. This is the standard approach for writing distributed arrays: each process defines a view covering its slice of the global array, then calls MPI_File_write as if writing a contiguous local buffer.

parallel-io-mpi.1781169013.txt.gz ยท Last modified: by 127.0.0.1