Optimizing your I/O Performance

For many applications a technique called file striping will increase I/O performance. A file is said to be striped when read and write operations access multiple disks concurrently. The default I/O striping width is 1 on the Shaheen II parallel filesystem. This is optimal for many cases especially when every MPI process writes to its own file resulting in as many files as number of processes used.

It is advised however to increase the stripe count in situations when multiple processes write to a single shared file as with MPI-IO and parallel HDF5 or NetCDF.

The Lustre filesystem stripe count can be modified using the following command with a maximum stripe count of 144:

lfs setstripe  --count [stripe-count] filename/directory