Best practices on Lustre

  • Edit/build code in user home area whenever possible
  • Open files as read-only whenever possible
  • Read small, shared files from a single task
  • Placing too many files in a single directory negatively impacts performance.
  • Avoid using ls -l
  • Avoid frequently opening files in append mode, writing small amounts of data, and closing the file.
  • Instead of reading a small file from every task, read the entire file from one task and broadcast the contents to all other tasks.
  • If possible, use a high-level I/O library ( HDF5, NetCDF)
  • The default I/O striping width is 1 on the Shaheen II parallel filesystem. This is optimal for many cases especially when every MPI process writes to its own file resulting in as many files as number of processes used.
  • The Lustre filesystem stripe count can be modified using the following command with a maximum stripe count of 144: lfs setstripe --count [stripe-count] filename/directory

For more details, please refer to https://www.hpc.kaust.edu.sa/training/2015/cray/material/4.01_IO.pdf