Removing multiple files efficiently

Using the standard Linux command rm to delete multiple files on a Lustre filesystem is not recommended. Huge numbers of files deleted with the rm command will be very slow since it will provoke an increased load on the metadata server, resulting in instabilities with the filesystem, and therefore affecting all users.

It is recommended to use munlink, an optimized Lustre-specific command, as in the following example:

find ./my_testrun1 -type f -print0 | xargs -0 munlink

• find ./ my_testrun1 -type f : will search files (-type f)  in the directory my_testrun1 and all its subdirectories
• | xargs -0 munlink : xargs will then convert the list of files, line by line, into an argument for munlink. The -0 flag is related to the format of the listed files; if you use -print0 in the find command you must use -0 in the xargs command.

Once all of the files are deleted, the directory and its subdirectories can be deleted as follows:

find ./my_testrun1 -type d -empty -delete