KAUST Supercomputing Laboratory Newsletter 28th September

Two Factor Authentication

Two Factor Authentication is now in operation for the Shaheen login nodes. If you have not already set up 2FA, please follow the instructions at https://www.hpc.kaust.edu.sa/content/two-factor-authentication-shaheen to be able to login.

RCAC Meeting

The project submission deadline for the next RCAC meeting is 30th September 2016. Please note that the RCAC meetings are held once per month. Projects received on or before this deadline will be included in the agenda for the next RCAC meeting, scheduled to be held in October 2016. The detailed procedure and the forms are available here:

https://www.hpc.kaust.edu.sa/account-applications

Tip of the Week: Removing multiple files efficiently

Using the standard Linux command rm to delete multiple files on a Lustre filesystem is not recommended. Huge numbers of files deleted with the rm command will be very slow since it will provoke an increased load on the metadata server, resulting in instabilities with the filesystem, and therefore affecting all users.

It is recommended to use munlink, an optimized Lustre-specific command, as in the following example:

find ./my_testrun1 -type f -print0 | xargs -0 munlink
  • find ./ my_testrun1 -type f : will search files (-type f)  in the directory my_testrun1 and all its subdirectories
  • | xargs -0 munlink : xargs will then convert the list of files, line by line, into an argument for numlimk. The -0 flag is related to the format of the listed files; if you use -print0 in the find command you must use -0 in the xargs command.

Once all of the files are deleted, the directory and its subdirectories can be deleted as follow:

find ./ my_testrun1 -depth -type d -empty -delete

Neser Last Day of Operation

Please note this system will be decommissioned on 30th November 2016.

After this date all data in /project and /home will be deleted. Please ensure that you have transferred any data you wish to retain.

Follow us on Twitter

Follow all the latest news on HPC within the Supercomputing Lab and at KAUST, on Twitter @KAUST_HPC.

Previous Announcements

http://www.hpc.kaust.edu.sa/announcements/

Previous Tips

http://www.hpc.kaust.edu.sa/tip/