KAUST Supercomputing Laboratory Newsletter 24th June

Shaheen users should keep in mind that /project space is usually NOT recommended for running jobs. You should be running your jobs on /scratch and, after post processing, you should copy ONLY necessary files to /project. Those of you who are used to running all jobs from /project, it is time to start running your jobs on /scratch. This is a standard practice on supercomputers. Please note this change will not drastically change the way you run your jobs. The idea is to have the directory of your input files on /project, and the job script should automatically move the necessary files to /scratch, run the job, delete temporary files, run analytics/postprocessing, and copy ONLY necessary files to /project. Here is a simple test case that we prepared for demonstartion. Just do the following:

 tar -xf /sw/xc40cle7/openfoam/work_lustre_scratch.tar

 cd work_lustre_scratch/

 cat job.sh

 sbatch job.sh 

Once you run this test case, you might want to take a look at the job script and edit it according to your specific test case. Do not hesitate to contact us if you need help. Reducing data on /project reduces the need for backing up data, which in turn helps all Shaheen users. 

The users should also remeber that any files under /scratch/<username> or /scratch/project/kxxx that have not been acessed in the last 60 days will be deleted automatically without warning and are unrecoverable. Please rememebr to back up any important data to your /project/kxxxx or /home/<username>. We are available for any follow up questions or clarifications (help@hpc.kaust.edu.sa).