Job Arrays

SLURM allows you to submit and manage multiple similar jobs quickly and easily thanks to job arrays. Job arrays can be specified using two techniques:

  • in a batch directive, 
    • #SBATCH --array=1-10
  • in the command line as 
    • cdl:~>sbatch --array=0-10  my_slurm_script.sh

This will generate a job array containing 10 jobs. If the sbatch command responds "Submitted batch job 100 " then the environment variables will be set as follows:

SLURM_JOBID=100
SLURM_ARRAY_JOB_ID=100
SLURM_ARRAY_TASK_ID=1

SLURM_JOBID=101
SLURM_ARRAY_JOB_ID=100
SLURM_ARRAY_TASK_ID=2
….

It is advised to update the job's stdin, stdout, and stderr filenames, as follows:

#SBATCH –outputs= slurm-%A_%a.out

where %A will be replaced by the value of SLURM_ARRAY_JOB_ID  and %a will be replaced by the value of SLURM_ARRAY_TASK_ID

To visualize the status , you can use the squeue  -u username command, however the jobs pending will appear in one line. For a better formatting and to check the status of both running and pending jobs, you can add the option –r

100_1  user1   kx        jarray   R None      2016-01-11T00:16:36   16:19:53    7:40:07     8
100_2  user1   kx        jarray   R None      2016-01-11T00:16:36   16:19:53    7:40:07     8
100_3  user1   kx        jarray   R None      2016-01-11T00:16:36   16:19:53    7:40:07     8
100_4  user1   kx        jarray   R None      2016-01-11T00:16:36   16:19:53    7:40:07     8
100_5  user1   kx        jarray   R None      2016-01-11T00:16:36   16:19:53    7:40:07     8
100_6  user1   kx        jarray  PD JobArrayT N/A                       0:00      10:00     8
100_7  user1   kx        jarray  PD JobArrayT N/A                       0:00      10:00     8
100_8  user1   kx        jarray  PD JobArrayT N/A                       0:00      10:00     8
100_9  user1   kx        jarray  PD JobArrayT N/A                       0:00      10:00     8
100_10 user1   kx        jarray  PD JobArrayT N/A                       0:00      10:00     8

To limit the number of simultaneously running tasks jobs to 2 for example, use the %2 as follows, "--array=0-10%2"

Tags: