GPU Nodes

The IBEX cluster contains different architectures of GPUs like Turing, Pascal and Volta^#. These different GPUs are accessed (for your source code compilation and job submission) using the following login nodes:

1. V100 and RTX 2080 Ti GPU nodes: vlogin.ibex.kaust.edu.sa

2. All other GPU nodes: glogin.ibex.kaust.edu.sa

The IBEX cluster has 32 GPU compute nodes (152 GPU cards)  and it’s summarized in Table 1. These various GPUs are accessed by the SLURM scheduling using the constraints "--gres=gpu:<$$$>:<#>”, where: <$$$> is the GPU name and <#> is for number of GPUs.

For example, “--gres=gpu:gtx1080ti:4” is for 4 GTX GPUs.

                                        Table 1. List of GPU architectures in IBEX Cluster

Note: The allocation of CPU memory can be done with `--mem=###G` constraint in SLURM job scripts. The amount of memory depends on the job characteristization. A good starting place would be at least as much as the GPU memory they will use. For example: 2 x v100 GPUs would allocate at least `--mem=64G` for the CPUs.

# Newly added into IBEX cluster.

$ The vlogin node has a single v100 GPU for compilation of the source code. We have  4 x v100 GPU cards are available in compute nodes and accessed through SLURM scheduling.

^ The usable node memory represents the available memory for job execution.

The CUDA libraries are different for the GPU nodes. Table 2 provides the guidelines and recommendations for the CUDA libraries for the specific GPU architecture for the GPU source code compilations.

                                          Table 2. CUDA libraries for GPU architecture

Some of the standard compilers, numerical libraries and GPU supported applications are available as a software modules. Table 3 provides the general guidelines of list of software modules available for specific GPU architecture.

                                        Table 3. Software modules for different GPU architecture

Alternatively, send an email:

1. Application installation/failure/support: 
   • cluster-apps@hpc.kaust.edu.sa
2. System issues/failure/support:
   • ​​cluster-systems@hpc.kaust.edu.sa