KAUST Supercomputing Laboratory Newsletter 23rd July 2020

In this newsletter:

  • System Maintenance
  • EID Holiday
  • RCAC meeting
  • KAUST supercomputer Shaheen II joins the fight against COVID-19
  • Tip of the Week: Using Burst Buffer for Complex Workflows
  • Follow us on Twitter
  • Previous Announcements
  • Previous Tips

 

System Maintenance

We would like to announce that our next maintenance session on Shaheen will be on Tuesday the 11th of August 2020. We will send another reminder and more details closer to the date. Please get in touch with us at help@hpc.kaust.edu.sa, should this cause any concerns.

EID Holiday

KAUST will be on holiday from 30th July until 8th August (inclusive) due to the EID Al-Adha Holiday. Please do not expect to receive replies to support requests during this period.

RCAC meeting

The project submission deadline for the next RCAC meeting is 31st July 2020. Please note that the RCAC meetings are held once per month. Projects received on or before the submission deadline will be included in the agenda for the subsequent RCAC meeting. The detailed procedures, updated templates and forms are available here: https://www.hpc.kaust.edu.sa/account-applications

 

KAUST supercomputer Shaheen II joins the fight against COVID-19

King Abdullah University of Science and Technology (KAUST) invites researchers from across the Kingdom to submit proposals for COVID-19-related research. Recognizing the urgency to address global challenges related to the COVID-19 pandemic through scientific discovery and innovation, the University’s Supercomputing Core Laboratory (KSL) is making computing resources—including the flagship Shaheen II supercomputer and its expert scientists—available to support research projects.

Topics may include but are not limited to: understanding the virus on a molecular level; understanding its fluid-dynamical transport; evaluating the repurposing of existing drugs; forecasting how the disease spreads; and finding ways to stop or slow down the pandemic.

Accepted proposals can access the following resources: (1) Shaheen II, a Cray XC-40 supercomputer based on Intel Haswell processors with nearly 200,000 compute cores tightly connected with Aries high-speed interconnect; (2) Ibex cluster, a high throughput computer system with about 500 computing nodes using Intel Skylake and Cascade Lake CPUs and Nvidia V100 GPUs; and (3) KSL staff scientists, who will provide support, training and consultancy to maximize impact. Through 30 June 2020, up to 15% of these resources will be reserved for fast-tracking competitive COVID-19 proposals through the KAUST Research Computing Allocation Committee.  Thereafter, such proposals remain welcome and will be considered in the standard process.

Applicants can apply for computing allocations using the COVID-19 Project Proposal form. Please submit the form to projects@hpc.kaust.edu.sa. Submitted proposals will be fast-tracked for processing.

Please contact help@hpc.kaust.edu.sa with any inquiries.

 

Tip of the week: Using Burst Buffer for Complex Workflows

Typical PDEs based simulation workflows entail the following steps: geometric modelling, meshing, setting up of boundary and initial conditions, solver, possible format change of output files, and post processing i.e. visualization and/or data analysis. Most of the workflows start with a very small data set and the final data that is needed to make scientific discovery is also very small (could be just a picture or a movie). But these workflows do invlove large intermediatery data that is not needed for the final scientific discovery.

Shaheen's DataWARP (burst buffer) is a good candidate for running such workflows. The main motivations are to reduce the data footprint and minimize metadata traffic on the lustre file system.

A test case can be viewed in /scratch/tmp/openfoam_bb_testcase.tar. You can find a ReadMe file in the folder that provides information about editing the  jobscript and running the test case.

This is a simple 100x100x100 cells laminar flow 3D cavity test case which is designed to demonstrate the value of DataWARP in optimizing the flow of data for complex workflows. The intermediate data only stays temporarily in Burst Buffer memory and the final processed data gets migrated to the lustre file system. This workflow reduces the data footprint and also minimize metadata traffic on the lustre file system.

The ideas from this CFD-based test can be applied to simulations in other domain sciences that involve workflows and temporary intermediate data.

In the next training session (or next month's Tip of the Week), we will demonstrate how to integrate data visualisation in this workflow. This type of workflow design is ideally suited for generating simulation based data for deep learning training.

Follow us on Twitter

Follow all the latest news on HPC within the Supercomputing Lab and at KAUST, on Twitter @KAUST_HPC.

Previous Announcements

http://www.hpc.kaust.edu.sa/announcements/

Previous Tips

http://www.hpc.kaust.edu.sa/tip/