Background
The HPC resources available through KSL are as follows:
‘Shaheen’ XC40 Supercomputer
Disk Storage
Lustre parallel file system with 17.6 PB usable, 23 PB raw. Over 500 GB/s bandwidth.
Tape Storage
Hierarchical storage with 200 TB disk cache and 20 PB of tape storage, using a Spectra Logic tape library and IBM tapes and drives.
Access to the HPC resources at KSL is strictly controlled to ensure that KAUST complies with the terms & conditions of the United States Government's (USG) Export License for the Shaheen Systems.
A failure by KAUST to comply with these terms & conditions could expose KAUST to punitive action by the USG up to and including refusal to allow KAUST to purchase future HPC systems from the major suppliers of HPC systems in the world, nearly all of whom are based in the United States.
Q:
Who is permitted access to the Shaheen Systems?
A:
For the purposes of export regulation compliance, the USG classifies countries into 4 groups:
No access to the KSL systems at KAUST is authorised to organisations, governments or nationals of Tier 4/Group E countries [Cuba, Iran, North Korea, Sudan, Syria.]. . – see https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=1&SID=a23d71ec9ad6388744228... for the latest list].
Q:
What nationalities are not permitted access to the Shaheen Systems?
A:
Most nationalities are permitted access to the Shaheen Systems, either before or after approval from the USG DoC, but nationals of Iran, Syria, Cuba, North Korea and Sudan are currently NOT permitted access under any circumstances.
Q:
What research work is permitted on the Shaheen Systems?
A:
Work in the following areas of research is permitted on the Shaheen Systems:
- solar energy,
- water desalination,
- clean combustion, catalysis,
- membranes,
- crops,
- material science (soft, composite & nano materials),
- Red Sea ecology, geosciences (including seismic, reservoir & groundwater modelling),
- computational biology,
- computational fluid dynamics,
- aeroacoustics,
- particle transport,
- climate,
- computational chemistry,
- multi-physics simulations (involving two or more of the areas mentioned here),
- computational science & visualization (including parallel algorithms, programming models, large-scale data analysis, computational steering, mathematical modelling, and system software),
- microbiology,
- bioscience,
- optoelectronics, and
- nanoelectronics.
Q:
What research work is NOT permitted on the Shaheen Systems?
A:
Work in the following areas of research is NOT permitted on the Shaheen Systems:
a. National security and/or intelligence work.
b. The design, development, production, testing, or use of conventional, nuclear, chemical, biological, or radiological weapons, or any component or subsystem specially designed for such devices.
c. Complete rocket systems (including ballistic missiles, space launch vehicles, and sounding rockets) or unmanned air vehicle systems (including cruise missile systems, target drones, reconnaissance drones, and pilot optional aircraft) capable of delivering conventional, nuclear, chemical, biological or radiological weapons, including any specially designed component or subsystem of such systems.
d. The design, development, production, use or maintenance of:
i. A nuclear fuel cycle facility (including facilities engaged in nuclear propulsion and related activities) or heavy water production plant in a country not party to the Nuclear Non-Proliferation Treaty.
ii. Any facility for the production of chemical, biological or radiological weapons.
iii. Any facility for the manufacture of chemicals, biological agents or radioisotopes capable of being used in chemical, biological or radiological weapons.
Q:
Where can I find details of the Terms & Conditions relating to usage of the Shaheen Systems?
A:
The latest version of ‘KAUST ‘Shaheen’ HPC Systems - Terms & Conditions of Usage’ can be found here.
Q:
How do I apply for access to the Shaheen Systems?
A:
There are 3 steps in applying for access to the Shaheen Systems as described below (Step 2 & Step 3 can be done in parallel):
1. Step 1 – Organisational Access Application (OAA)
An OAA must be submitted by your institution to confirm that your institution understands and will comply with the export regulation compliance rules applying to the Shaheen Systems, and also lists those personnel from the institution who are authorised to act as Principal Investigators (PIs) and approve new users on behalf of your institution. The latest version of the OAA form can be found here.
When an OAA is received, KSL Management will check that is fully and correctly completed, and then review the OAA to identify if the institution is allowed access to the Shaheen Systems under the terms & conditions of the Shaheen export license. If the institution is allowed access, the applicant will be notified, advised (if applicable) of any limitations on access, e.g. only users from Tier 1 countries will be permitted access prior to specific approval from the USG DoC, and then asked to submit Individual Access Applications (IAAs) and Project Proposals (PPs) as necessary (see Steps 2 & 3 below).
If the institution is not allowed access, the applicant will be notified and asked if they wish KAUST to request approval from the USG DoC. Please note that obtaining approval for a new institution from the USG DoC can take several months.
2. Step 2 – Individual Access Application (IAA)
Users are required to submit Individual Access Applications (IAAs) together with copies of their passport (or other national ID) and their institution ID card. The IAA must be signed by a PI listed in the institutions’ OAA, and by the User. The latest version of the IAA form can be found here.
The purpose of the IAA is to:
- Communicate the identity, nationality, contact details, etc., of the User.
- Communicate on which projects the User will work and confirm that the PIapproves of the User working on his/her project.
- Confirm that the User has read, and will comply with, the Terms & Conditions for Usage for the Shaheen Systems (available here)
- Communicate the locations (IP addresses) from which the User will access the Shaheen Systems.
When an IAA is received, KSL Management will check that is fully and correctly completed, and then review the IAA to determine if the User is allowed access to the Shaheen Systems under the terms & conditions of the export license.
If access is approved for the User, and the User’s project has already been approved, KSL will set up a User account, usually within 2-3 days. Note that User accounts cannot be set up unless there is an active project to which they can be assigned; likewise, projects cannot be activated unless one or more Users assigned to that project.
If access is not approved for the User, the User and PI will be notified.
3. Step 3 – Project Proposals (PPs)
PIs are required to submit Project Proposals (PPs) for ALL work that will be done on the Shaheen Systems. The latest version of the Project Proposal form, along with a completed sample, can be found here. The information required in the PP includes:
- Details of the PI, Investigators & Collaborators working on the project
- A detailed project description
- A statement of the expected scientific impact of the project
- Details of the codes & libraries being used in project
- The current status of codes, e.g. their level of scaling
- The resource requirements for the project, including consultancy & enablement support services (if required)
- Any confidentiality and/or legal issues that may exist with the project
The purpose of the PP is to:
- Ensure that the work proposed does not violate the terms & conditions of the Shaheen export license.
- Provide the information necessary to ensure that the computational resources requested in the PP are suitable for the work proposed (and vice versa).
- Provide information for a decision on the amount of computational resources that will be assigned to the project.
Q:
How much time can I get on the Shaheen Systems?
A:
There are two types of projects on the Shaheen Systems:
- Pre-Production Projects
A Pre-Production Project usually lasts for up to 6 months, and provides access for system familiarisation, code porting, performance assessment, and other preproduction work, e.g. scalable algorithm development, code optimizationand development, etc.
A Development Project usually only requires a Computational Readiness Review (CRR) prior to being set up.
- Production Projects
A Production Project requires that applications have been ported and tuned, and that performance assessments have been completed.
Production Projects can be allocated significant computational resources, e.g. over 10,000,000 core-hours or more. Production Projects must undergo both a CRR and a Scientific Review (SR), andthe work proposed should be in alignment with KAUST’s Mission (see http://www.kaust.edu.sa/about/vision_mission.html#mission )
Q:
How will my Project Proposal be assessed?
A:
Project Proposals (PPs) can undergo three stages of review:
Computational Readiness Review (CRR)
CRRs are required for both Development and Production Projects, and are carried out by KSLs’ Computational Scientists, usually within 2-3 days of a PP being received. CRRs check the following:
- The amount of computational resources requested.
- The readiness of the codes for the work proposed.
- The appropriateness of the codes for the computational resources
- requested.
- The scaling capabilities of the codes.
- The project timetable.
As part of the CRR, the PP will be graded on a scale of 1 to 5 as follows:
- Once the CRR is completed, KSL Management will decide if the project is a Development Project or a Production Project.
- A Development Project will be set up immediately.
- A Production Project will be sent for Scientific Review (SR) before a decision is made on the allocation of computational resources to that project by the KSL
- Resource Allocation Committee (KSL RAC).
Scientific Review (SR)
SRs are carried out by KAUST Faculty & Researchers, usually members of KSLs’ Resource Allocation Committee (RAC), and are required for all Production Project Proposals.
During a SR, a minimum of three reviewers are asked to assess the following:
- The scientific and/or technical merit of the project in the context of KAUST’s Mission (see http://www.kaust.edu.sa/about/vision_mission.html#mission ).
- The likelihood that the project will make high-impact scientific advances in the field of study through the use of KSLs’ resources.
- The scientific and/or technical merit and originality of the research proposed
- The likelihood that the conceptual framework & methods used will lead to scientifically valid conclusions
- The degree to which the proposed research makes use of the best available mathematical algorithms, computer science methods, and state-of-the-art data management and visualization techniques.
- The adequacy of the requested computational resources for the research proposed.
- The scientific reputation & quality of recent research by the PI and other key personnel in the project team.
- The project's ability to make effective use of the computational resources requested.
- The reviewer’s own ability to evaluate the project proposal.
As a result of the SR, the PP will graded 1 to 3 (1 = Rejected, 2 = Accepted with minor changes, 3 = Accepted).
KSL Resource Allocation Committee (KSL RAC)
Once the CRR and SR are completed, all Production PPs are discussed and reviewed by the KSL Resource Allocation Committee (KSL RAC) who will make a decision on the computational resources to be allocated to the project. The KSL RAC oversees the allocation of computational resources of KSL on behalf of the KSL Management Committee (KSLMC) and KAUST. The KSL RAC comprises technical staff from KSL and various key stakeholders in KAUST, including the Research and Academic Divisions. The specific functions of KSL RAC include the following:
- Scientific and technical review of project proposals, to the greatest degree possible within the limitations of available resources, assignment of the available computational and/or consulting
- Services necessary to meet the needs of approved projects, scientific and technical review of project proposals for KSL-directed research activities
- Scientific and technical review of project progress and resource utilization versus approved plan for each project, and advising KSLMC on all purchases of new HPC systems.
The results of all KSL RAC reviews are reported to KSLMC, and form the basis upon which KSLMC assesses the effectiveness of KSL policies and procedures in meeting the competing demands of all users.
Q:
What reports are required on the progress of my Project?
A:
PIs are required to submit quarterly reports and final reports on their project.
The Project Quarterly Progress Report covers the following:
- Resources Allocated & Utilised
- Summary of Project Objectives
- Summary of Problems Encountered (if any)
- Summary of Results over Period
- List of Publications from the Project (with complete references)
- Summary of Plans for Continuation of Project
The Project Final Report covers the following:
- Resources Allocated & Utilised
- Summary of Project Objectives
- Summary of Problems Encountered (if any)
- Summary of Results & Conclusions
- List of Publications from the Project (with complete references)
Q:
How do I get help with obtaining access to the Shaheen Systems?
A:
All questions relating to obtaining access to the Shaheen systems should be emailed to help@hpc.kaust.edu.sa.