Transferring large files with GLOBUS

The supported methods for file transfer on Shaheen are SCP, SFTP, and Globus. 

SCP and SFTP are available for local area transfers of small files; they are not recommended for use with large data transfers due to poor performance and excess resource utilization on the login nodes. A reminder that you need to initiate the transfer from your machine as the KAUST firewall will not permit outbound port 22 connections.

Globus can be used for large files, even Terabytes of data. It's easy to set up a Globus endpoint on a user's machine and upload or download files between Shaheen and any other endpoint, like your laptop or another HPC systems. Globus makes it possible to easily transfer files to and from any machine – even if behind a firewall or NAT without any administrative privileges – using Globus Connect Personal.  

Please follow instruction to download and install it: https://www.globus.org/globus-connect-personal 

Once setup and installation on your machine is completed, login to your Globus account

• If you are not located in KAUST campus, search for 'Shaheen dm1' to get our data mover server 'dm1.hpc.kaust.edu.sa'.
• If you are connected on KAUST network, search for 'Shaheen dm2' to get our data mover server 'dm2.hpc.kaust.edu.sa'.  

Once you are connected to the GLOBUS GUI browser, follow the steps shared in this tutorial (Starting from page 36)