HPC Carpentry part 2 - High Performance Computing: Cheatsheets for Queuing System Quick Reference

HPC systems typically provides login nodes and a set of worker nodes.

The resources found on independent (worker) nodes can vary in volume and type (amount of RAM, processor architecture, availability of network mounted file systems, etc.).

Files saved on one node are available on all nodes.

The scheduler handles how compute resources are shared between users.

Everything you do should be run through the scheduler.

A job is just a shell script.

If in doubt, request more resources than you will need.

Load software with module load softwareName

Unload software with module purge

The module system handles software versioning and package conflicts for you automatically.

wget downloads a file from the internet.

scp transfer files to and from your computer.

You can use an SFTP client like FileZilla to transfer files through a GUI.

The smaller your job, the faster it will schedule.

For a constant proble size, most parallel codes will eventually stop getting faster with more processes.

If you have a serial code, it will still only run on one core even if you ask for multiple cores in your submission script.

Job arrays allow you to launch many jobs that are each assigned a different index value, using just one submission script.

Clusters are designed to handle large number of users

The shared resources to be careful with are memory and cpus on the login nodes, and I/O

Your data on the system is your responsibility.

Plan and test large data transfers.

It is often best to convert many files to a single archive file before transferring.

Don’t run stuff on the login node.

Really don’t run stuff on the login node.

When in doubt, talk to the cluster support team first