Submitting a Job

Assuming you are logged on to Artemis with the training credentials lets run compute jobs on Artemis HPC. Please use your own unikey credentials after todays training.

As mentioned, pbs scripts communicate your requirements to the scheduler. These scripts tend to follow the basic pattern:

  • a, Write pbs declarations to communicate with the scheduler.

  • b, Load modules required for your scripts (including environments)

  • c, Run your scripts and manage directory paths and/or shell variables

The below series of commands will create a folder in a specific location with the /project folder, extract data and rename a file which we will use to run our scripts. As we have logged in with the training credentials, we should have access to the /project/Training folder and the compressed file housed within it.

On the Artemis Terminal type the following commands:

cd /project/Training

mkdir <yourname>

cd <yourname>

git clone https://github.com/Sydney-Informatics-Hub/datahpc.git

cd datahpc

cp index.pbs basic.pbs

Not using these notes with the Training project?

Use wget https://github.com/Sydney-Informatics-Hub/IntroHPCData/archive/refs/tags/20240206.zip in place of the tar command to download all the scripts and data to use with these notes.

Creating and running pbs files are how you communicate the the HPC Scheduler which managed your script and the resources of the HPC. Refer to the HPC Context section for more info on what the scheduler does and implications in its use.

Open up the nano text editor and adjust the basic.pbs with:

nano basic.pbs

We will alter its contents to run a simple hello world script. Specify the Training project and defaultQ queue in the PBS declarations. Delete the existing content below the PBS declarations of basic and add the below content to create a hello world to test Artemis

basic.pbs should contain*:

#!/bin/bash
#PBS -P Training
#PBS -N test-hello
#PBS -l select=1:ncpus=1:mem=4gb
#PBS -l walltime=00:10:00
#PBS -q defaultQ

cd /project/Training/<yourname>/datahpc/
mkdir New_job
cp basic.pbs New_job/copy.pbs
perl hello.pl "YourName"

Submit the basic.pbs script to the scheduler with

qsub basic.pbs

How do we know if our script ran successfully? Investigate the output and error files that were automatically generated whose name refers to the #PBS -N flag and includes the job number printed when qsub was engaged.