Running Jobs under MOAB / Torque


Prepare a job script

Within MOAB/Torque, it is not possible to submit a binary executable directly. It is also more convenient to use a job script, since it allows to embed options, i.e. you can define the resources your job requests/needs to run under the control of MOAB/Torque.

Please note that when submitting a job it should not be expected to start immediately. This is due to the scheduler running with certain intervals, trying
to “optimize” resource usage. You can help the scheduler to make better decisions by specifying details in the job script. The most important one being walltime.
If walltime is not specified, the scheduler has to assume the max. walltime for the
particular queue in question, and uses that as a basis for its decisions.

Example script:

#!/bin/sh

# embedded options to qsub - start with #PBS
# -- our name ---
#PBS -N Sleeper
# -- estimated wall clock time (execution time): hh:mm:ss --
#PBS -l walltime=00:01:30
# -- run in the current working (submission) directory --
if test X$PBS_ENVIRONMENT = XPBS_BATCH; then cd $PBS_O_WORKDIR; fi

# here follow the commands you want to execute
#
/bin/echo Here I am. Sleeping now at: `date`
time=60
sleep $time
echo Now it is: `date`

As you can see above , it is possible to add qsub options to your script file as special comment lines, starting with ‘#PBS’. These lines are not read by any other shell (they start with the comment meta character ‘#’), i.e. in principle you can use your script outside MOAB/Torque, too.

Please note: All the lines with ‘#PBS’ have to come first in the script file, before the first command! Otherwise they will be ignored!

Explanation of the options used above:

-N job_name

With this option you can specify the name of the job, as it should be used by MOAB/Torque. Without this option, the name of your job is the filename of the job script submitted.

-l walltime=[DD:hh:mm:ss]

Specifies the wall clock time in which your job will finish after it gets started. If you have an estimate for your jobs (from experience), specify it! This helps the scheduler to make better use of the resources. If you don’t specify a walltime, the default will apply, e.g. 72 hours for the hpc queue.

if test X$PBS_ENVIRONMENT = XPBS_BATCH; then cd $PBS_O_WORKDIR; fi

While running a batch job (when the PBS_ENVIRONMENT environment variable equals PBS_BATCH) enter the working directory current at the time of the job submission (cd $PBS_O_WORKDIR), i.e. the directory from where you have used qsub to submit the job to MOAB/Torque. The PBS_O_WORKDIR is a special environment variable set by Torque. If your job relies on a specific working-directory, e.g. where it can find input files, it might be better to add a cd working-directory line to the job script. Please note: MOAB/Torque does NOT support spaces in the name of the directories!

If you do NOT use this option, the job’s output files (standard output/error) will always be put in your home directory!

Other useful options:

-o filename

Specify the filename where MOAB/Torque should save the messages from the standard output channel of your job. Without the -o option, the name of your job (usually the name of the job script, if not specified with -N) plus the extension .o## is used, where ‘##’ is the id of the job as assigned by MOAB/Torque.

-e filename

Specify the filename where MOAB/Torque should save the messages from the standard error channel of your job. Without the -e option, the name of your job (usually the name of the job script, if not specified with -N) plus the extension .e## is used, where ‘##’ is the number of the job as assigned by MOAB/Torque.

For a more detailed list of options see man qsub.