ABAQUS jobs under LSF

General Info

ABAQUS, is a suite of programs for finite element analysis, with a series of CAD utilities.

It is quite used in different engineering fields. It is installed on the cluster, where it can be run taking advantage of the multi-core machines and the larger amount of memory available.

ABAQUS is not available by default for everyone. Please write an email to support@hpc.dtu.dk so that we can add you to the ABAQUS-user-list, create a scratch-directory for you and also please tell us who is responsible for you, so that we can figure things out, when there are some license-issues.

Model Preparation/Short simulations

As most of these programs, it is mainly meant to be used through a Graphical User Interface. To open the GUI, open a terminal and type the command

/appl/ABAQUS/Commands/abq2020 cae -mesa

Where cae stands for “Complete Abaqus Environment”. In this way the program runs interactively, as on a local computer. The interface can be used for preparing the model to be simulated, to run the simulation, and for post-processing of the simulation data.

However, on the cluster this is not the way to use the program. Unless the simulation is very light (i.e. it does not require a lot of computational time, processors, and memory), the user should only use the cae to prepare the model, and to post-process the simulation results. The real simulation should instead be performed as a batch job, without GUI.

So an ABAQUS analysis should be split in three phases:

  • Preparation of the simulation: ABAQUS CAE (or other tools);
  • Simulation: ABAQUS batch job;
  • Post-processing: ABAQUS CAE (or other tools).

ABAQUS job scripts

A basic job script for running ABAQUS on the cluster is shown here:

### General options 
### -- set the job Name -- 
#BSUB -J ABAQUS_Parallel 
### -- ask for number of cores (default: 1) -- 
#BSUB -n 8 
### -- specify that the cores are on a single host -- 
#BSUB -R "span[hosts=1]" 
### -- set walltime limit: hh:mm -- 
#BSUB -W 48:00 
### -- specify that we need 2GB of memory per core/slot -- 
#BSUB -R "rusage[mem=2GB]"
### -- set the email address -- 
# please uncomment the following line and put in your e-mail address, 
# if you want to receive e-mail notifications on a non-default address 
##BSUB -u your_email_address 
### -- send notification at start -- 
### -- send notification at completion-- 
### -- Specify the output and error file. %J is the job-id -- 
### -- -o and -e mean append, -oo and -eo mean overwrite -- 
#BSUB -o Output_%J.out 
#BSUB -e Error_%J.err 

# -- program invocation here -- 

$ABAQ job=my_sample_job input=model.inp cpus=$LSB_DJOB_NUMPROC memory="16 gb" interactive

If there are licenses available, then it will start. For the description of all the BSUB options, and their meaning please refer to this page. Specifically, in this script 8 cores on a single node are reserved.

#BSUB -n 8
#BSUB -R "span[hosts=1]"

for 48 hours
#BSUB -W 48:00.
Then we assign for convenience the path for the ABAQUS executable to a variable
and finally we call the program, with the ABAQUS job file as argument:
$ABAQ job=my_sample_job input=model.inp cpus=$LSB_DJOB_NUMPROC memory="16 gb" interactive.
In this command line there are 4 options:

  • job=my_sample_job: this passes the user job-file as argument to ABAQUS;
  • input=model.inp: this passes the file with the model input file as argument to ABAQUS;
  • cpus=$LSB_DJOB_NUMPROC: this tells ABAQUS to run in parallel, using the number of cores requested. LSB_DJOB_NUMPROC is a variable automatically set to the number of cores that your script reserves.
  • memory="16 gb": this option tells abaqus the amount of RAM available for the job. This must correspond to the the TOTAL amount of memory requested in the job, in this example 2gb*8=16gb. For an explanation, look at the MEMORY note below.
  • interactive: this option prevents ABAQUS from getting stuck waiting for user input.
  • ABAQUS can run in parallel, but this does not mean that it can really speed-up your run. Do not ask for too many cores, if you are not sure that the program can really use them. Make some tests comparing the execution time on short parts of your model, changing the number of cores, and check the behaviour. There is a limited amount of licenses, also.
  • Please make sure that you are starting your bigger simulations on one of our scratch-filesystems. You can ask for a scratch-directory via support@hpc.dtu.dk
  • MEMORY: ABAQUS assigns a percentage (on linux 80%) of the memory available on the machine to be used as RAM and swap space, to speed up calculation. But this percentage is applied to the physical memory of the machine, not on the memory requested in the jobscript. We recommend to set this value to be the actual memory requested in the jobscript, with the command line option memory="16 gb" . This value must correspond to the total memory requested for the job, remember that the amount of memory specified with the rusage[mem=2GB] option is per core.

Passing a script

If you want to pass a python script with the instructions to execute to ABAQUS, the syntax is different. Assuming that your script name is myScript.py, the right command line is:

$ABAQ cae noGUI=myScript

Notice that you do not have to specify the “.py” extension.