MDCS configuration for MATLAB on the LSF clusters (pre-R2019a)

Note: this is only valid for MATLAB R2017b, R2018a and R2018b!!!

MATLAB can be used in parallel across different nodes thanks to its MATLAB Distributed Computing Server (MDCS) product.
To be able to use it on the DCC HPC LSF cluster, MATLAB needs to be configured correctly, so that it becomes aware of the underlying cluster management software.

This procedure needs to be done only once (for each different matlab version).

Login to the LSF cluster (instructions here)
Open an interactive session with the command linuxsh -X .
Start a MATLAB session typing matlab (or the command for a specific MATLAB version if you do not want to run the default one)

In MATLAB, start the configuration of the cluster with the command configCluster. You will get some messages:

>> configCluster

                >> % Must set MemUsage and WallTime before submitting jobs to DCC.  E.g.
                >> c = parcluster('dcc');
                >> % 2 GB/core
                >> c.AdditionalProperties.MemUsage = '2GB';
                >> % 5 hours
                >> c.AdditionalProperties.WallTime = '05:00';
                >> c.saveProfile

You have now configured a cluster called dcc, with some default settings (e.g maximum number of workers=32), but you need to set some other general Additional Properties. Among others, your email address, the queue name, the walltime, the memory usage per worker/core. For example:
```
>> % Get a handle to the cluster 
>> c = parcluster('dcc');
>> % Specify e-mail address to receive notifications about your job 
>> % c.AdditionalProperties.EmailAddress = 'user-id@dtu.dk';
>> % Request 2 GB/core
>> c.AdditionalProperties.MemUsage = '2GB';
>> % Specify 8 procs per node (i.e. 8 Workers) 
>> c.AdditionalProperties.ProcsPerNode = 8;
>> % Specify the walltime (e.g. 5 hours)
>> c.AdditionalProperties.WallTime = '05:00'; 
>> c.saveProfile
```
The first command c = parcluster('dcc') creates a cluster object for the session.
The following commands set some additional properties for that cluster.
The last line, c.saveProfile saves the settings for the cluster, so they persist across different sessions. You can always check the settings with the command c.AdditionalProperties.
To clear a value, assign the property an empty value (”, [], or false).
```
>> % Turn off email notifications
>> c.AdditionalProperties.EmailAddress = '';
```
Those values becomes the default values for the cluster. Those can be overwritten simply reassigning them in a session, without saving them.

Now you are ready to use the DCC HPC LSF clusters with the following standard settings:

Name: dcc
Used queue: hpc
Number of workers: up to 32

Please adapt the profile, if you have different needs, e.g. you want/have to use a different queue, or want to increase the number of workers.

Note: DTU has only a shared license for 256 workers, so please be sensible in your usage of the MDCS. Up to 20 workers, the ‘local’ profile can be used as well, and does not use any of the MDCS licenses!

How does it work?

In your script, you can select the cluster with a command like

>> clust=parcluster('dcc')

Then in the script you can create a pool of worker, specifying to use that specific cluster, and the number of workers (in the example 30)

>> p=parpool(clust,30)

With this operation, MATLAB automatically submits a job to the cluster for you using the Additional Properties defined in the profile or in previous lines in the script, and keeps track of the workers.
From this point on everything works as if the workers were running locally on a single machine.
Remember that if you kill the MATLAB session, also the workers are killed automatically.

If you want to use the MDCS in a job script, please refer to this page.