Slurm job scheduler and MATLAB in terminal¶

The batch system used at UPPMAX, HPC2N, LUNARC, NSC, and PDC is called SLURM. The same is the case at most of the Swedish HPC centres.

SLURM is an Open Source job scheduler, which provides three key functions: - Keeps track of available system resources - Enforces local system resource usage and job scheduling policies - Manages a job queue, distributing work across resources according to policies

Users ask for compute resources via the sbatch command (as those who attended yesterday’s R session will already know). - To run a batch job, you must create and submit a SLURM submit file (also called a batch submit file, a batch script, or a job script). - Guides and documentation at: https://docs.hpc2n.umu.se/documentation/batchsystem/intro/ and https://docs.uppmax.uu.se/cluster_guides/slurm/ and https://lunarc-documentation.readthedocs.io/en/latest/manual/submitting_jobs/manual_basic_job/ and https://www.nsc.liu.se/support/batch-jobs/introduction/ and https://support.pdc.kth.se/doc/run_jobs/job_scheduling/

MATLAB is well integrated with SLURM, so there are several ways to run these jobs: - Using the job scheduler (batch command) in MATLAB Desktop/graphical interface (This is the Recommended Use). - Starting a parpool with a predefined cluster (This allows for more interactivity). - Writing a batch script as for any other software and submitting the job with the sbatch command from SLURM (This could be useful if you want to run long jobs and you don’t need to modify the code in the meantime).

In the following sections we will extend these concepts.

Useful commands to the batch system¶

Before going into MATLAB specifics for batch jobs, we should look briefly at some useful commands:

• Submit job: sbatch <jobscript.sh>
• Get list of your jobs: squeue --me
• Check on a specific job: scontrol show job <job-id>
• Delete a specific job: scancel <job-id>
• Useful info about a job: sacct -l -j <job-id> | less -S
• Url to a page with info about the job (Kebnekaise only): job-usage <job-id>

First time configuration¶

In order to be able to submit jobs to the SLURM queue, you usually need to configure MATLAB the first time you load any particular version. This configuration step will provide a set of default specifications for batch and parallel jobs called a cluster profile.

Configuration only needs to be run once per version of MATLAB on each cluster. These specifications can be changed or added to at runtime, and it is possible to have more than one profile for a single release.

The instructions for each facility are linked below: - HPC2N: Configure MATLAB - UPPMAX: MATLAB configuration - LUNARC: Configuration at the command line - NSC: MATLAB (scroll down to “Before submitting your first job”). - PDC: no need to configure

For most clusters, configuration can be done at the regular terminal using a shell script called configCluster.sh. It is also possible to run configCluster (without “.sh”) on the MATLAB command line; for NSC, this is the only way to run the configuration program. The tabs below demonstrate the preferred method for each cluster where configuration is required the first time you use a given version of MATLAB.

configCluster.sh <project-id>

configCluster.sh

configCluster.sh <project-id>

[bbrydsoe@cosmos3 ~]$ configCluster.sh lu2024-7-68
salloc: Granted job allocation 927531
salloc: Waiting for resource configuration
salloc: Nodes cn011 are ready for job

                           < M A T L A B (R) >
                 Copyright 1984-2023 The MathWorks, Inc.
            R2023b Update 7 (23.2.0.2515942) 64-bit (glnxa64)
                             January 30, 2024

To get started, type doc.
For product information, visit www.mathworks.com.

ip =

   "10.21.0.11"

   [1] aurora
   [2] cosmos
2
Select a cluster [1-2]: >>Complete.  Default cluster profile set to "cosmos R2023b".

   Must set AccountName and WallTime before submitting jobs to COSMOS.  E.g.

   >> c = parcluster;
   >> c.AdditionalProperties.AccountName = 'account-name';
   >> % 5 hour walltime
   >> c.AdditionalProperties.WallTime = '05:00:00';
   >> c.saveProfile

MATLAB is configured for multi-node parallelism.

salloc: Relinquishing job allocation 927531
salloc: Job allocation 927531 has been revoked.
[bbrydsoe@cosmos3 ~]$

module load MATLAB/2024a-hpc1-bdist
matlab -nodisplay -nodesktop -nosplash -softwareopengl
>> configCluster

MATLAB terminal interface¶

Here we will discuss:

• Starting MATLAB on the command line
• Job settings
  • c.parcluster
  • c.AdditionalProperties.
  • c.batch
• Starting a job from within MATLAB

This section will show you how to use MATLAB completely from the shell/terminal without having to open the GUI. This can be useful if you only have a regular SSH connection or otherwise need to run something fast and lightweight instead of having to open the GUI. This is an extra advantage when you have a poor network connection.

Starting MATLAB¶

To start MATLAB on the command line, without running the GUI, load the MATLAB version and do

matlab -singleCompThread -nodisplay -nosplash -nodesktop

This starts MATLAB. Not all of the flags shown are needed at every center, but it does no harm to include them.

Working in MATLAB¶

Of course, we can work in MATLAB like this in exactly the same way as in the GUI, as the example below shows

$ matlab -singleCompThread -nodisplay -nosplash -nodesktop
Opening log file:  /home/b/bbrydsoe/java.log.43927

                                 < M A T L A B (R) >
                       Copyright 1984-2023 The MathWorks, Inc.
                  R2023a Update 4 (9.14.0.2306882) 64-bit (glnxa64)
                                    June 19, 2023


To get started, type doc.
For product information, visit www.mathworks.com.

>> a = [ 1 2 3 ; 4 5 6; 7 8 9];
>> b = [ 7 5 6 ; 2 0 8; 5 7 1];
>> c = a + b

c =

    8     7     9
    6     5    14
   12    15    10

>> d = a - b

d =

   -6    -3    -3
    2     5    -2
    2     1     8

>> e = c + d;
>> e

e =

    2     4     6
    8    10    12
   14    16    18

>>

Running MATLAB code with batch¶

Now going to look at running in batch on the compute nodes.

Job settings at the command line¶

If you want to run a MATLAB program on the cluster with batch, you have to set some things for the job. Start MATLAB and input the following:

>> c=parcluster('CLUSTER');
>> c.AdditionalProperties.AccountName = 'PROJECT-ID';
>> c.AdditionalProperties.WallTime = 'HHH1:MM:SS';
>> c.saveProfile

To list the content of your profile, enter c.AdditionalProperties at the prompt.

ml PDC/23.12 matlab/r2024a-ps
matlab -nodisplay -nodesktop -nosplash

Example, for HPC2N

Asking for 1 hour walltime.

>> c=parcluster('kebnekaise');
>> c.AdditionalProperties.AccountName = 'hpc2n2025-062';
>> c.AdditionalProperties.WallTime = '01:00:00';
>> c.saveProfile

Example, for PDC

Asking for 1 hour. Starting from personal computer.

bbrydsoe@enterprise:~$ ssh -X dardel.pdc.kth.se
Last login: Thu Mar 20 17:02:49 2025 from enterprise.hpc2n.umu.se

 2025-03-14 at 15:39 [dardel]

System maintenance done, Dardel is running jobs since a few hours.

 --== Welcome to Dardel! ==--
bbrydsoe@login1:~>

bbrydsoe@login1:~> salloc -c 24 -t 1:00:00 -A naiss2025-22-262 -p shared
salloc: Pending job allocation 9050479
salloc: job 9050479 queued and waiting for resources
salloc: job 9050479 has been allocated resources
salloc: Granted job allocation 9050479
salloc: Waiting for resource configuration
salloc: Nodes nid002585 are ready for job
bbrydsoe@login1:~> ssh nid002585
bbrydsoe@nid002585:~> ml PDC/23.12 matlab/r2024a-ps
bbrydsoe@nid002585:~> matlab -nodisplay -nodesktop -nosplash

                                    < M A T L A B (R) >
                          Copyright 1984-2024 The MathWorks, Inc.
                     R2024a Update 3 (24.1.0.2603908) 64-bit (glnxa64)
                                        May 2, 2024


To get started, type doc.
For product information, visit www.mathworks.com.

>> c=parcluster;
>>

Running a job from within the MATLAB terminal interface¶

When starting a simple MATLAB program inside MATLAB on the terminal, by default, it will use your cluster profile which you just created and saved above. If you have a script file called myScript.m, the code to run it as a batch job with the default settings is:

>> job = batch('myScript');

The batch command does not block MATLAB so you can continue working while computations take place. If you want to block MATLAB until the job finishes, use the wait function on the job object, like this:

wait(job);

By default, MATLAB saves the Command Window output from the batch job to the diary of the job. To retrieve it, use the diary function:

diary(job)

After the job finishes, fetch the results by using the load function:

load(job,'x');

or using the .fetchOutputs attribute:

job.fetchOutputs{:}

Serial¶

After starting MATLAB, you can get a handle to the cluster (remember, on Rackham and Dardel, just use c=parcluster;) like this:

>> c=parcluster('CLUSTER')

For example, given that

• myfcn is a command or serial MATLAB program.
• N is the number of output arguments from the evaluated function
• x1, x2, x3,… are the input arguments

then starting a batch job with these inputs would look like:

j = c.batch(@myfcn, N, {x1,x2,x3,...})

This job handle j then allows us to:

• Query the job status with j.State
• Fetch the result, if the job is finished, using j.fetchOutputs{:}, and
• Delete the job when it is not needed anymore using j.delete

If you are running a lot of jobs, or if you want to quit MATLAB and restart it at a later time, you can retrieve the list of jobs and fetch the outputs of a specific job by index. For example:

Get the list of jobs:

jobs = c.Jobs

Retrieve the output of the second job:

j2=jobs(2)
output = j2.fetchOutputs{:}

Parallel¶

Running parallel batch jobs are quite similar to running serial jobs. We just need to specify a MATLAB Pool to use and of course MATLAB code that is parallelized. This is easiest illustrated with an example.

To make a pool of workers, and to give input etc., the basic syntax is:

>> job = c.batch(@SCRIPT, #output, {input1, input2, input3, ...}, 'pool', #workers);

Example: Running a simple MATLAB script, parallel-example.m, giving the input “16”, creating 4 workers, expecting 1 output. The name of the job handle is up to you (here it is j).

>> j = c.batch(@parallel_example, 1, {16}, 'pool', 4);

Let us try running this on Kebnekaise, including checking state and then getting output:

>> j = c.batch(@parallel_example, 1, {16}, 'pool', 4);

additionalSubmitArgs =

  '--ntasks=5 --cpus-per-task=1 -A hpc2n2025-062 -t 01:00:00'

>> j.State

ans =

   'running'

>> j.State

ans =

   'finished'

>> j.fetchOutputs{:}

ans =

   9.3387

>>

parpool¶

On the clusters where this works, you can start a parpool and then (for instance) run a parallel code inside MATLAB.

Parpool Example, for PDC: As shown earlier, first start an interactive session, login to the compute node you got, then load matlab and start it. Then create a parpool of the size (at most) that you asked for in number of cores.

>> p=parpool(24)
Starting parallel pool (parpool) using the 'Processes' profile ... connected to 24 workers.
p =
Pool with properties:
Connected: true
NumWorkers: 24
Cluster: local
AttachedFiles: {}
IdleTimeout: 30 minute(s) (30 minutes remaining)
SpmdEnabled: true
>> parallel_example
ans =
8.9287

There is more information about batch jobs here on Mathworks.

Batch scripts containing MATLAB code¶

Here we will discuss creating a batch script to run MATLAB in - Serial - Parallel

While we can submit batch jobs (or even batch jobs of batch jobs) from inside MATLAB (and that may be the most common way of using the batch system with MATLAB), it is also possible to create a batch submit script and use that to run MATLAB.

The difference here is that when the batch script has been submitted, you cannot make changes to your job. It is not interactive. That is also an advantage - you can submit the job, log out, and then come back later and see the results.

Serial batch jobs¶

Here is an example of a serial batch job formatted for each of the 5 facilities covered.

#!/bin/bash
# Change to your actual project number later
#SBATCH -A uppmax2025-2-272
# Asking for 1 core
#SBATCH -n 1
# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module add matlab/R2023b

# Executing the matlab program monte_carlo_pi.m for the value n=100000
# (n is number of steps - see program).
# The command 'time' is timing the execution
time matlab -nojvm -nodisplay -r "monte_carlo_pi(100000)"

#!/bin/bash
# Change to your actual project number later
#SBATCH -A hpc2n2025-062
# Asking for 1 core
#SBATCH -n 1
# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module add MATLAB/2023a.Update4

# Executing the matlab program monte_carlo_pi.m for the value n=100000
# (n is number of steps - see program).
# The command 'time' is timing the execution
time matlab -nojvm -nodisplay -r "monte_carlo_pi(100000)"

#!/bin/bash
# Change to your actual project number later
#SBATCH -A lu2025-7-24
# Asking for 1 core
#SBATCH -n 1
# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module add matlab/2023b

# Executing the matlab program monte_carlo_pi.m for the value n=100000
# (n is number of steps - see program).
# The command 'time' is timing the execution
time matlab -nojvm -nodisplay -r "monte_carlo_pi(100000)"

#!/bin/bash
# Change to your actual project number later
#SBATCH -A naiss2025-22-262
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --ntasks-per-core=1
# Asking for 15 min (change as you want)
#SBATCH -t 00:15:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1
module load MATLAB/2024a-hpc1-bdist

# Executing the matlab program monte_carlo_pi.m for the value n=100000
# (n is number of steps - see program).
# The command 'time' is timing the execution
time matlab -singleCompThread -nojvm -nodisplay -r "monte_carlo_pi(100000)"

#!/bin/bash
# Change to your actual project number later
#SBATCH -A naiss2025-22-262
#SBATCH -n 1
# Asking for 15 min (change as you want)
#SBATCH -t 00:15:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out
#SBATCH -p main

# Clean the environment
module purge > /dev/null 2>&1
module load  PDC/23.12 matlab/r2024a-ps

# Executing the matlab program monte_carlo_pi.m for the value n=100000
# (n is number of steps - see program).
# The command 'time' is timing the execution
time matlab -singleCompThread -nojvm -nodisplay -r "monte_carlo_pi(100000)"

You can download monte_carlo_pi.m here here or find it under matlab in the exercises directory.

You submit it with

sbatch <batchscript.sh>

Where <batchscript.sh> is the name you gave your batchscript. You can find ones for each of the clusters in the exercises -> matlab directory, named monte_carlo_pi_<cluster>.sh.

Parallel batch script¶

This is an example batch script for parallel MATLAB. Adjust to your preferred facility by cross-referencing the tabs above.

#!/bin/bash
# Change to your actual project number
#SBATCH -A XXXX-YY-ZZZ
#SBATCH --ntasks-per-node=<how many tasks>
#SBATCH --nodes <how many nodes>

# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module add MATLAB/<version>

# Executing a parallel matlab program
srun matlab -nojvm -nodisplay -nodesktop -nosplash -r parallel-matlab-script.m

Inside the MATLAB code, the number of CPU-cores (NumWorkers in MATLAB terminology) can be specified when creating the parallel pool, for example, with 8 threads:

>> poolobj = parpool('local', 8);

  ```bash
  #!/bin/bash
  # Change to your actual project number
  #SBATCH -A uppmax2025-2-272
  # Remember, there are 4 workers and 1 master!
  #SBATCH --ntasks=5
  #SBATCH --cpus-per-task=1
  #SBATCH --ntasks-per-node=5
  #SBATCH --ntasks-per-core=1
  # Asking for 30 min (change as you want)
  #SBATCH -t 00:30:00
  #SBATCH --error=matlab_%J.err
  #SBATCH --output=matlab_%J.out

  # Clean the environment
  module purge > /dev/null 2>&1

  # Change depending on resource and MATLAB version
  # to find out available versions: module spider matlab
  module add matlab/R2023b

  # Executing a parallel matlab program
  srun matlab -nojvm -nodisplay -nodesktop -nosplash -r "parallel_example(16)"
  ```

  ```bash
  #!/bin/bash
  # Change to your actual project number
  #SBATCH -A hpc2n2025-062
  # Remember, there are 4 workers and 1 master!
  #SBATCH --ntasks=5
  #SBATCH --cpus-per-task=1
  #SBATCH --ntasks-per-node=5
  #SBATCH --ntasks-per-core=1
  # Asking for 30 min (change as you want)
  #SBATCH -t 00:30:00
  #SBATCH --error=matlab_%J.err
  #SBATCH --output=matlab_%J.out

  # Clean the environment
  module purge > /dev/null 2>&1

  # Change depending on resource and MATLAB version
  # to find out available versions: module spider matlab
  module add MATLAB/2023a.Update4

  # Executing a parallel matlab program
  srun matlab -nojvm -nodisplay -nodesktop -nosplash -r "parallel_example(16)"
  ```

  ```bash
  #!/bin/bash
  # Change to your actual project number
  #SBATCH -A lu2025-7-24
  # Remember, there are 4 workers and 1 master!
  #SBATCH --ntasks=5
  #SBATCH --cpus-per-task=1
  #SBATCH --ntasks-per-node=5
  #SBATCH --ntasks-per-core=1
  # Asking for 30 min (change as you want)
  #SBATCH -t 00:30:00
  #SBATCH --error=matlab_%J.err
  #SBATCH --output=matlab_%J.out

  # Clean the environment
  module purge > /dev/null 2>&1

  # Change depending on resource and MATLAB version
  # to find out available versions: module spider matlab
  module add matlab/2023b

  # Executing a parallel matlab program
  srun matlab -nojvm -nodisplay -nodesktop -nosplash -r "parallel_example(16)"
  ```

  ```bash
  #!/bin/bash
  # Change to your actual project number
  #SBATCH -A naiss2025-22-262
  # Remember, there are 4 workers and 1 master!
  #SBATCH --ntasks=5
  #SBATCH --cpus-per-task=1
  #SBATCH --ntasks-per-core=1
  # Asking for 30 min (change as you want)
  #SBATCH -t 00:30:00
  #SBATCH --error=matlab_%J.err
  #SBATCH --output=matlab_%J.out

  # Clean the environment
  module purge > /dev/null 2>&1

  # Change depending on resource and MATLAB version
  # to find out available versions: module spider matlab
  module add MATLAB/2024a-hpc1-bdist

  # Executing a parallel matlab program
  srun matlab -nojvm -nodisplay -nodesktop -nosplash -r "parallel_example(16)"
  ```

  ```bash
  #!/bin/bash
  # Change to your actual project number
  #SBATCH -A naiss2025-22-262
  # Remember, there are 4 workers and 1 master!
  #SBATCH -p shared
  #SBATCH -n 5
  # Asking for 30 min (change as you want)
  #SBATCH -t 00:30:00
  #SBATCH --error=matlab_%J.err
  #SBATCH --output=matlab_%J.out

  # Clean the environment
  module purge > /dev/null 2>&1

  # Change depending on resource and MATLAB version
  # to find out available versions: module spider matlab
  module add PDC/23.12 matlab/r2024a-ps

  # Executing a parallel matlab program
  matlab -nodisplay -nodesktop -nosplash -r "parallel_example(16)"
  ```

GPU code¶

Here we cover how to use GPUs with MATLAB - Inside MATLAB - In a batch script

In order to use GPUs, you have to ask for them.

Inside MATLAB¶

In order to use GPUs from inside MATLAB, you must add them as additional properties to your profile.

Remember, after it is saved to your profile it will use GPUs again next time you submit a job, even if you don’t want GPUs there. To reset this, do:

c.AdditionalProperties.GpuCard = '';
c.AdditionalProperties.GpusPerNode = '';

This is how you add GPUs to use in batch jobs submitted inside MATLAB:

interactive -A uppmax2025-2-272 -n 2 -M snowy --gres=gpu:1  -t 2:00:00

ml matlab/R2023b

matlab -singleCompThread

matlab -singleCompThread -nodisplay -nosplash -nodesktop

c.AdditionalProperties.GpusPerNode = 1;
c.saveProfile

c.AdditionalProperties.GpuCard = 'card-type';
c.AdditionalProperties.GpusPerNode = '#gpus';
c.saveProfile

c.AdditionalProperties.GpusPerNode = #GPUs;
c.saveProfile

c.AdditionalProperties.GPUsPerNode = #GPUs;
c.saveProfile

!!! example “Challenge 5. Add/remove GPUs in your cluster profile.

Try to add GPUs to your cluster profile and save it. Run `c.AdditionalProperties` to see what was added. Then do `c.AdditionalProperties.GpusPerNode = '';` to remove it. See that it was removed.

In batch scripts¶

How to request a GPU node varies somewhat between clusters. Refer to the following templates:

#!/bin/bash
# Change to your actual project number
#SBATCH -A naiss20224-22-1202
#SBATCH -n 2
#SBATCH -M snowy
#SBATCH --gres=gpu:1
# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module add matlab/R2023b

# Executing a GPU matlab program
matlab -nodisplay -nosplash -r "gpu-matlab-script.m"

#!/bin/bash
# Change to your actual project number
#SBATCH -A hpc2n2025-062
#SBATCH -n 1
#SBATCH --gpus=<#gpus>
#SBATCH -C <gpu-type>
# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module load MATLAB/2023a.Update4

# Executing a GPU matlab program
matlab -nodisplay -nosplash -r "gpu-matlab-script.m"

=== NSC

```bash
#!/bin/bash
# Change to your actual project number
#SBATCH -A naiss2025-22-262
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=1
#SBATCH --ntasks-per-core=1
# The number of GPUs.#gpus, can be 1 or 2
#SBATCH --gpus-per-task=1

# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module load MATLAB/2024a-hpc1-bdist

# Executing a GPU matlab program
matlab -singleCompThread -nodisplay -nosplash -r "gpu-matlab-script.m"

#!/bin/bash
# Change to your actual project number
#SBATCH -A naiss2025-22-262
#SBATCH --ntasks-per-node=1
#SBATCH -N 1
# Ask for GPUs
#SBATCH -p gpu

# Asking for 30 min (change as you want)
#SBATCH -t 00:30:00
#SBATCH --error=matlab_%J.err
#SBATCH --output=matlab_%J.out

# Clean the environment
module purge > /dev/null 2>&1

# Change depending on resource and MATLAB version
# to find out available versions: module spider matlab
module load PDC/23.12 R/4.4.1-cpeGNU-23.12 rocm/5.7.0

# Executing a GPU matlab program
matlab -singleCompThread -nodisplay -nosplash -r "gpu-matlab-script.m"