Skip to content

Lesson plan Bianca Intermediate 2026-11-23

2026-11-19

I started with copying the existing material to a subfolder, so I can start from scratch, with the existing material as a sketch. First thing I already see, that there is no 'Why' in the older material.

There are only three GPU modules:

My module spider results
 _   _ ____  ____  __  __    _    __  __
| | | |  _ \|  _ \|  \/  |  / \   \ \/ /   | System:    sens2025560-bianca
| | | | |_) | |_) | |\/| | / _ \   \  /    | User:      richel
| |_| |  __/|  __/| |  | |/ ___ \  /  \    | 
 \___/|_|   |_|   |_|  |_/_/   \_\/_/\_\   | 

###############################################################################

        User Guides: http://www.uppmax.uu.se/support/user-guides
        FAQ: http://www.uppmax.uu.se/support/faq

        Write to support@uppmax.uu.se, if you have questions or comments.


[richel@sens2025560-bianca richel]$ module spider gpu

----------------------------------------------------------------------------
  Guppy:
----------------------------------------------------------------------------
     Versions:
        Guppy/5.0.16-cpu
        Guppy/5.0.16-gpu
        Guppy/6.0.6-cpu
        Guppy/6.0.6-gpu
        Guppy/6.3.7-cpu
        Guppy/6.3.7-gpu
        Guppy/6.4.2-cpu
        Guppy/6.4.2-gpu
        Guppy/6.5.7-cpu
        Guppy/6.5.7-gpu

----------------------------------------------------------------------------
  For detailed information about a specific "Guppy" package (including how to lo
ad the modules) use the module's full name.
  Note that names that have a trailing (E) are extensions provided by other modu
les.
  For example:

     $ module spider Guppy/6.5.7-gpu
----------------------------------------------------------------------------

----------------------------------------------------------------------------
  Megalodon: Megalodon/2.5.0-gpu
----------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the
 "Megalodon/2.5.0-gpu" module is available to load.

      bioinfo-tools

    Help:
      Megalodon - use Megalodon 

      Description

      Megalodon is a research command line tool to extract high accuracy modifie
d base and sequence variant calls from raw nanopore reads by anchoring the infor
mation rich basecalling neural network output to a reference genome/transcriptom
e.

      Version 2.5.0-gpu

      https://github.com/nanoporetech/megalodon

      Usage:



----------------------------------------------------------------------------
  python_ML_packages: python_ML_packages/3.9.5-gpu
----------------------------------------------------------------------------

    This module can be loaded directly: module load python_ML_packages/3.9.5-gpu

    Help:
      python_ML_packages - use python_ML_packages 

      Version 3.9.5-gpu

[richel@sens2025560-bianca richel]$ 
My module spider guppy
[richel@sens2025560-bianca richel]$ module spider Guppy/6.5.7-gpu

----------------------------------------------------------------------------
  Guppy: Guppy/6.5.7-gpu
----------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the
 "Guppy/6.5.7-gpu" module is available to load.

      bioinfo-tools

    Help:
      Guppy - use Guppy/6.5.7-gpu, GPU-enabled

      Description

      Oxford Nanopore neural-net basecaller, GPU enabled

          Version 6.5.7-gpu

          https://nanoporetech.com/nanopore-sequencing-data-analysis

      This is the -gpu version of this module.  This is only available on cluste
rs
      containing GPUs.  The -cpu version of this module is available on all clus
ters.

      Configuration files for use with the -c option are available at
      $GUPPY_ROOT/data/filename.cfg after this module is loaded. Several example
s of
      available configuration files:

              -c $GUPPY_ROOT/data/dna_r9.4.1_450bps_fast.cfg

              -c $GUPPY_ROOT/data/dna_r9.4.1_450bps_hac.cfg

              -c $GUPPY_ROOT/data/dna_r9.4.1_450bps_sup.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_5khz_modbases_5mc_cg_s
up_prom.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_5khz_sup.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_fast.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_fast_mk1c.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_fast_prom.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_hac.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_hac_mk1c.cfg

              -c $GUPPY_ROOT/data/dna_r10.4.1_e8.2_400bps_hac_prom.cfg

      This command will show a complete listing of available configuration files
 and
      other provided data:

           ls -l $GUPPY_ROOT/data/
Module Description
Guppy Oxford Nanopore neural-net basecaller, GPU enabled
Megalodon Base and sequence variant caller from raw nanopore reads
python_ML_packages/3.9.5-gpu Python GPU packages
<

I read at the Bianca technical summary that

  • 148 (ordinary/thin) compute nodes
  • 10 fat compute nodes
  • 27 GPU nodes

So around 17% of all Bianca nodes are GPU nodes.

Bianca works differently on GPU nodes:

[richel@sens2025560-bianca richel]$ nvidia-smi
bash: nvidia-smi: command not found

Let's try an interactive session:

[richel@sens2025560-bianca richel]$ interactive -A sens2025560 -C gpu
You receive the high interactive priority.
You may run for at most 12 hours.
Please note that in 12 wallclock hours this interactive job will use 192 core hours.

Please, use no more than 14 GB of RAM.

Waiting for job 669 to start...
^CKilling program 669...

OK, let's specify one GPU per node:

[richel@sens2025560-bianca richel]$ interactive -A sens2025560 -C gpu --gpus-per-node=1
You receive the high interactive priority.
You may run for at most 12 hours.
Please note that in 12 wallclock hours this interactive job will use 192 core hours.

Please, use no more than 14 GB of RAM.

Waiting for job 670 to start...

Hence:

  • -C gpu is identical to -C gpu --gpus-per-node=1
  • By default, when you book a GPU, you get 1
  • 1 GPU hour is 16 CPU hours
[richel@sens2025560-bianca richel]$ interactive -A sens2025560 -C gpu --gpus=1 --gpus-per-node=1
You receive the high interactive priority.
You may run for at most 12 hours.
Please note that in 12 wallclock hours this interactive job will use 192 core hours.

Please, use no more than 14 GB of RAM.

Waiting for job 671 to start...

Hence:

  • -C gpu is identical to -C gpu --gpus-per-node=1
  • -C gpu --gpus-per-node=1 is identical to -C gpu --gpus=1 --gpus-per-node=1
  • By default, when you book a GPU, you get 1 GPU
  • 1 GPU hour is 16 CPU hours
[richel@sens2025560-bianca richel]$ date
Thu Nov 19 15:30:36 CET 2026
[richel@sens2025560-bianca richel]$ date; interactive -A sens2025560 -C gpu --gpus=1 --gpus-per-node=1 ; date
Thu Nov 19 15:30:47 CET 2026
You receive the high interactive priority.
You may run for at most 12 hours.
Please note that in 12 wallclock hours this interactive job will use 192 core hours.

Please, use no more than 14 GB of RAM.

Waiting for job 672 to start...

It takes around 20 minutes to obtain an interactive session on a GPU node:

  • Start: Thu Nov 19 15:30:47 CET 2026
  • End: Thu Nov 19 15:50:37 CET 2026
  • Difference: 20 minutes
[richel@sens2025560-bianca richel]$ date; interactive -A sens2025560 -C gpu --gpus=1 --gpus-per-node=1 ; date
Thu Nov 19 15:30:47 CET 2026
You receive the high interactive priority.
You may run for at most 12 hours.
Please note that in 12 wallclock hours this interactive job will use 192 core hours.

Please, use no more than 14 GB of RAM.

Waiting for job 672 to start...
Starting job now -- you waited for 1150 seconds.
[screen is terminating]
Connection to sens2025560-b201 closed.
Thu Nov 19 15:50:37 CET 2026
[richel@sens2025560-bianca richel]$ 

Starting a new job, this time it was a matter of seconds:

 _   _ ____  ____  __  __    _    __  __
| | | |  _ \|  _ \|  \/  |  / \   \ \/ /   | System:    sens2025560-b201
| | | | |_) | |_) | |\/| | / _ \   \  /    | User:      richel
| |_| |  __/|  __/| |  | |/ ___ \  /  \    | 
 \___/|_|   |_|   |_|  |_/_/   \_\/_/\_\   | 

###############################################################################

        User Guides: http://www.uppmax.uu.se/support/user-guides
        FAQ: http://www.uppmax.uu.se/support/faq

        Write to support@uppmax.uu.se, if you have questions or comments.


[richel@sens2025560-b201 richel]$ 

The b201 indicates I am on a GPU I guess...

[richel@sens2025560-b201 richel]$ nvidia-
nvidia-bug-report.sh     nvidia-debugdump         nvidia-sleep.sh
nvidia-cuda-mps-control  nvidia-modprobe          nvidia-smi
nvidia-cuda-mps-server   nvidia-persistenced      nvidia-xconfig
[richel@sens2025560-b201 richel]$ nvidia-

Running nvidia-smi:

[richel@sens2025560-b201 richel]$ nvidia-smi
Thu Nov 19 15:53:56 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100-PCIE-40GB          Off |   00000000:00:08.0 Off |                    0 |
| N/A   34C    P0             36W /  250W |       1MiB /  40960MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

No GPU:

[richel@sens2025560-bianca richel]$ module load python_ML_packages/3.9.5-gpu
Lmod Warning: 
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: CUDAcore/11.1.1
(required by: CUDA/11.1.1-GCC-10.2.0, GDRCopy/2.1-GCCcore-10.2.0-CUDA-11.1.1,
UCX/1.9.0-GCCcore-10.2.0-CUDA-11.1.1)
-------------------------------------------------------------------------------

[richel@sens2025560-bianca richel]$ python -c "import torch; print(torch.__version__); print(torch.version.cuda); print(torch.cuda.get_device_properties(0)); print(torch.randn(1).cuda())"
1.9.0+cu111
11.1
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/sw/apps/python_ML_packages/3.9.5-gpu/bianca/lib/python3.9/site-packages/torch/cuda/__init__.py", line 309, in get_device_properties
    _lazy_init()  # will define _get_device_properties
  File "/sw/apps/python_ML_packages/3.9.5-gpu/bianca/lib/python3.9/site-packages/torch/cuda/__init__.py", line 172, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx
[richel@sens2025560-bianca richel]$ 

On no GPU, the version is found:

[richel@sens2025560-bianca richel]$ python -c "import torch; print(torch.__version__)"
1.9.0+cu111

On no GPU, the CUDA version is found too:

[richel@sens2025560-bianca richel]$ python -c "import torch; print(torch.version.cuda)"
11.1

On no GPU, here we see a first fail, which is useful:

[richel@sens2025560-bianca richel]$ python -c "import torch; print(torch.cuda.get_device_properties(0))"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/sw/apps/python_ML_packages/3.9.5-gpu/bianca/lib/python3.9/site-packages/torch/cuda/__init__.py", line 309, in get_device_properties
    _lazy_init()  # will define _get_device_properties
  File "/sw/apps/python_ML_packages/3.9.5-gpu/bianca/lib/python3.9/site-packages/torch/cuda/__init__.py", line 172, in _lazy_init
    torch._C._cuda_init()
RuntimeError: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx

Yes GPU:

[richel@sens2025560-b201 richel]$ module load python_ML_packages/3.9.5-gpu
Lmod Warning: 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: CUDAcore/11.1.1 (required by: CUDA/11.1.1-GCC-10.2.0, GDRCopy/2.1-GCCcore-10.2.0-CUDA-11.1.1,
UCX/1.9.0-GCCcore-10.2.0-CUDA-11.1.1)
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------



[richel@sens2025560-b201 richel]$ python -c "import torch; print(torch.__version__); print(torch.version.cuda); print(torch.cuda.get_device_properties(0)); print(torch.randn(1).cuda())"
1.9.0+cu111
11.1
_CudaDeviceProperties(name='NVIDIA A100-PCIE-40GB', major=8, minor=0, total_memory=40326MB, multi_processor_count=108)
tensor([-0.6620], device='cuda:0')
[richel@sens2025560-b201 richel]$ 

2026-11-20

Most of my notes are in the course material. I am happy to finally learn about GPU.