Welcome to “Using Python in an HPC environment” course material

Content

  • This course aims to give a brief, but comprehensive introduction to using Python in an HPC environment.

  • You will learn how to

    • use modules to load Python

    • find site installed Python packages

    • install packages yourself

    • use virtual environments,

    • write a batch script for running Python

    • use Python in parallel

    • use Python for ML and on GPUs.

  • This course will consist of lectures interspersed with hands-on sessions where you get to try out what you have just learned.

  • We aim to give this course in spring and fall every year.

Course learning outcomes

<!– TODO: BC: Update –>

  • You can find and load a Python package module

  • You can determine if a Python package is installed

  • You can use an isolated environment

  • You can install Python packages

  • You can write and submit batch job scripts

  • You can use the compute nodes interactively

  • You can use a Python IDE

  • You can create plots in matplotlib

  • Use Python in parallel

  • Use Python for ML

  • Use GPUs with Python

Schedule Spring 2025

Schedule

See Schedule

Warning

Not covered

  • Improve Python coding skills

  • Specifics of other clusters

Course approach to deal with multiple HPC centers

Target group

  • The course is for present or presumptive users at the NAISS resources at PDC and NSC as well as UPPMAX, HPC2N, LUNARC, or possibly other clusters in Sweden.

  • The course is a NAISS cooperation with teachers from UPPMAX (Rackham, Snowy, Bianca), HPC2N (Kebnekaise), and LUNARC (Cosmos) and will focus on systems at the Swedish academic HPC centres with NAISS personnel.

  • Although there are differences we will only have few seperate sessions.

  • Most participants will use The NAISS resources Tetralith system at NSC or Dardel system at PDC for the course.

    • Alvis at C3SE (Chalmers) may be added in the future.

  • Users with the “local” affiliation below, can work at the following clusters:

    • Kebnekaise: UmU, IRF, MIUN, SLU, LTU.

    • Cosmos: LU.

    • Rackham/Snowy: UU.

  • The general information given in the course will be true for all/most HPC centres in Sweden.

    • The examples will often have specific information, like module names and versions, which may vary. What you learn here should help you to make any changes needed for the other centres.

    • When present, links to the Python documentation at other NAISS centres are given in the corresponding session.

Cluster documentation of Python

Some practicals

  • Code of Conduct

    • Be nice to each other!

  • Zoom

    • You should have gotten an email with the links

    • Zoom policy:

      • Zoom chat (maintained by co-teachers):

        • technical issues of zoom

        • technical issues of your settings

        • direct communication

        • each teacher may have somewhat different approach

      • collaboration document (see below):

        • “explain again”

        • elaborating the course content

        • solutions for your own work

    • Recording policy:

      • All lectures and demos will be available on HPC2Ns Youtube channel after the course.

      • Due to different teaching styles, the teacher of a session decides if a lesson will be:

        • recorded without learners

        • recorded live with learners

      • The teachers will be clear if it is recorded or not and it will be visible in the ZOOM.

      • For the live-recordings

        • The questions asked per microphone during these sessions will be recorded

        • If you don’t want your voice to appear:

          • use the collaboration document (see below)

    • The Zoom main room is used for most lectures

    • Some sessions use breakout rooms for exercises, some of which use a silent room

Q/A collabration document

Warning

  • Please be sure that you have gone through the Pre-requirements

    • It mentions the familiarity with the LINUX command line.

    • The applications to connect to the clusters: terminals and ThinLinc (remote graphical desktop)

  • This course does not aim to improve your coding skills. Rather you will learn to understand the ecosystems and navigations for the the different languages on a HPC cluster.

Content of the course

Pre-requirements:

Common:

Lessons day 1 (Intro to Python):

Lessons day 2 (packages and analysis):

Lessons day 3 (advanced analysis):

Lessons day 4 (parallel and ML):

Extra: