Meeting 14 Feb 2025
Agenda
Status
Follow-up from last meeting:
schedule/teachers
Discussions
Next meeting
ToDos
Status
Advertisement out
Day 1: Log in and use Basic Python in a cluster environment
Day 2: Use and install packages with pip and conda in virtual environments and doing basic analysis interactively
Day 3-4: Deeper analysis with pandas and seaborn, HPC usage with big data, parallelisms and Machine Learning with and without GPUs.
Registration is active
Follow-up from last meeting
What goes to which day? And Teachers
Earlier sessions from HPC-python (to be placed out)
intro (just first day?) [name=RB]
load (just first day?) [name=RB]
install packages (expanded) [name=BC]
Compute nodes [name=BB]
batch [name=BB]
interactive [name=B]
Desktop on demand [name=RP]
IDEs [name=BB]
matplotlib [name=RP]
GPU [name=BB]
Pandas [name=RP]
seaborn etc [name=RP]
parallel [name=PO]
big data and formats [name=BC]
Dimensionality reduction? [name=AH]
ML/DL [name=JY]
Day 1 (copied from NAISS ‘Intro to Python’)
:warning: Does not include loading a module
BC: could we have “loading lmod module” here?
Yes it is included
We must make videos and follow GDPR laws, as demanded by the HPC2N training coordinator
Can send pre-recorded videos
Can send post-recorded videos
Birgitte will edit in-lesson recordings, if shared documents (without names) are used and not too much dialogues
Pedro and Yayant want this :-)
Why do our videos have comments disabled?
Because at HPC2N this was decided
Time |
Topic |
Teacher |
|---|---|---|
09:00-10:00 |
Using the Python interpreter, includes login |
Richel |
10:00-10:15 |
Break |
. |
10:15-11:00 |
The way of the program, includes creating and running a script |
Richel |
11:00-11:15 |
Break |
. |
11:15-12:00 |
Working with Python scripts |
Richel |
12:00-13:00 |
Break |
. |
13:00-14:00 |
Variables, expressions and statements: variables |
Richel |
14:00-14:15 |
Break |
. |
14:15-15:00 |
Variables, expressions and statements: operators |
Richel |
15:00-15:15 |
Break |
. |
15:15-15:45 |
Variables, expressions and statements: user input |
Richel |
15:45-16:00 |
Evaluation |
Richel |
Day 2
[VOTED] IDEs
Packages
matplotlib
??pandas here already?
Suggestions from Björn
Time |
Topic |
Teacher |
|---|---|---|
09:00-10:00 |
Packages |
Richel |
10:00-10:15 |
Break |
. |
10:15-11:00 |
Isolated environments conda/pip |
Björn+Richel (co-teaching) |
11:00-11:15 |
Break |
. |
11:15-12:00 |
Basic Slurm and interactive, ?OnDemand,?parallel session |
Birgitte |
12:00-13:00 |
Break |
. |
13:00-14:00 |
IDEs: Jupyter/vscode/Spyder [1] |
Jayant |
14:00-14:15 |
Break |
. |
14:15-15:00 |
matplotlib 1/2 |
?Rebecca, else Björn |
15:00-15:15 |
Break |
. |
15:15-15:45 |
matplotlib 2/2 |
?Rebecca, else Björn |
15:45-16:00 |
Evaluation |
. |
[1] Spyder only works on LUNARC, for an old version. Using virtual environments may get this to work on other session. Björn will investigate this :+1:
Day 3
Deeper analysis with pandas and seaborn, HPC usage with big data, parallelisms and Machine Learning with and without GPUs.
Time |
Topic |
Teacher |
|---|---|---|
09:00-10:00 |
Pandas |
?Rebecca ?Richel |
10:00-10:15 |
Break |
|
10:15-11:00 |
Pandas (enough time?) |
?Rebecca ?Richel |
11:00-11:15 |
Break |
. |
11:15-12:00 |
Seaborn (enough time)? |
?Rebecca ?Björn |
12:00-13:00 |
Break |
. |
13:00-14:00 |
Slurm and batch jobs |
Birgitte |
14:00-14:15 |
Break |
. |
14:15-15:00 |
Formats and Big data |
Björn (more time than earlier) |
15:00-15:15 |
Break |
. |
15:15-15:45 |
Formats and Big Data |
Björn |
15:45-16:00 |
Evaluation |
. |
Day 4
Deeper analysis with pandas and seaborn, HPC usage with big data, parallelisms and Machine Learning with and without GPUs.
Suggestions from Björn
Time |
Topic |
Teacher |
|---|---|---|
09:00-10:00 |
Parallelism |
Pedro |
10:00-10:15 |
Break |
. |
10:15-11:00 |
Parallelism |
Pedro |
11:00-11:15 |
Break |
. |
11:15-12:00 |
GPU, batch jobs |
Birgitte |
12:00-13:00 |
Break |
. |
13:00-14:00 |
ML/DL |
Jayant |
14:00-14:15 |
Break |
. |
14:15-15:00 |
ML/DL |
Jayant |
15:00-15:15 |
Break |
. |
15:15-15:45 |
ML/DL |
Jayant + Anders Hast |
15:45-16:00 |
Evaluation |
. |
Discussions
Still 2 repos?
Pro: Intro to Python (Day 1) is given separately more often
Yes: BC, BB, PO, RB, JY
No:
MkDocs? Candidate NAISS course template
Yes: RB, JY, BB, BC
No: PO
Transition to MkDocs now?
Yes: RB, BB, BC, JY
No: PO
How:
parallel file tree in
docsfoldernot needed
rst and md files in same dir structure
Python versions: which one?
There were problems last time on Tetralith, we can request updates in time :+1:
Who will talk to NSC?
First find out if there is a problem at all, else BB will send an email :-)
Use same as last time? v3.11.>4
Yes: RB,
No:
Problems with virtual environments?
On Tetralith we think this happened, we created a virtual environment to solved
Next meeting
Week 9: Friday 28th February 11:00-12:00
ToDos
All:
Look at evaluation and think about changes
LOs
Tests on Dardel
teacher
Björn: HPC2N need access to UPPMAX project for training
Check packages on Dardel and possible solutions
Someone
ask NSC/PDC(?) of updates
Ad for newsletter
Living ad
Online training course: Using Python in an HPC environment, 24-25 and 28-29 April 2025 Collaboration between UPPMAX, HPC2N and LUNARC
This online 4-day workshop aims to give a brief, but comprehensive introduction to using Python in a Swedish academic High-Performance Computing (HPC) environment. It covers a wide range of levels in using Python and you can decide which days to follow.
The content of the different days are
Day 1: Log in and use Basic Python in a cluster environment
Day 2: Use and install packages with pip and conda in virtual environments and doing basic analysis interactively
Day 3-4: Deeper analysis with pandas and seaborn, HPC usage with big data, parallelisms and Machine Learning with and without GPUs.
Remote/online participation: We will use Zoom. More information about connecting and about accounts and course projects and such will be sent to the participants close to the course. Tetralith at NSC will mainly be used for demos but written material should guide users within academic Sweden on the resources they already are using.
Prerequisites: Familiarity with the Linux command line. Basic Python is NOT required, but recommended. Attend the first day for an introduction!
Date and time: 24-25 & 28-29 April, 9-17 each day.
For more info and registration, please visit: https://www.hpc2n.umu.se/events/courses/2025/spring/hpc-python
Living long document that can be merged into webpage document
Online training course: Using Python in an HPC environment, 24-25 and 28-29 April 2025 Collaboration between UPPMAX, HPC2N and LUNARC
This online 4-day workshop aims to give a brief, but comprehensive introduction to using Python in a Swedish academic High-Performance Computing (HPC) environment. It covers a wide range of levels in using Python and you can decide which days to follow.
If you are a complete beginner, perhaps the first day(s) is sufficient for you, but you may follow the other days as well for inspiration. If you already are a Python programmer you could still benefit from attending the login session the first day.
The goals of the different days are
Day 1:
Log in to a cluster and load python interpreter
Use variables, expressions and statements with Python
Day 2:
Load and use site-installed Python modules
Use and install python packages with pip and/or Conda
Use virtual environments
Ask for compute resources, write batch scripts
Do basic analysis with Matplotlib in Jupyter
Day 3-4:
Use compute resources and batch jobs, including parallelism, GPUs and big data
Use Pandas and Seaborn
Use Python for ML
Remote/online participation: We will use Zoom. More information about connecting and about accounts and course projects and such will be sent to the participants close to the course. Tetralith at NSC will mainly be used for demos but written material should guide users within academic Sweden on the resources they already are using.
Prerequisites: Familiarity with the Linux command line. Basic Python is NOT required, but recommended. Attend the first day for an introduction!
Date and time: 24-25 & 28-29 April, 9-17 each day.
For more info and registration, please visit: https://www.hpc2n.umu.se/events/courses/2025/spring/hpc-python
Old
Online training course: Using Python in an HPC environment, 5-6 December 2024 Collaboration between UPPMAX, HPC2N, and LUNARC
This two-day online course aims to give a brief, but comprehensive introduction to using Python in an HPC environment. You will learn how to use modules to load Python, how to find site installed Python packages, as well as how to install packages yourself. In addition, you will learn how to use virtual environments, write a batch script for running Python, use Python in parallel, and how to use Python for ML and on GPUs. In addition, the second day will be more advanced, with a focus on Pandas.
The course is a cooperation between UPPMAX, HPC2N, and LUNARC. The instructors will use UPPMAX’s systems for demos and there will be hands-on exercises for the participants.
This course will consist of lectures interspersed with hands-on sessions where you get to try out what you have just learned.
Remote/online participation: The course will be completely online and we will use Zoom. More information about connecting and such will be sent to the participants close to the course.
The goal for the course is that you will be able to • Load Python modules and site-installed Python packages • Create a virtual environment and install your own Python packages to it • Write a batch script for running Python • Use Python in parallel • Use Python for ML • Use GPUs with Python • Use pandas • Learn about matplotlib
• Prerequisites: familiarity with the LINUX command line, basic Python
• Date and time: 5-6 December 2024, 9-17 each day.
• For more info and registration, please visit: https://www.hpc2n.umu.se/events/courses/2024/fall/hpc-python