Introduction Python

Welcome page and syllabus

• Also link at House symbol 🏠 at top of page

Learning outcomes

• Load Python modules and site-installed Python packages

• Create a virtual environment

• Install Python packages with pip (Kebnekaise, Rackham, Snowy)

• Write a batch script for running Python

Exercises FIX

• You can download the exercises from the course GitHub repo, under the “Exercises” directory: https://github.com/UPPMAX/R-python-julia-HPC

Your expectations?

• Find best practices for using Python at UPPMAX and HPC2N

• Packages

• Use the HPC performance with Python

Not covered

• Improve python coding skills

• Other clusters (though they work in a mostly very similar way)

Preliminary schedule

Preliminary times

Time	Topic	Activity
9:00	Course intro	5m
9:05	Syllabus	5m
9.10	Python in general	Lecture 10 m
9.20	Loading modules and running Python codes	Lecture+code along 25m
9.45	Coffee break
10:00	Packages	Lecture+code along 45m
10.45	Leg stretch
11:00	Isolated environments (venv, conda)	Lecture+code along 60m
12:00	LUNCH BREAK
13:00	SLURM Batch scripts for Python jobs	Lecture+code along 20m
13.20	Using GPUs	10m
13:30	Parallel session — HPC2N: Jupyter	15m
	Parallel session — UPPMAX: Jupyter + interactive
13:45	Break
14:00	Parallel and multi-threaded functions	25m
14.25	Summary
14.35	Evaluation
14.45	Q&A on-demand
15:00	END

What is python?

As you probably already know…

• “Python combines remarkable power with very clear syntax.

• It has modules, classes, exceptions, very high level dynamic data types, and dynamic typing.

• There are interfaces to many system calls and libraries, as well as to various windowing systems. …“

In particular, what sets Python apart from other languages is its fantastic open-source ecosystem for scientific computing and machine learning with libraries like NumPy, SciPy, scikit-learn and Pytorch.

• Official Python documentation is found here https://www.python.org/doc/ .

• Python forum is found here https://python-forum.io/ .

Material for improving your programming skills

First level

The Carpentries teaches basic lab skills for research computing.

• Programming with Python

• Plotting and Programming in Python

Second level

CodeRefinery develops and maintains training material on software best practices for researchers that already write code. Their material addresses all academic disciplines and tries to be as programming language-independent as possible.

• Lessons

• Data visualization using Python

• Jupyter

Aalto Scientific Computing

• Data analysis workflows with R and Python

• Python for Scientific Computing

  • Introduction to packages

Third level

ENCCS (EuroCC National Competence Centre Sweden) is a national centre that supports industry, public administration and academia accessing and using European supercomputers. They give higher-level training of programming and specific software.

• High Performance Data Analytics in Python

• The youtube video Thinking about Concurrency is a good introduction to writing concurrent programs in Python

• The book High Performance Python is a good resource for ways of speeding up Python code.

Other NAISS centres

• LUNARC

  • Python

  • Jupyter

• C3SE

  • Python

  • Jupyter

• NSC

  • Python

• PDC

  • Python

Objectives

We will:

• Teach you how to navigate the module system

• Show you how to find out which versions of Python and packages are installed

• Use the package handler pip

• Explain how to create and use virtual environments

• Show you how to run batch jobs

• Show some examples with parallel computing and using GPUs

Most of this will be the same or very similar to how it is done at other HPC centres in Sweden