Using IDEs

There are several popular IDEs that are commonly used for interactive work with Python. Here we will show how to load Jupyter, VS Code, and Spyder.

Jupyter

Note

Jupyter is web application that (among other things) allows literature programming for Python. That is, Jupyter allows to create documents where Python code is shown and run and its results shown, surrounded by written text (e.g. English).

Additionally, Jupyter allows to share files and hence includes a file manager.

Jupyter is:

  • started and run on a server, for example, an interactive node

  • displayed in a web browser, such as firefox.

Jupyter can be slow when using a remote desktop website (e.g. rackham-gui.uppmax.uu.se or kebnekaise-tl.hpc2n.umu.se).

  • For HPC2N, as JupyterLab it is only accessible from within HPC2N’s domain, and there is no way to improve any slowness

  • For UPPMAX, one can use a locally installed ThinLinc client to speed up Jupyter. See the UPPMAX documentation on ThinLinc on how to install the ThinLinc client locally

  • For LUNARC, you can run Jupyter either in compute nodes through Anaconda or through the LUNARC HPC desktop. The latter is recommended. There is information about Jupyter at LUNARC in their documentation.

  • For NSC, you can start Thinlinc and run Jupyter on a login node, or use a browser on your local computer with SSH tunneling which could be faster.

Note

Depending on your requirement of GPU or CPU, you can use Jupyter on either Rackham or Snowy compute node.

1. Login to a remote desktop

Alt1. Login to the remote desktop website at rackham-gui.uppmax.uu.se Alt2. Login to your local ThinLinc client

2. start an interactive session

Start a terminal. Within that terminal, start an interactive session from the login node (change to the correct NAISS project ID)

$ interactive -A uppmax2025-2-393 -t 4:00:00

3. start Jupyter in the interactive session

Within your terminal with the interactive session, load a modern Python module:

module load python/3.11.8

Then, start jupyter-notebook (or jupyter-lab):

jupyter-notebook --ip 0.0.0.0 --no-browser

This will start a jupyter server session so leave this terminal open. The terminal will also display multiple URLs.

4. Connect to the running Jupyter server

On ThinLinc

If you use the ThinLinc, depending on which Jupyter server (Rackham or Snowy) you want to launch on web browser

  • start firefox on the ThinLinc.

  • browse to the URLs, which will be similar to http://r[xxx]:8888/?token=5c3aeee9fbfc7a11c4a64b2b549622231388241c2

  • Paste the url and it will start the Jupyter interface on ThinLinc and all calculations and files will be on Rackham.

On own computer

If you want to connect to the Jupyter server running on Rackham/Snowy from your own computer, you can do this by using SSH tunneling. Which means forwarding the port of the interactive node to your local computer.

  • On Linux or Mac this is done by running in another terminal. Make sure you have the ports changed if they are not at the default 8888.

$ ssh -L 8888:r486:8888 username@rackham.uppmax.uu.se
  • If you use Windows it may be better to do this in the PowerShell instead of a WSL2 terminal.

  • If you use PuTTY - you need to change the settings in “Tunnels” accordingly (could be done for the current connection as well).

  • On your computer open the URL you got from step 3. on your webbrowser but replace r486 with localhost i.e. you get something like this

http://localhost:8888/?token=5c3aeee9fbfc75f7a11c4a64b2b5b7ec49622231388241c2 or http://127.0.0.0:8888/?token=5c3aeee9fbfc75f7a11c4a64b2b5b7ec49622231388241c2

  • This should bring the jupyter interface on your computer and all calculations and files will be on Rackham.

Warning

Running Jupyter in a virtual environment

You could also use jupyter (-lab or -notebook) in a virtual environment.

If you decide to use the –system-site-packages configuration you will get jupyter from the python modules you created your virtual environment with. However, you won’t find your locally installed packages from that jupyter session. To solve this reinstall jupyter within the virtual environment by force:

$ pip install -I jupyter

and run:

$ jupyter-notebook

Be sure to start the kernel with the virtual environment name, like “Example”, and not “Python 3 (ipykernel)”.

JupyterLab 🚀

JupyterLab is the next-generation web-based user interface for Project Jupyter. It provides an interactive development environment for working with notebooks, code, and data. JupyterLab offers a more flexible and powerful interface compared to the classic Jupyter Notebook, allowing users to arrange multiple documents and activities side by side in tabs or split screens.

Jupyter Notebook 📝

Jupyter Notebook is a sibling to other notebook authoring applications under the Project Jupyter umbrella, like JupyterLab. Jupyter Notebook offers a lightweight, simplified experience compared to JupyterLab.

  • Running python requires a kernel to be started. IPython is the default kernel for Jupyter if you want to run python code. You can also install other kernels to run other programming languages.

  • A Jupyter Notebook is made up of cells. There are two main types of cells: Code cells and Markdown cells.

  • Code cells 🧩 allow you to write and execute Python code. When you run a code cell, the output is displayed directly below the cell.

  • Markdown cells 📝 allow you to write formatted text using Markdown syntax. This is useful for adding explanations, headings, lists, links, and even LaTeX equations to your notebook.

  • You can run a cell by selecting it and pressing Shift + Enter or by clicking the “Run” button in the toolbar ▶️.

  • Cell execution order matters - Jupyter allows cells to be run in any order, but the execution number in square brackets shows the order in which cells were actually run.

  • Restarting the kernel 🔄 is useful when you want to clear all variables and start fresh. Go to “Kernel” > “Restart”.

  • Clearing all outputs 🧹 can be done by selecting “Cell” > “All Output” > “Clear” from the menu.

  • Exporting notebooks 📤 to different formats (HTML, PDF, Markdown, etc.) can be done via “File” > “Download as” or “Export Notebook As”.

  • Using magic commands ✨ like %timeit, %matplotlib inline, %load, and %%writefile can enhance your workflow significantly.

Exercise

  • Try Jupyter interface with the following Notebook Code.

  • Either start an interactive session with 4-8 cores and 1-2 hr walltime or stay on the login node.

  • cd into your project directory and start Jupyter Notebook or JupyterLab as described in previous sections. (Optional: load matplotlib module by searching it with module spider matplotlib and then loading it.)

  • Create a new Python 3 notebook.

Spyder

Note

Spyder is a powerful and flexible IDE originally developed to be the main scripting environment for scientific Anaconda users. It is designed to enable quick and easily repeatable experimentation, with automatic syntax checking, auto-complete suggestions, a runtime variable browser, and a graphics window that makes plots easy to manipulate after creation without additional code.

Spyder is available independent of Anaconda, but conda is still the recommended installer. Packages from the conda-forge source repo are still open-source, so conda is still usable on some facilities despite the recent changes in licensing. It is also possible to build a pip environment with Spyder, although this is only recommended for experienced Python users running on Linux operating systems.

To use Spyder on one of the HPC center resources, you must have a Thinlinc window open and logged into your choice of HPC resource. For personal use, it is relatively easy to install as a standalone package on Windows or Mac, and there is also the option of using Spyder online via Binder.

On COSMOS, the recommended way to use Spyder is to use the On-Demand version in the Applications menu, under Applications - Python. All compatible packages should be configured to load upon launching, so you should only have to specify walltime and maybe a few extra resource settings with the GfxLauncher so that spyder will run on the compute nodes. Refer to the Desktop On Demand documentation to help you fill in GfxLauncher prompt.

Avoid launching Spyder from the command line on the login node.

Features

When you open Spyder you will see a three-pane layout similar to other scientific IDEs: a large Editor on the left, and one or more utility panes on the right and bottom. The exact arrangement can be customised and saved as a layout.

Main panes and useful tabs

  • Editor (left) 📝
    • Edit multiple files in tabs : use the Run toolbar (green ▶️) to run the current file or selection.

    • You can mark cells with # %% (or # %%% for multi level cells) and run cells interactively.

    • Use the burger menu on the editor pane to split, undock or save layout.

  • Utility pane (Top-right) 🧰
    • Help 🔍: documentation and contextual help for the symbol under the cursor.

    • Variable explorer 🧾 : table view of in-memory variables with types and values; inspect, edit or remove variables.

    • Files 📂 : file browser for the current working directory.

    • Plots 📈 : interactive plots panel (may be a tab in recent versions); can be undocked to a separate window for better interaction.

  • Console and History (Bottom-right) 🖥️
    • IPython console(s) : interactive Python connected to the currently selected kernel/environment.

    • History log : previously executed commands from the console.

Exercise

  • Try Spyder interface with the following code.

  • Either start an interactive session with 4-8 cores and 1-2 hr walltime or stay on the login node.

  • cd into your project directory.

  • Start Spyder from your python env that you created in previous session.

  • Type out (or copy) the following code into the IDE editor and run it to see the results.

VS Code

Note

VS Code is a powerful and flexible IDE that is popular among developers for its ease of use and flexibility. It is designed to be a lightweight and fast editor that can be customized to suit the user’s needs. It has a built-in terminal, debugger, and Git integration, and can be extended with a wide range of plugins.

VS Code can be downloaded and installed on your local machine from the VS Code website. It is also available on the HPC center resources, but the installation process is different for each center.

VS Code is available on ThinLinc on UPPMAX and LUNARC only. On HPC2N and NSC, you will have to install it on your own laptop. At UPPMAX(Rackham) load it using module load VSCodium, this is an open source version of VS Code. At LUNARC(Cosmos) you can find it under Applications->Programming->Visual Studio Code.

However, VS Code is best used on your local machine, as it is a resource-intensive application that can slow down the ThinLinc interface. The VS Code Server can be installed on all the HPCs that give your the ability to run your code on the HPCs but edit it on your local machine. Similarly, you can also install your faviroute extensions on the HPCs and use them on your local machine. Care should be taken while assigning the correct installation directories for the extensions because otherwise they get installed in home directory and eat up all the space.

Install VS Code on your local machine and follow the steps below to connect to the HPC center resources.

When you first establish the ssh connection to Rackham, your VSCode server directory .vscode-server will be created in your home folder /home/[username]. This also where VS Code will install all your extensions that can quickly fill up your home directory.

Features

VS Code provides a flexible, extensible interface that is well suited for editing, debugging, and remote development. Below are the main panes, useful workflows, and configuration tips to get the most out of VS Code when working locally or connected to an HPC resource.

Main panes and useful panels

  • Explorer (left) 📁
    • File tree and quick file operations.

    • Right-click to open terminals, reveal in OS file manager, or run commands.

  • Editor (center) ✍️
    • Supports tabs, split editors, and editor groups.

    • Rich file-type support (syntax, linting, formatting).

    • Notebook editor for .ipynb files when Jupyter extension is installed.

  • Side bar (left) 🔧
    • Run & Debug, Source Control, Extensions, Search, Remote Explorer.

  • Panel (bottom) 🖥️
    • Integrated Terminal(s): run jobs, activate envs, and submit sbatch scripts.

    • Output, Problems, Debug Console, and Tests.

  • Status bar (bottom) ℹ️
    • Shows current interpreter, Git branch, line endings, and active extensions.

    • Click the interpreter area to switch Python interpreters or kernels.

Exercises

Exercise

  • Try running a Scipy and a Pytorch example in your favorite IDE.

  • Create a Virtual env using your faviroute package manager and install the packages.

  • For an extra challenge: Run the same code in .ipynb format in your IDE. This requires you to install jupyter notebook in your virtual environment.

Learning outcomes:
  • How to use IDE on any system

  • How to run code on IDE

  • How to install and manage extensions on remote VSCode server