Skip to content

Install with pip to Bianca

Read through the content below

Installation principle

  • Log in to Transit
  • (If not done already) Mount the wharf of your project.
  • Go to project directory
  • Download package
    • pip download <package name>
  • You will get a zip file for each package
  • From Bianca session move the file(s) to correct place
  • Typical place to put python packages: ~/.local/lib/python<version>/site-packages/

Check for packages

  • from the Python shell with the import command

  • from BASH shell with the

  • pip list command

  • Ex. ml help python/3.9.5 at UPPMAX

Is it not there? Then proceed!

Info

Methods:

  • You can just download a python package on Transit with pip, then install from the .whl file in wharf on Bianca.
  • Rackham users: Install it on Rackham. Perhaps you need it here as well! Then transfer to wharf and Bianca local python library.

Users without access to Rackham/Pelle

  • NOTE that if you install a package this way, you need to handle any dependencies yourself.

Use transit!

  1. Log in to transit
  2. Go to the mounted project folderr
  3. Load Python of desired version (IMPORTANT!)
  4. Download with pip
  5. On Bianca: Load Python of desired version (IMPORTANT!)
  6. pip install

Transit

  • Log in to transit: ssh <username>@transit.uppmax.uu.se
  • (If not done already) Mount the wharf of your project.
user@transit:~$ mount_wharf sens2025560
Mounting wharf (accessible for you only) to /home/<user>/sens2025560
<user>-sens2025560@bianca-sftp.uppmax.uu.se's password:
  • Navigate to your wharf folder
cd sens2025560

Download on Transit

  • Load the Python version you plan to use
$ ml python/<version>
  • Download!
$ pip download <package-name>==<version>
  • You should get a .whl file
Interested in requirements files etc? 
pip --help
output

console Usage: pip download [options] <requirement specifier> [package-index-options] ... pip download [options] -r <requirements file> [package-index-options] ... pip download [options] <vcs project url> ... pip download [options] <local project path> ... pip download [options] <archive url/path> ...

Installation part on Bianca

  • Log in to Bianca and the relevant project
  • Load the same python version
    • Ex. ml python/3.9.5
pip install --user --no-index --find-links /proj/sens2025560/nobackup/wharf/$USER/$USER-sens2025560 <package-name>==<version>
  • NOTE: you don't need the full name of the .whl file!

Test it in Python

  • Start a python console
python
  • Import the package with import <package>
  • This should give no errors!

Exercise

Install numpy-2.0.0 for Python/3.9.5

Answer
  • Using Transit and how to navigate to the porject folder is shown above.
  • Load Python: ml python/3.9.5
  • Download numpy-2.0.0: pip download numpy==2.0.0
How will that look like?

pip download numpy==2.0.0 Collecting numpy==2.0.0 Downloading numpy-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.3 MB) |████████████████████████████████| 19.3 MB 18.5 MB/s Saved ./numpy-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl Successfully downloaded numpy

  • Resulting file name: numpy-2.0.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

On Bianca:

ml python/3.9.5
pip install --user --no-index --find-links /proj/sens2025560/nobackup/wharf/$USER/$USER-sens2025560 numpy=2.0.0

Rackham users (will not work soon)

Rackham

$ ml python/<version>
$ pip download <package-name>

Transit set-up

  • Log in to transit: ssh <username>@transit.uppmax.uu.se
  • (If not done already) Mount the wharf of your project.
user@transit:~$ mount_wharf sens2025560
Mounting wharf (accessible for you only) to /home/<user>/sens2025560
<user>-sens2025560@bianca-sftp.uppmax.uu.se's password:
  • Navigate to your wharf folder
cd sens2025560
  • Download!
$ pip download <package-name>

Transfer to the wharf (only if Rackham)

sftp douglas@bianca-sftp
sftp> cd sens2017625/
sftp> dir
sftp>

If you have not uploaded anything to your wharf, this will be empty. It might have a few things in it.

Now, upload to the wharf the package <package-name> and all the dependency packages pip download got you.

sftp> put -r <package-name>

Install on Bianca

On Bianca install it (Yes, you can do it from this place) by telling pip where to look for packages and dependencies

  • In this case i should have ended up in: /proj/sens2025560/nobackup/wharf/$USER/$USER-sens2025560
$ ml python
$ pip install --user --no-index --find-links /proj/sens2025560/nobackup/wharf/$USER/$USER-sens2025560 <package-name>

Then the package ends up in ~/.local/lib/python<version>/site-packages/ .

If many files or packages

you may want to tar before copying to include all possible symbolic links:

$ tar cfz <tarfile.tar.gz> <package>

and in target directory (wharf_mnt) on Bianca:

$ tar xfz <tarfile.tar.gz> #if there is a tar file!
$ mv –a  <file(s)> ~/.local/lib/python<version>/site-packages/

Isolated/virtual environments

Tip

  • We HIGHLY recommend using a virtual environment during installation, since this makes it easier to install for different versions of Python.
  • However you can also create virtual environments on Bianca from downloaded packages, see above.

Note

Isolated environments solve a couple of problems:

  • You can install specific package, also older, versions into them.
  • You can create one for each project and no problem if the two projects require different versions.
  • You can remove the environment and create a new one, if not needed or with errors.

Example, where python packages from the loaded module are used (--system-site-packages)

$ module load python/3.6.8
$ python -m venv --system-site-packages <path>/projectB

“projectB” is the name of the virtual environment. The directory “projectB” is created in the present working directory. The -m flag makes sure that you use the libraries from the python version you are using.

  • Activate and install with pip (package one by one or from requirements.txt)
$ source <path>/projectB/bin/activate
  • Note that your prompt is changing to start with (analysis) to show that you are within an environment.
  • Install the packages from the file::
$ pip install -r requirements.txt

$ pip list   # check
$ deactivate
  • Virtual environments can be saved easily anywhere

Transfer to the wharf

sftp douglas@bianca-sftp
sftp> cd sens2017625/
sftp> dir
sftp>

If you have not uploaded anything to your wharf, this will be empty. It might have a few things in it.

sftp> put -r <path>/projectB

Move to site-packages folder On Bianca

cd /proj/sens2025560/nobackup/wharf/bjornc/bjornc-sens2025560/
mv –a  projectB <path to any place, like project folder>
(Not tested for Transit) Install on Rackham/Transit and then transfer to Bianca

Info

The package ends up on in ~/.local/lib/python<version>/site-packages/ .

  • Note that python<version> is omitting the last number (bug fix), like 3.8 for python-3.8.7.

Warning

  • If you don't have Rackham, try with Transit and tweak accoring to instruction for the download part above

Install on Rackham/Transit

$ ml python/<version>        # this is to make use the correct python version and possible dependencies already available
$ pip install --user <package-name>
  • If there is a requirements.txt file with the content of packages to be installed:
pip install --user -r requirements.txt

Then the package(s) ends up in ~/.local/lib/python<version>/site-packages/ .

Transfer to the wharf

sftp douglas@bianca-sftp
sftp> cd sens2017625/
sftp> dir
sftp>

If you have not uploaded anything to your wharf, this will be empty. It might have a few things in it.

  • Alt1: If you would like all your locally installed packages:
sftp> put -r .local/lib/python<version>/site-packages/

  • Alt 2: Just transfer the latest installed python package(s)

  • Check what was installed. It may have been several dependency packages as well. Look at the times!

sftp>  lls -lrt .local/lib/python<version>/site-packages/

sftp> put -r .local/lib/python<version>/site-packages/<package name 1>
# and if several packages
sftp> put -r .local/lib/python<version>/site-packages/<package name 2>
# and so on...

Move to site-packages folder On Bianca

cd /proj/sens2025560/nobackup/wharf/bjornc/bjornc-sens2025560/
mv –a  <file(s)> ~/.local/lib/python<version>/site-packages/

Error

If problems arise, send an email to support@uppmax.uu.se and we'll help you.