File transfer using rsync¶
Learning outcomes
- Practice using the documentation of your favorite HPC cluster
- Can transfer files using
rsync
For teachers
Teaching goals are:
- Learners have practiced using the documentation of their favorite HPC cluster
- Learners have transferred files using
rsync
Prior questions:
- How can one do a file transfer from the terminal?
- Can you name some programs that can do file transfer from the terminal?
- Who has heard of
rsync? - Who has already used
rsync?
Notes:
- Bianca users will have the most complex procedure by far
Feedback:
- When would you recommend a colleague to use
rsync, and when not?
Why?¶
Fairly sure you'll want to upload or download files to your favorite HPC cluster. Here we do so.
We use a free and open-source command-line tool to do this,
called rsync.
It works under Linux, Mac and Windows.
Exercises¶
Need a video?
| HPC Cluster | YouTube video |
|---|---|
| Alvis | YouTube video |
| Berzelius | Cannot: is not a NAISS cluster |
| Bianca | YouTube video |
| COSMOS | YouTube video |
| Dardel | YouTube video |
| Kebnekaise | Cannot: is not a NAISS cluster |
| LUMI | YouTube video |
| Pelle | YouTube video |
| Rackham | YouTube video |
| Tetralith | YouTube video |
| Vera | Cannot: is not a NAISS cluster |
How difficult will this be?
This depends on:
- operating system
- your HPC cluster
Per operating system, it differs how easy it is to install rsync:
| Operating system | Difficulty |
|---|---|
| Linux | Easy |
| Mac | Unsure |
| Windows | Unsure |
| HPC Cluster | Difficulty | Reason |
|---|---|---|
| Alvis | Medium | No specialized documentation |
| Berzelius | Easy | Has specialized documentation |
| Bianca | Hard | Has specialized documentation, complex procedure |
| COSMOS | Medium | No specialized documentation |
| Dardel | Medium | No specialized documentation |
| Kebnekaise | Easy | Has specialized documentation |
| LUMI | Medium | CSC refuses to document this, but UPPMAX has specialized documentation |
| Pelle | Easy | Has specialized documentation |
| Rackham | Easy | Has specialized documentation |
| Tetralith | Easy | Has specialized documentation |
| Vera | Easy | Has specialized documentation |
Exercise 1: transfer a file¶
Use the documentation of your HPC cluster.
- For maximally 5 minutes, search for how to transfer files
to/from your HPC cluster
using
rsync. Which URL is it described? Take a look at the answer if you cannot find it: sometimes there is no documentation
Where is that documentation?
| HPC Cluster | Documentation |
|---|---|
| Alvis | Documentation that is closest. For now, use the UPPMAX documentation |
| Berzelius | Documentation |
| Bianca | Documentation |
| COSMOS | None. Documentation that is closest. Use the scp documentation and replace scp by rsync |
| Dardel | Documentation that is closest |
| Kebnekaise | Documentation |
| LUMI | CSC refuses to document this, but UPPMAX has specialized documentation |
| Pelle | Documentation |
| Rackham | Documentation |
| Sigma | Documentation that is closest. For now, use the UPPMAX documentation |
| Tetralith | Documentation |
| Vera | Documentation that is closest. For now, use the UPPMAX documentation |
If the center maintaining you HPC cluster has not documented how to use
rsync,
follow the Rackham documentation.
- Copy a file from your local computer to your HPC cluster's home folder. Verify that this worked.
- Copy a file from your HPC cluster's home folder to your local computer. Verify that this worked.