Spring 2024¶

Dates¶

• 11 March 11:00-12:00: On-boarding
• 12 March: Python
• 13 March: Julia
• 14 March: R

Evaluation meeting 2024-03-27 at 13.30¶

• Student evaluations:

  • Python
  • Julia
  • R

• [Decided in meeting] Make the Python day last until 16:00

• [Decided in meeting] Seperate email for introduction to course and the prerequisites, most important info at the top; it’s OK to repeat info

• We conclude that the course is improving from the course ratings, all of them have gone up

• [Ri, all, suggest/vote, 5 mins] I think Python by far got the most attention by us. What about try to have seperate meetings going per day?

  • BiBjRi: Yes
• [Ri, all] I am happy to see the prereqs page and the obvious love put in it. I am unsure of the page’s purpose. Could we come up with a purpose of the page and vote? I volunteer to do the work. If we discuss: The goal of the prereqs page is …
  • BiBjRi: … that our learners can test/verify that they have the minimum skillset and have installed the minimum requirements for the course
  • [other votes]: [other suggestions]
• [Pedro, discuss] Make course material (even) more course agnostic by adding more links to other centers in course material. Bi: we should focus on one or two centers. Bi: in the near future, we may already be using Dardel, so we’ll do this automatically. Bi: for UmU, course material must gathered at the course’s site (i.e. cannot only link to UmU pages).
• [Bi, discuss] Will we use Rackham or Dardel in the fall? We will decide before summer.
• [Richel, Python & R, discuss] I think the summary of the evaluations of Python and R is ‘talk less, demonstrate slower, more time for exercises’. To me, this seems like a recurring pattern. What about we help encourage outselves to commit to fixing this? My suggestion: to the learners’ evaluation, add ‘Which section(s) scheduled enough time for exercises?’ as a means to encourage use to do better
  • [VOTED] BiBjPeRi: Yes
• [Ri, Python & R, discuss] When reflecting on the evaluation results, Most questions were useless to me, as they did not help answer my teaching questions. I cannot predict which questions are mandatory, nor which are useful to others. Result: we inherited these questions and we do use these to some extent
  • [Agreed] We should check on adding text comments, next to the ‘give a rating’ qestions
  • [Agreed] Which section(s) scheduled enough time for exercises?
  • [Is more-or-less already in, keep the original question] What should we keep doing?
  • [Is more-or-less already in, keep the original question] What should we improve?
  • [Is more-or-less already in, keep the original question] Other comments?

• [AGREED][Ri, Python] Suggest at least 30 mins of time for UPPMAX’s interactive+Jupyter: 15 mins is too short

• Fall planning

  • week 43: Tue 22 Oct -Fri 25 Oct
    • onboarding: Mon 21 Oct
    • Python: Tuesday 22nd Oct:
    • Julia: Wednesday 23rd Oct:
    • R: Thursday 24th Oct:
    • Matlab: Friday 25th Oct: Bi + Bj + Pe
• HPC Python: May 15th: leave everything the same: Bi + Bj + Pe teach, Ri assists
  • with updates of course from earlier learnings
  • py/3.11 (GPU UPPMAX perhaps other solution)

Planning meeting 2024-03-11 9:30-(max)10:30¶

• [Richel, 1 min, FYI] Richel has not gotten the link checker to work, probably does not have the time to do so
  • skip for now
• .wordlist can be updated by all of us
• [Richel, 3 mins, FYI] Richel has not tested Python packages and Python isolated environments on Kebnekaise yet, instead used the code that was present. AFAIK, I have no access to Kebnekaise
  • Birgitte has already double-checked, so we assume it to work :-)
• Should --devcore (UPPMAX only) be included in the session on batch?
  • [ ] Björn will add it :-)
• Onboarding at 11:00, we predict few people show up :-)
• [Richel, 3 mins, discuss] Time to put schedule of HackMD on Sphinx website?
  • Schedule can be assumed to be up-to-date…
  • Tip: Richel uses this website to create RST tables :-)
• [Richel, 3 mins, discuss] Evaluations before Q&A on all days?
  • [x] If yes: who does that? Björn will do
• Evaluations: Birgitte has them, will be in the Q&A document
• Parallel functions
  • [x] move to last Jl and R days
  • demo on kebnekaise
    • [x] Björn added demo box
  • exercises on batch
    • make clearer instructions how to run
      • UPPMAX: Björn
      • HPC2N: Pedro
• When parallel tracks, an Umeåner needs to start the recording in both Zoom rooms
  • First Zoom room:
    • [Zoom URL]
    • Meeting ID: 619 2226 4837
    • Passcode: 416707
  • Second Zoom room:
    • In the Q&A document: [Zoom URL]
• [x] Reschedule R to mimic Python day: Björn will do so
• [x] Reschedule Julia to mimic Python day: Björn will contact Pedro when ready
• Exercises Julia added by Pedro
  • [x] all except last exercise
  • [ ] batch exercise

Planning meeting 2024-03-06 at 13:00-14:00¶

• Status
  • [Richel, Python, technical] : python_ML_packages/3.9.5-gpu seems to works perfectly with the pytorch_fitting_gpu.py script:

module load python_ML_packages/3.9.5-gpu
git clone https://github.com/UPPMAX/R-python-julia-HPC
cd R-python-julia-HPC
pip install -r requirements.txt
cd Exercises
python pytorch_fitting_gpu.py

• Birgitte: pytorch_fitting_gpu-snowy.sh does not work. Richel: I indeed needed pip install -r requirements.txt. Hence, the metapackage does not work correctly, as you need to install pytorch using pip
• Birgitte: seaborn-code.py in the course content does not work

• comment(BC):

  • Seems we have to stick with 3.9.5, since installation of 3.11.8 has not succeeded yet
  • If 3.11.8 is installed, use that one. Else, use 3.9.5.
  • Decision: Thursday March 7 12:00

• Discussion

  • [Richel, Python, isolated environments] The schedule states **10.45**|Isolated environments (ML, venv, **conda**)|Richel (45-->60), with a stress on conda. Sharing the new teaching goals for isolated environments (below), however, showed the need for venv again. What need the teaching goals be?

        - learners have created, activated, used and deactivated a conda virtual environment
        - learners have used an ML package
        - learners have heard about venv

or back to the earlier: YES

        - learners have created, activated, used and deactivated a venv
        - learners have used an ML package
        - learners have heard about conda

Conclusion: use the good old venv, Richel misunderstood :-)

• Björn moved some content from index page to prereqs. https://uppmax.github.io/R-python-julia-HPC/prereqs.html

  • the links for improving programming skills
  • most of the cluster information
  • Comments?
    • Add using 2FA on the ThinLinc local client /Björn
  • To discuss:
    • Is it OK to take a quick explanation of the most important things
      • login & compute nodes and cores
      • similarities and differences between Kebnekaise and UPPMAX clusters
    • DONE
  • Richel: teacher is boss :-) . The UPPMAX docs are tested and nice and used by other courses: prefer working on one source of content
    • SSH X forwardinghttp://docs.uppmax.uu.se/software/ssh_x_forwarding/
      • DONE
    • Björn adds XMING in prereqs
      • DONE
    • Refer to UPPMAX doc Remote desktop via webbrowser, including using 2FA
      • DONE
    • Text editors: http://docs.uppmax.uu.se/software/text_editors/
      • DONE
    • Linux: http://docs.uppmax.uu.se/getting_started/linux/
      • DONE

• [Richel, 5 mins] Suggest to conformize folder names at https://github.com/UPPMAX/R-python-julia-HPC: Exercises, R, docs, summaryPython.rst, packages_suggest.rst, sequential-python.sh

  • Richel’s suggestion: all_lower_snake_case_with_underscores, Richel volunteers to do so
  • Birgitte: this year folders only, next time I’ll promise to rename the files :-) .
    • exercises, folder_with_more_names
    • [ ] Richel does it today before 23:59
    • [ ] Richel checks link checker

• Birgitte: some UPPMAX questions. These were already answered :-)

• Type-alongs
  • Björn added .. type-along:: are the tags for type-along blocks.
  • Björn can continue for R
  • Instructors will add this to their content if it is used in Python
• Next events:
  • Onboarding: Monday March 11th 11:00-(max)12:00
  • Planning meeting: Monday March 11th 9:30-(max)10:30

Planning meeting 2024-02-29 11:00-12:00¶

• Richel will be absent

• Björn and Richel thought that they had a nearly-mandatory event in the afternoon at the R day on March 14th. However, it was in the morning. Vote:

  • Björn and Richel teach in the afternoon of the R day, Birgitte and Pedro teach in the morning:
  • Björn and Richel cancel the nearly-mandatory event and teach in the morning: We do this
  • Move the R day to another date:
  • Latest info: the nearly-mandatory event may be prostponed for other reasons

• Status:

  • Problems with ML_packages installation

• ToDos

  • Richel
    • check 3.11 compatibility with exercises at UPPMAX
    • will go through https://uppmax.github.io/R-python-julia-HPC/python/isolated.html and check if it work with python/3.11.8 combined with ML_packages/3.11.8-GPU Tuesday 5th March
  • Björn:
    • type-along boxes
      • julia done?
    • index page strip
    • Prereqs more info
    • Julia parts updates
    • quiz

Planning meeting 2024-02-22 11:00–12:00¶

• Status

  • Diana is installing ML_packages/3.11.8-GPU today
    • theano problem??
    • Theano is not compatible with modern numpy 1.25.x (as we have with Python 3.1
    • pytensor is the continuation
    • but use first in HPC-python course
    • pytensor will be installed at Rackham

• ToDos

  • Björn:
    • check 3.11 compatibility with exercises at UPPMAX
    • type-along boxes
    • Prereqs more info

Schedule for the spring 2024 version¶

Python (Tuesday 2024-03-12)¶

Julia (Wednesday 2024-03-13)¶

R (Thursday 2024-03-14)¶

Planning meeting 2024-02-15 11:00–12:00¶

• How many registered
  • 55

• Status

  • BC: started prereq page to be filled

• Discuss

  • Skip bianca parallel session for all days. Vote yes!
  • No intro at 9.00 for slurm

• ToDos:

  • Björn:
    • type-along boxes
    • Py-ml UPPMAX
    • Prereqs more info
  • Pedro: several docs for parallel

Planning meeting 2024-02-09 11:00–12:00¶

• Discuss:
  • depth of parallelism/ML for python
    • same level as for julia and R?
    • Yes, show an ML example
  • We use R 4.1.x! R versions 4.3.1 + 4.3. (Birgitte checks the packages)
    • HPC2N has R/4.3.2 but a lot fewer packages than for R/4.0.4. I have inquired about getting some packages installed as modules
    • 4.3.2 is lacking many packages, including the Bioconductor bundle which is not yet available for installation.
    • 4.1.x chosen instead.
    • UPPMAX: 4.1.1
  • python_ML new version would be nice [name=Björn] checks
    • wish: 3.11.(3,8)
    • At HPC2N, Python 3.11.3 is installed. All relevant packages except Theano and ASE are available for that version of Python. We can probably ask for those packages if we need them
    • At UPPMAX: 3.11.8
  • julia 1.8.5
  • [R, suggest vote] follow pace of schedule or follow pace of learners?
    • [VOTED FOR] Pace of schedule with flexibility of 15 mins in both directions
      • pro:
        • easy to schedule who teaches what
        • comfortable to teach
        • higher evaluation scores
        • clear for learners when takes place what
      • con:
        • can mismatch with learners
      • votes: Birgitte, Björn, Pedro
    • Pace of learners
      • pro:
        • the right speed
        • needs procedure, e.g. https://richelbilderbeek.github.io/awk_course/misc/exercise_procedure/
      • con:
        • harder to schedule
        • not everyone comfortable with teaching like that
        • setting learners to work will result in lower evaluation scores
      • votes: Richel
  • [R, suggest vote] evaluation yes/no?
    • [VOTED] Yes (pro: we get feedback, con: can be unpleasant): Birgitte, Björn, Pedro, Richel
    • No (pro: not unpleasant, con: no feedback to improve on): -
  • [R, suggest vote] order: evaluation-then-Q&A or Q&A-then-evaluation?
    • [VOTED] evaluation-then-Q&A (pro: learners can leave at Q&A): Birgitte, Björn, Pedro, Richel
    • Q&A-then-evaluation: -
  • [R, suggest vote] publish evaluation results online yes/no?
    • [VOTED] Yes (pro: shows transparency (e.g. https://github.com/richelbilderbeek/teaching), con: not everyone can rightly value an evaluation): Birgitte, Björn, Pedro, Richel
    • No (pro: no trouble, con: links and info get lost):
• Menti responsible (the person needs to nbe available all day)
  • Python day: no polls, so no person needed
  • Julia day: Björn
  • R day: Birgitte
• [False positive] Although https://www.uppmax.uu.se/support/courses-and-workshops/r-python-julia links to https://uppmax.github.io/uppmax_intro_python/ it does not matter: it is not the course material :-)

Planning meeting 2024-01-31¶

• [Vote] Use Zoom Poll or Menti or text in shared document: will use Menti. Birgitte will prepare
  • Zoom Poll: only host
    • Votes: Birgitte, Pedro
  • Menti: everyone more-or-less needs separate account
    • Votes: Richel, Björn, Birgitte
  • Plain text (in a shared document)
    • Votes: Richel, Pedro, (Birgitte)
  • MENTI WON!
• [Richel, FYI] Björn and/or Richel can teach R at March 14th in the morning, but not in the afternoon.
  • We rescheduled so that Uppsala people teach in the morning and the Umeå people in the afternoon
• [Richel, vote] some UPPMAX courses are restructured by referring to the UPPMAX documentation pages. In that way, content can be found in exactly one place. An example can be found here. As this course also teaches Kebnekaise, referring to its documentation would work too. Richel volunteers to do so for the Python course (as he taught another similar Python course too). Good idea yes/no?
  • Important with new tab. Try this time!
  • VOTE:
    • for: Richel, Björn, Birgitte, (Pedro)
    • against: (Pedro)
• Evaluations
  • Both Julia & R: Type-along clearer (boxes) and talk about it!
  • good level
  • more exercises
  • more about ThinLinc (technical prereqs can be expanded with more info and background and pros and cons)

Schedule¶

• slurm is integrated with interactive/batch each day (take a break!)

R (Thursday 2024-03-14)¶

• Teachers
  • Python (Tue)
    • Richèl, same topics as previous time: Packages, Isolated environments, Rackham: Jupyter, Bianca: Conda, interactive
    • Birgitte, same topics as previous time
  • Julia (Wed)
    • Björn (same as last time)
    • Pedro
  • R (Thu)
    • Richel can teach morning only
    • Björn teach first hour (afternoon if needed) or standby

Planning meeting (Jan 24 at 11)¶

• Discuss

  • menti

  • Zoom can also do something similar called a Poll, which is ideal for quicker things. It does need to be shared :-)

    • 

  • We will vote between using Zoom Poll and Menti next week Wednesday 31 Jan at 13:00, Birgitte will prepare

  • We used Python version 3.9.5, because that worked well with ML. We try to go to a newer version of Python, if that has made to works well with ML too.

• Evaluations

  • python: done, see results here
  • Julia
  • R
  • General
    • cheat sheets
    • short intros
    • slurm first 30 minutes everyday or so
    • think about extending

• Teachers

  • Python (Tue)
    • Richèl, same topics as previous time: Packages, Isolated environments, Rackham: Jupyter, Bianca: Conda, interactive
    • Birgitte, same topics as previous time
  • Julia (Wed)
    • Björn (same as last time)
    • Pedro
  • R (Thu)
    • Richel can teach morning only
    • Björn teach first hour (afternoon if needed) or standby

• Needed updates

  • Py module

    • UPPMAX update ML

  • R module

    • newer?
  • Don’t go too deep. Examples:
    • determining number of cores
    • ML and multiple GPUs?