Parallel Python: multithreading, multiprocessing and parallel python programming for HPC

Synopsis: This course is aimed at researchers who currently want to accelerate their Python codes.

Target audience: Users of HPC, novice to intermediate Python users who want to explore multiprocessing in Python primarly for use on parallel architecture.

Follow-on courses: This course is part of the Artemis HPC Training Series. A lot of these examples come from https://github.com/csc-training/hpc-python and https://support.pawsey.org.au/documentation/display/US/Training+Material, see these sites for more in-depth courses on all sorts of cool stuff.


Prerequisites

You must have competency on the Unix/Linux command line. If you are interested in learning HPC but have no Unix/Linux command-line skills, you MUST first take an Introduction to Unix/Linux course, followed by Introduction to Artemis HPC.

You should be able to read and write basic Python scripts.

You are assumed to have your own Artemis account, know how to connect to Artemis, navigate its filesystem, and submit jobs to the PBS Scheduler. If you are not familiar with HPC, you should take Introduction to Artemis HPC before attempting this course. This will teach you the basic requirements around HPC environments.

OWN LAPTOP REQUIRED.


Course survey!

Please fill out our course survey before you leave!

Help us help you! :smiley:

Schedule

Setup Install software and data needed to begin the lesson
00:00 1. 1. Accelerating Python Who are the Sydney Informatics Hub?
How do you make your Python faster?
00:10 2. 2. Simple methods Which method for acceleration should I choose?
00:35 3. Break! Break
00:40 4. 3. Connecting to Artemis HPC How can you scale up beyond your local machine?
00:50 5. 4. Traditional python approaches to multi cpu and nodes How to utilise multiple cpus and nodes on Artemis
01:20 6. Break! Break
01:25 7. 5. Intro to Dask and Dask Dataframes Use a modern python library and elegant syntax for performance benefits
01:55 8. 6. Introducing dask Bag and more dask examples How do I deal with large irregular data and show me some real world examples of Dask
02:35 Finish

The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor.