Intro to GPU computing
Overview
Teaching: 5 min
Exercises: 2 minQuestions
What is a GPU?
How do I get access to a GPU?
GPU or CPU?
How do I develop code for GPU computing?
Objectives
Understand why GPUs are cool
Setting up a development environment
This episode explains how to setup a working GPU in a MS Windows environment. You will find to get a well-functioning GPU development environment, there is a precious balance between your operating system (Windows 10 in this case), your model of GPU (NIVIDA in our case), your GPU driver, your CPU/GPU programming language/version (C++/CUDA in this case) and your IDE (Visual Studio 2017 in this training session). Additionally there are often many libraries that must work together including tensorflow, numpy, keras, etc.
This talk is structured for no experience in the categories below. Depending on the participants level of experience in each category, the trainer can tailor the training session to suit the audience’s needs.
Survey of participants to tailor session:
- Familiarity with GPU (1 to 5)
- 1-No Experience
- 3-Worked on several projects but only one type of GPU
- 5-Worked with several types GPUs (OpenCL, CUDA) and remotely (HPC, AWS, Argus, etc)
- PC experience (1 to 5)
- 1-use Mac/Linux
- 3-Some PC use,
- 5-Mainly PC user, on a Win10 machine now
- rdp experience (1 to 5)
- 1-Use local machines
- 3-Use ssh and HPC for a few projects
- 5-Frequent rdp, AWS, gotomypc User
- Visual Studio (VS) experience (1 to 5)
- 1-No experience VS
- 3-Experience with 1 version (e.g. VS 2017 Community)
- 5-Lots of VS Experience incl 2015/2017/2019, Comm/Prof/Enter
- CUDA/C/C++ experience (1 to 5)
- 1-Mainly interpreted languages (i.e. R/Python),
- 3-1 compiled language (e.g. C/C++/CUDA/Fortran/Pascal/Visual Basic/Java)
- 5-Several compiled languages incl C/C++/CUDA
For example: a quick response on chat could be “GPU-1, PC-5, rdp-3, VS-1, C-3”. Have a think and respond via chat if on Zoom.
What is a GPU?
A Graphics Processing Unit (GPU) as opposed to a Central Processing Unit (CPU). Originally intended for sending an image to the pixels on the screen, someone then figured out that by treating data as a texture map you could perform lots of calculations simultaneously, and the age of the GPGPU (general-purpose graphics processing unit) began.
How do I get access to a GPU?
Your PC!
Sydney University HPC Artemis
Sydney University Virtual Research Desktop Argus (In this training session we will be using an Argus Windows 10 machine.)
Other clouds like GCP, AWS, Azure, NGC, etc
Why do I want to use a GPU?
We can visualise the CPU and GPU as something like this:
| CPU strengths | GPU strengths |
|---|---|
| Very large main memory | High bandwidth main memory |
| Very fast clock speeds | Latency tolerant via parallelism |
| Latency optimized via large caches | Significantly more compute resources |
| Small number of threads can run very quickly | High throughput |
| High performance/watt |
| CPU weaknesses | GPU weaknesses |
|---|---|
| Relatively low memory bandwidth | Relatively low memory capacity |
| Low performance/watt | Low per-thread performance |
Some keywords
A thread is like a single computational task or instruction or kernel that is run on the GPU (or CPU).
A thread block is a group of threads in the same location on the GPU so they can communicate easily with each other.
A grid is the collection of thread blocks.
When executing your kernel (i.e. function), the typical CUDA command looks like this:
//The numbers possible here are dependent on your GPU hardware.
YourKernel<<<NumberOfBlocks,NumberOfThreadsPerBlock>>>(...)
Use device query (or gpuQuerey in Matlab) to see some details about your GPU.
Having proposed GPU (or CPU) compute time on grant applications can show you are conducting accelerated research! Get in touch with us for help estimating your resource usage, or for any assitance, https://informatics.sydney.edu.au/
How do you develop code for GPU computing?
The most common languages for developing code for GPUs are CUDA, OpenCL, and OpenACC. These are all low-level and require fairly strong programming capabilities. However, high-level languages like Python and Matlab and subsequent packages within them (keras, tensorflow, theano, etc), can make use of your GPU by essentially writing CUDA (or OpenCL) for you!
This training session will focus on a MS Windows environment which predominately has NVIDIA GPUs and also uses Visual Studio to develop C/C++/CUDA software. OpenCL is the other standard and works on the other most popular GPU card, AMD Radeons (Mac GPU of choice) and will not be discussed in this training session.
Key Points
GPU vs CPU
Suitable problems for GPU
CUDA, and high level GPU environments
Where to get one