Workshop: Scaling CUDA C++ Applications to Multiple Nodes
Friday, 13 September 2024 -
09:00
Monday, 9 September 2024
Tuesday, 10 September 2024
Wednesday, 11 September 2024
Thursday, 12 September 2024
Friday, 13 September 2024
09:00
Introduction (- Meet the instructor. - Get familiar with your GPU-accelerated interactive JupyterLab environment.)
09:00 - 09:30
09:30
Multi-GPU Programming Paradigms: (- Survey multiple techniques for programming CUDA C++ applications for multiple GPUs using a Monte Carlo approximation of a CUDA C++ program: – Use CUDA to utilize multiple GPUs. – Learn how to enable and use direct peer-to-peer memory communication. – Write an SPMD version with CUDA-aware MPI.)
09:30 - 11:30
11:30
Lunch break
11:30 - 12:30
12:30
Introduction to NVSHMEM (Learn how to write code with NVSHMEM and understand its symmetric memory model: – Use NVSHMEM to write SPMD code for multiple GPUs. – Utilize symmetric memory to let all GPUs access data on other GPUs. – Make GPU-initiated memory transfers.)
12:30 - 14:30
14:30
Coffee break
14:30 - 14:45
14:45
Halo Exchanges with NVSHMEM: (Practice common coding motifs like halo exchanges and domain decomposition using NVSHMEM, and work on the assessment: – Write an NVSHMEM implementation of a Laplace equation Jacobi solver. – Refactor a single GPU 1D wave equation solver with NVSHMEM.)
14:45 - 16:30
16:30
Final Review (– Complete the assessment and earn a certificate. – Review key learnings and wrap up questions. – Learn about application tradeoffs on GPU clusters. – Take the workshop survey.) Conveners: Domen Verber, Jani Dugonik
16:30 - 17:00