Name: Workshop: Scaling CUDA C++ Applications to Multiple Nodes
Start: 2024-09-13T09:00:00+02:00
End: 2024-09-13T17:00:00+02:00
Location: MS Teams

Workshop: Scaling CUDA C++ Applications to Multiple Nodes

Friday, 13 September 2024 - 09:00

Monday, 9 September 2024
Tuesday, 10 September 2024
Wednesday, 11 September 2024
Thursday, 12 September 2024
Friday, 13 September 2024

09:00 Introduction (- Meet the instructor. - Get familiar with your GPU-accelerated interactive JupyterLab environment.)

09:00 - 09:30

09:30 Multi-GPU Programming Paradigms: (- Survey multiple techniques for programming CUDA C++ applications for multiple GPUs using a Monte Carlo approximation of a CUDA C++ program: – Use CUDA to utilize multiple GPUs. – Learn how to enable and use direct peer-to-peer memory communication. – Write an SPMD version with CUDA-aware MPI.)

09:30 - 11:30

11:30 Lunch break

11:30 - 12:30

12:30 Introduction to NVSHMEM (Learn how to write code with NVSHMEM and understand its symmetric memory model: – Use NVSHMEM to write SPMD code for multiple GPUs. – Utilize symmetric memory to let all GPUs access data on other GPUs. – Make GPU-initiated memory transfers.)

12:30 - 14:30

14:30 Coffee break

14:30 - 14:45

14:45 Halo Exchanges with NVSHMEM: (Practice common coding motifs like halo exchanges and domain decomposition using NVSHMEM, and work on the assessment: – Write an NVSHMEM implementation of a Laplace equation Jacobi solver. – Refactor a single GPU 1D wave equation solver with NVSHMEM.)

14:45 - 16:30

16:30 Final Review (– Complete the assessment and earn a certificate. – Review key learnings and wrap up questions. – Learn about application tradeoffs on GPU clusters. – Take the workshop survey.) Conveners: Domen Verber, Jani Dugonik

16:30 - 17:00

Choose timezone