Izvajalec / Course provider: Faculty of information studies in Novo mesto (FIŠ)
Predavatelji / Instructors: Robi Pritržnik, Pavle Boškoski, Miloš Ivanvić, Srdjan Škrbič, Biljana Miljeva Boshkoska
Workshop HPC in Practice IV: Parallel Processing for AI Projects
This intensive 8-hour course (delivered online, in two 4-hour sessions) looks at Multi-GPU orchestration and Distributed Data Parallel (DDP) for AI work on the HPC Trdina supercomputer. Many AI jobs are still written as a single sequence of steps, even when the machine has unused GPUs and extra nodes. Here we show how DDP lets you split demanding workloads — from deep learning training to statistical simulation — across that hardware instead of waiting on one process at a time. The course mixes the core ideas of process orchestration and inter-GPU communication with hands-on work on two concrete AI/statistics problems: Monte Carlo simulation and Variational Bayes inference.
Learning objectives: The aim is a working grasp of Multi-GPU orchestration and DDP, not just the theory. You will practise setting up and configuring a DDP environment, managing distributed tensors across GPU units, and deciding which stages of a workload actually benefit from being distributed. Hands-on exercises cover scaling Monte Carlo sampling across the cluster and applying DDP to Variational Bayes inference to accelerate optimisation over large datasets. We also look at how to tune your code afterwards to get the most out of the available GPU acceleration.
Course content: Session 1 opens with the technical configuration of Multi-GPU/DDP environments — process orchestration, communication protocols between GPU units, and managing distributed tensors on HPC infrastructure — then moves into a hands-on implementation of parallel Monte Carlo algorithms, distributing large-scale sampling across the cluster. Session 2 turns to statistical inference: applying DDP to Variational Bayes to accelerate probabilistic modelling and approximate distributions over large datasets, followed by a practical block on optimising and fine-tuning code for maximum GPU acceleration in real AI projects.
Learning outcomes: After the course, you should be able to configure and run a Multi-GPU DDP environment on an HPC cluster, apply DDP to both simulation-based (Monte Carlo) and inference-based (Variational Bayes) AI workloads, and optimise your code to make efficient use of the GPU resources available. The point is to leave with skills you can apply directly to your own research projects on HPC Trdina.
SESSION 1: Multi-GPU Orchestration and Distributed Data Parallel (DDP)
When: Monday, January 18, 2026 | 9:00 AM – 1:00 PM (including break)
Where: UNM FIS
-
9:00 – 10:30 | Technical Configuration of DDP Environments Learn the technical setup for Multi-GPU systems. We will focus on process orchestration, communication protocols between GPU units, and managing distributed tensors on high-performance infrastructure. Lecturer: Srdjan Škrbić and Biljana Mileva Boškoska
-
11:00 – 12:30 | Scaling Monte Carlo Simulations with DDP Move beyond theory by implementing parallel versions of Monte Carlo algorithms. Learn how to use DDP to distribute massive sampling tasks across the entire HPC cluster to achieve drastic speedups. Lecturer: Miloš Ivanović
SESSION 2: Advanced Statistical Inference and Code Optimization
When: Tuesday, January 19, 2026 | 9:00 AM – 1:00 PM (including break)
Where: UNM FIS
-
9:00 – 10:30 | Variational Bayes: Accelerating Probabilistic Models Apply DDP to Variational Bayes inference. We will use distributed computing to accelerate optimization steps and approximate distributions over large datasets, transforming deep learning methodologies into statistical tools. Lecturers: Pavle Boškoski and Robi Pritržnik
-
11:00 – 12:30 | GPU Optimization for Real-World AI Projects The final push: fine-tuning your code for maximum GPU acceleration. Gain the skills to convert serial processes into highly scalable distributed systems ready for your own research projects. Lecturers: Srdjan Škrbić and Miloš Ivanović
Registration is mandatory!