Workshop: Introduction to High-Throughput-Computing: SnakeMake

Excerpt: In this workshop we will learn the basics of SnakeMake, how to translate a basic workflow, which is partly trivially parallelizable and in parts not, into SnakeMake, and how to utilize HPC resources in conjunction with SnakeMake. We will take a look at a generic workflow, consisting of a set of tasks, with varying interdependencies; we will learn how to identify these interdependencies between our tasks and how to design a workflow diagram around them.

Description: We will learn the basics of SnakeMake, which is an open-source tool that allows users to describe complex workflows with a hybrid of Python and shell scripting. By transferring our workflow diagram into a SnakeMake we will learn how to automate our workflow.

Initial development of our workflow in SnakeMake will be in a terminal setup, as we do generically in our day-to-day lives, however we will then learn how to scale up and perform these tasks on a local supercomputing cluster.

Difficulty: Basic

Language: English

Prerequisite knowledge: Basic knowledge of Linux, the Terminal, and Python

Target audience: Beginners and more experienced people interested in optimizing and automating their workflows and ultimately transferring them to High-Performance Computing machines.

Skills to be gained:

• workflow design, identification of interdependencies
• basics of SnakeMake
• transfering a generic workflow into SnakeMake
• testing our SnakeMake in an interactive environment, and upscaling it for use on a supercomputing cluster

Maximum number of participants: 30

Workflow: The training is live, in the afternoon. Part of the training will be in the form of a lecture and part will be a practical session, where we will make use of the NSC Cluster at the Jožef Stefan Institute (as an educational member of SLING).

Location: Faculty of Mathematics and Physics (predavalnica P.02) Jadranska 21, 1000 Ljubljana, Slovenia

Organizer:

Lecturers: