Course provider | izvajalec: Jožef Stefan Institute (JSI)
Instructors | predavatelji: Gašper Slapničar (JSI), Mitja Luštrek (JSI), Amer Mujagić (JSI)
The workshop will start with a short introduction on foundation models (FMs) for sensor data, which are large general-purpose models that can be adapted to various tasks. The most prominent example are in the fields of language (LLMs) and computer vision, but they are increasingly used for other types of sensor data.
The main topic of the workshop will be a service developed for SLAIF that facilitates fine-tuning of FMs for sensor data. The service enables handling diverse data types, configuring fine-tuning workflows, and rigorous evaluation of the resulting task-specific models. The workflow configuration includes the FM backbones, frozen layers, predictive heads, basic hyperparameter optimisation and more.
The workshop will first thoroughly explain the features of the service. It will then walk the participants through its operation on the toy example of fine-tuning a photoplethysmography FM for the task of estimating heart rate. They will be able to follow on their own laptops if desired. At the end we will discuss the interests of workshop participants to guide future training events.