Poravnava in prilagajanje velikih jezikovnih modelov po meri uporabnika | Alignment and adaptation of large language models to user needs

Izvajalec / Course provider: Jožef Stefan Institute (JSI)

Predavatelji / Instructors: Matej Martinc (JSI), Boshko Koloski (JSI)

Learning objectives: Gain practical knowledge of Large Language Model (LLM) alignment using reinforcement learning methods to ensure safety, reduce bias, and follow complex user instructions.

Course content:

For the successful implementation of LLMs in business and scientific environments, models require more than just knowledge; they need alignment with ethical standards and specific user preferences. This practical workshop will cover:

• Fundamentals of Alignment: Why simple next-token prediction is not enough and how to guide models toward helpfulness, honesty, and harmlessness.
• Reinforcement Learning from Human Feedback (RLHF): An overview of algorithms (e.g., PPO, DPO, GRPO) for fine-tuning models (such as Gemma and GaMS).
• Debiasing: Techniques for identifying and reducing stereotypes and biases in generated responses.
• Adaptation to user needs: How to use the alignment process to ensure a model adheres to the specific tone, style, or professional constraints of a particular organization.

Learning outcomes:

• Understanding the process of model alignment using reinforcement learning.
• The ability to implement techniques to reduce bias in model responses.
• Practical knowledge of adapting model behavior to the specific requirements of a client or research project.