Lab 00
Your teaching assistants for the labs and the authors of these notes.
Teaching assistant · Labs
Hi! I'm Elitsa. I'm interested in the robustness of LLM judges, evaluation, and the broader questions of frontier AI safety.
At Sofia University I work with Dr. Dimitar Dimitrov, Prof. Ivan Koychev and Prof. Preslav Nakov (MBZUAI). Previously a software engineering intern at Google (Zurich) and Uber (Amsterdam & Sofia).
I also collaborate with Violeta Kastreva (ETH Zurich) on recent research.
Teaching assistant · Labs
I recently interned at Amazon Web Services (Berlin), and lately I can't stop thinking about multi-agent systems and NLP: how far you can push a handful of cooperating models before things get interesting.
Outside of that, one of my favorite things is working with students: I'm a teaching assistant at Sofia University, and watching a concept finally click is hard to beat.
Fifteen labs in three parts: search, machine learning and language models.
| Lab | At a glance | What it is about |
|---|---|---|
| Part 1 · Search and reasoning | ||
| 01Uninformed Search | State spaces and blind search: BFS, DFS, uniform-cost and iterative deepening. | |
| 02Informed Search | Heuristics guide the search: greedy best-first, A* and its variants. | |
| 03Constraint Satisfaction | Variables, domains and constraints: backtracking, AC-3 and min-conflicts. | |
| 04Genetic Algorithms | Evolution as search: selection, crossover, mutation and a population of candidates. | |
| 05Games | Adversarial search: game trees, minimax and alpha-beta pruning. | |
| Part 2 · Machine learning | ||
| 06Introduction to ML | Learning from data: regression vs classification, training vs test, bias and variance. | |
| 6.5Linear & Logistic Regression | Fitting a line with gradient descent, then turning it into probabilities with the sigmoid. | |
| 07K-Nearest Neighbours | Classify a point by a vote of its closest neighbours: distances, scaling and choosing K. | |
| 08Naive Bayes Classifier | Bayes' theorem with an independence assumption: spam filtering and Laplace smoothing. | |
| 09Decision Trees | Split the data with yes/no questions: entropy, information gain, ID3 and pruning. | |
| 10K-Means and Clustering | Finding groups without labels: K-means, K-means++, silhouette and hierarchical clustering. | |
| 11Neural Networks | From one neuron to multilayer networks: activations, gradient descent and back-propagation. | |
| Part 3 · Language models | ||
| 12Transformers and Generative Models | How LLMs work: tokens and embeddings, self-attention, and generating text one token at a time. | |
| 13Post-training and Alignment | From a pretrained model to an assistant: fine-tuning, reward models, RLHF and DPO. | |
| 14Evaluation, Metrics and Data Analysis | Measuring what matters: precision and recall, text metrics, LLM judges, and reading plots critically. | |
SemEval is the yearly international workshop on semantic evaluation: organisers publish a task with data and a metric, teams build systems, and all systems are scored on the same hidden test set.
We are among the organisers of Task 9: Dr. Dimitar Dimitrov, Elitsa Yotkova and Prof. Ivan Koychev, together with Giovanni Da San Martino, Preslav Nakov, Jakub Piskorski and colleagues from Padova, MBZUAI, Helsinki, Porto, Beira Interior and ETH Zurich. It builds on the SemEval-2023 task on persuasion techniques, framing and news genre.
Framing is choosing which aspects of an issue to make salient, so that a reader takes away a particular meaning. Images can reinforce, complement or contradict the framing of the text.
In AI, the main results are published first at conferences, not in journals. Two groups matter most for this course.
Machine learning
Natural language processing · ACL*
36 auto-graded questions, 100 points, no maths: check your AI vocabulary before the course. Abbreviations, embeddings, pretraining vs. post-training, prompting, RAG, tools and agents, hallucinations, MoE, quantization and distillation.