Lab 12 · Quiz
Transformers Quiz
Tokenization
Self-attention
Transformer block
Next-token prediction
Work in progress This quiz is still being reviewed: questions may change, and some parts may still contain mistakes.
Before you start
Quiz for Lab 12: Transformers and Generative Models. Answer every task, then press Check answers at the bottom: each answer is marked, the explanations appear, and you get a score per skill.
Assumptions unless stated otherwise
- Token IDs are identifiers, not meaningful numerical quantities.
- The attention weights in one row sum to 1.
- In a causal language model, a position may attend only to itself and earlier positions.
- Higher logits give higher softmax probabilities.
How answers are scored
- Numbers: type the value as a decimal, e.g. 0.58; large numbers may use separators (38,400,000).
- Orderings and token splits: click the buttons in order (⌫ Undo removes the last one); only an exact match scores.
- Select all that apply: scores only when the selection is exactly right.
- Rows: one choice per row; each correct row earns its share of the points.
- Your answers are saved in this browser only, so you can leave and come back. Nothing is sent anywhere.