Lab 08
Neural Text Representations: Tokens, Embeddings, Self-Attention and Transformers
From TF-IDF vectors to learned vectors
Sparse · lexical
A TF-IDF vector
Each dimension corresponds to a vocabulary term. Most entries are 0: only the terms in the text get a weight.
Dense · learned
A learned dense vector
Most dimensions are non-zero, and they do not correspond to individual words: the representation is learned from data.
Which raises the important question: where do these numbers actually come from? The rest of the lab answers it.
Sparse vs dense
| Property | Sparse lexical vector | Learned dense vector |
|---|---|---|
| Dimensions | vocabulary terms | learned features |
| Typical size | tens of thousands | hundreds to a few thousand (here 384) |
| Mostly zeros | yes | no |
| Easy to interpret directly | relatively | usually not |
| Handles exact terms well | yes | variable |
| Can capture semantic similarity | limited | often better |
❗ A warning. Do not say "dense vector = meaning". A dense vector is a learned representation, and its usefulness depends on the training data, the training objective, the model architecture and the domain.
Tokens and token embeddings
Neural models do not consume raw strings directly; they consume tokens. A simple tokenizer turns retrieval systems are useful into ["retrieval", "systems", "are", "useful"]. Modern language models often use subword tokens instead: a rarer word is split into pieces (conceptually retrieval → retriev + al). The exact split depends on the tokenizer.
Try it: split a text into tokens
Type a text, or pick an example. Common words stay whole; rarer words are split into pieces. A dashed chip continues the previous word.
💡 retrieval is common enough to be a single token, while embeddings is split into em + bed + ding + s. A tokenizer has a fixed vocabulary of words and word pieces; any text is written with them.
Tokens become IDs, IDs look up vectors
The tokenizer maps each token to an integer ID (for example retrieval → 26384). The model does not treat the integer itself as meaning: the integer is used to look up a vector. Each token ID maps to a learned vector, its token embedding:
See it: the lookup table
One row per vocabulary entry (tens of thousands of rows), and each row is a learned vector of hundreds of numbers; only the first few are shown. Click a token.
A token embedding is not yet contextual
The word bank has many meanings, but the tokenizer only sees the string bank. Every sentence below gets the same token ID, so it looks up the same row of the table. Click a picture.
Four different meanings, one identical starting vector. But the meaning depends on context. We need a mechanism that lets the surrounding tokens change the representation: that is the role of the Transformer.
Self-attention
What is self-attention?
Read this sentence: The animal did not cross the street because it was tired. Who is it? You know immediately that it is the animal, because you looked back at the other words. A neural model needs a way to do the same.
- Self-attention is that mechanism: every word looks at all the other words of the same text and decides how much each one matters to it.
- Each word updates its vector by mixing in information from the words it paid attention to.
See it: where does it look?
The shading shows how much attention it gives to each word (the weights add up to 100%). Switch the last word and watch the attention move. Illustrative numbers, chosen to show the idea.
Why do we need it?
After the lookup table (Part II) every token has a vector, but that vector knows nothing about the rest of the text. Language is full of words that only make sense in context:
- Pronouns: it means the animal in one sentence and the street in another.
- Ambiguous words: bank starts from the same row in river bank and bank account.
- Small words that change everything: did not cross means the opposite of did cross.
The purpose of self-attention: turn context-free token vectors into context-aware ones.
How it works: Query, Key, Value
Think of self-attention as retrieval inside the sentence. Each token produces three vectors:
Query (Q)
“What information am I looking for?”
Key (K)
“What kind of information do I contain / advertise?”
Value (V)
“What information should I actually give you if you attend to me?”
Query asks. Key matches. Value answers.
A real-world analogy: a library
Imagine you enter a library and ask: “I need a book about neural networks.” Your request is the query. Every book has metadata, its key; the content you read is its value.
Book A
keycooking, recipes, Italian
no match
Book B
keyneural networks, deep learning, AI
strong match → open it
valuethe book's actual contents
Book C
keyhistory, Rome, empire
no match
- Query = what you are searching for.
- Key = how each item describes itself for matching.
- Value = the actual information you retrieve.
Where do the query, key and value come from?
Suppose the input token representations (the rows from the lookup table) are \(x_1, x_2, \dots, x_n\), stacked as the rows of \(X\). The Transformer learns three matrices \(W_Q, W_K, W_V\) and computes:
So for one token, say animal, its vector \(x\) goes through all three:
In self-attention, Q, K and V all come from the same sequence of token representations, but they are projected differently.
- Learned and shared: the three matrices are learned in training, and every word uses the same ones.
- Smaller: each matrix maps the long vector to a much shorter one, for example 768 numbers → 64. Comparing short vectors is cheaper, and the model runs several of these small attentions side by side, each with its own three matrices; their outputs are joined back to the full size (see multi-head attention below).
The same steps with small numbers
For one word:
- 1. Score: compare its query with the key of every word (a dot product). A high score means a good match.
See it: the score is a dot product
The query of it is \(q = [1, 2]\). Pick a word to compare its key with the query: multiply the numbers position by position, then add. The more the key points in the same direction as the query, the higher the score.
- 2. Weights: turn the scores into weights that add up to 1 (softmax): the better the match, the bigger the share of attention.
See it: softmax turns scores into weights
Softmax takes \(e^{\text{score}}\) for every word and divides by the total, so the weights are positive and add up to 1. Move the sliders. Try giving one word a much higher score, or raising all four scores by the same amount.
- 3. Mix: its new vector is the weighted sum of all the values.
Try it: the three steps for it
Four words, tiny 2-number vectors (invented for the example; real ones have hundreds of numbers). Step through with ▶.
First, \(QK^\top\) answers: where should I look? Then \(\operatorname{softmax}\!\big(QK^\top / \sqrt{d_k}\big)\,V\) answers: what information should I retrieve from those locations? The full scaled dot-product attention:
Dividing by \(\sqrt{d_k}\) (the length of the key vectors) only keeps the scores from growing too large. You do not need to derive this; understand the steps.
❗ Important caveat. Descriptions such as “animate”, “noun” or “pronoun looking for an antecedent” are conceptual explanations. In a real trained Transformer, individual dimensions of Q, K and V usually do not correspond cleanly to human-interpretable concepts: the information is stored in distributed learned representations. So do not think “dimension 3 = noun, dimension 7 = animate”; think of the vectors as learned numbers whose geometry helps the network perform useful matching and information transfer.
Multi-head attention
One attention operation, with its own \(W_Q, W_K, W_V\), is called a head. Multi-head attention runs several heads in parallel and joins their outputs back together. Conceptually, different heads can learn different kinds of relationships. For example, different heads might become sensitive to:
This is only an intuitive picture: heads are not guaranteed to correspond neatly to one linguistic relationship.
Feed-forward networks
A feed-forward network is the most basic kind of neural network: numbers go in, pass through a layer of simple units, and new numbers come out. Information only flows forward, from input to output, which gives it its name.
On their own, feed-forward networks are used for many everyday prediction tasks: deciding whether an email is spam, predicting a house price from its features, recognizing handwritten digits. Here we use one to turn each token's vector into a more useful one.
What does one unit do?
- Weighted sum: it looks at all the input numbers, multiplies each one by a learned weight and adds them up, plus one more learned number, the bias. The result says how strongly a certain pattern is present.
- On or off: then a simple rule called ReLU: a negative result becomes 0 (the pattern is absent), a positive one is kept.
- Many units: a layer has many units, each checking a different pattern; a second step combines their answers into the output numbers.
Try it: one unit
Three numbers from the vector of it after self-attention, and one unit that checks for “a living thing that is tired”. Move the sliders. The labels and numbers are invented to show the idea: real units are not this easy to name.
What information does it give us?
- Combinations: a unit can detect a combination, such as “living thing” and “tired”, that no single input number shows on its own.
- A more useful vector: the output is a new vector in which these detected patterns are written more clearly.
- Learned, not written: nobody chooses the patterns; the weights are learned from data.
- One token at a time: the network works on each token's vector separately. Tokens do not exchange information here; that is what self-attention does.
What patterns do real units learn?
Our “tired living thing” unit was invented. Researchers have looked inside real trained language models, which stack many feed-forward networks on top of each other, and checked which texts make a unit fire most strongly. Geva et al. (2021) did this for a model with 16 layers trained on Wikipedia text; for almost every unit they sampled, people could name the pattern. Some of the units they report, with one text that makes each fire:
layer 1word pattern
text ending with “substitutes”
“In German service, they were used as substitutes”
layer 5word + meaning
text ending with “press”, often news-related
“Groening told the press”
layer 6word + meaning
military, ending with “base” / “bases”
“Aircraft from all four carriers attacked the Australian base”
layer 10meaning
a “part of” relation
“He was also a part of the Indian delegation”
layer 13meaning
text ending with a time range
“Weekend tolls are in effect from 7:00 pm Friday until”
layer 16meaning
TV shows
“Time shifting viewing added 57 percent to the episode’s”
A unit also writes something back when it fires: it pushes the vector towards certain words. Geva et al. (2022) listed these words for units of the same Wikipedia model and of GPT-2 (12 layers, an early model of the family behind ChatGPT):
Wikipedia model · layer 1grammar
pronouns
Her, She, Their, they, His
Wikipedia model · layer 6grammar
adverbs
largely, rapidly, effectively, normally
Wikipedia model · layer 13meaning
groups of people
policymakers, geneticists, Ohioans
GPT-2 · layer 3meaning
measurements
kg, percent, yards, pounds, hours
GPT-2 · layer 8grammar
question / relative words
which, whose, whom, where, who
GPT-2 · layer 11meaning
food and drinks
drinks, coffee, tea, soda, burgers, sushi
- From words to meaning: units in early layers react to surface patterns (a particular word at the end); units in later layers react to meaning (a relation, a time range, a topic).
- Facts: Dai et al. (2022) found units that help a model complete The capital of Ireland is ___ with Dublin. Changing those few units could update or erase that one fact, while other facts were barely affected.
- Not always this clean: these are the clearest cases; many units mix several patterns, and a pattern is usually spread over many units.
💡 A useful picture: in self-attention the tokens talk to each other and gather information; in the feed-forward network each token thinks about what it has gathered.
Transformers
What is a Transformer?
A Transformer is a neural network architecture for sequences such as text, built from stacked layers of self-attention (Part III) and small feed-forward networks (Part IV). It was introduced in 2017 in the paper Attention Is All You Need (Vaswani et al.) for machine translation, and it is now the basis of almost all modern language models, including GPT-style large language models such as ChatGPT.
- Input: a sequence of token vectors (Part II).
- Output: one context-aware vector per token.
- Core idea: let every token look at every other token directly (self-attention), and repeat this in many layers.
How did we get here?
- 2013–14Static word embeddingsword2vec, GloVe: one fixed vector per word, the lookup table of Part II.
- 2014Recurrent networksRead the text one word at a time, carrying a running memory (seq2seq).
- 2014AttentionAdded to translation models: look back at all input words when producing each output word.
- 2017TransformerOnly attention: every token looks at every other token, all in parallel.
- 2018GPTLarge pre-trained Transformers that generate text: the family behind ChatGPT.
- 2019Sentence embeddingsTransformers turned into one vector per sentence, for search. Lab 09.
- Static word embeddings (word2vec, 2013; GloVe, 2014) learned one dense vector per word. Useful, but bank always gets the same vector: no context.
- Recurrent networks (RNNs, LSTMs; sequence-to-sequence models, 2014) read the text one word at a time, passing a running memory along. They capture context, but they are slow to train (each step waits for the previous one) and information from far-away words fades along the way.
- Attention (Bahdanau et al., 2014) was first added on top of recurrent translation models, so that each output word could look back at all the input words.
- The Transformer (2017) keeps only the attention and drops the word-by-word reading: every token reaches every other token in one step, and all tokens are processed in parallel, which makes training on GPUs much faster.
- Then came large pre-trained Transformers such as GPT (2018), the family behind ChatGPT, and Transformers trained to produce sentence embeddings (2019), the models behind semantic search (Lab 09) and, later in this course, RAG.
See it: word by word vs all at once
Information from animal must pass through every step to reach it.
it reaches animal directly, and all tokens are processed at the same time.
Transformer layers
A Transformer contains repeated layers; each layer refines the representations.
Inside one layer
self-attention: tokens exchange information↓ feed-forward network: each token transformed on its ownplus residual connections and normalization around both.
You do not need to derive every component. The key point: a Transformer repeatedly lets tokens interact and transforms their representations.
Residual connections and normalization
Two small tricks make it possible to stack many layers on top of each other. Both are applied around every block (self-attention and the feed-forward network):
- Residual connection: the block's result is added to its input instead of replacing it: new vector = \(x + \text{block}(x)\). The block only writes a correction on top of the vector, so the information already there is never lost, and very deep stacks can still be trained. The idea comes from image recognition (He et al., 2015).
- Normalization: after the addition, each token's vector is rescaled so that its numbers have average 0 and spread 1 (then a learned scale and shift). Without it, the numbers could grow or shrink a little more after every layer (layer normalization, Ba et al., 2016).
See it: one block, step by step
A token vector with four numbers, and what a block wants to add to it (invented numbers). Switch the residual connection off to see what is lost.
Position matters
A Transformer also needs information about token position. Otherwise dog bites man and man bites dog would contain the same bag of token embeddings. Different Transformer families encode position in different ways (some add a learned position vector to each token embedding). For this course: token identity + position + context → contextual representation.
See it: the same words in another order
Without position information, the model would see each sentence as a bag of tokens: which tokens occur, but not where. Pick a pair and switch positions off and on.
💡 For a search engine, flights from Sofia to London and flights from London to Sofia are different requests: the order of the words carries meaning.
Summary: key takeaways
- Text is split into tokens; each token ID looks up a learned vector, its token embedding, the same in every context.
- A token embedding is only the starting representation; a Transformer makes it contextual.
- Self-attention lets tokens use information from other tokens: the query asks, the key matches, the value answers.
- A Transformer stacks self-attention layers and adds position information, so word order and context shape every token vector.