Transformers & Large Language Models

Every token (word piece) is turned into three vectors. A Query is what the token is looking for. A Key is what each token advertises about itself. A Value is the actual information a token carries.

To work out how much word A should attend to word B, you compare A's Query with B's Key. The comparison is a dot product — multiply the matching numbers and add them up. A bigger dot product means a better match, so more attention.

The raw dot product can grow large when vectors are long, which makes the next step (softmax) unstable. The fix is to divide by the square root of the vector length, written d_k. That single division is why it's called scaled dot-product attention.

Here it is with no libraries — just two small vectors and one score:

# Scaled dot-product attention — by hand, no libraries
# We compare one query vector against one key vector.

import math

def dot(a, b):
    # Sum of element-wise products = the raw "match" score
    return sum(x * y for x, y in zip(a, b))

# A query (what a word is looking for) and a key (what a word offers)
query = [1.0, 0.0, 1.0]   # e.g. the word "it"
key   = [1.0, 1.0, 1.0]   # e.g. the word "cat"

raw_score = dot(query, key)        # 1*1 + 0*1 + 1*1 = 2.0
d_k = len(query)                 
...

A real token produces one score for every other token. Softmax converts that list of scores into weights between 0 and 1 that add up to exactly 1.0 — your 100% attention budget. The highest score gets the biggest slice.

This worked example softmaxes three scores by hand:

# Softmax turns raw scores into attention weights that sum to 1.0
import math

def softmax(scores):
    m = max(scores)                       # subtract the max for numerical safety
    exps = [math.exp(s - m) for s in scores]
    total = sum(exps)
    return [e / total for e in exps]      # each weight is between 0 and 1

# Three attention scores: "it" attends to cat / mat / sky
scores = [2.0, 1.0, 0.1]
weights = softmax(scores)

for label, w in zip(["cat", "mat", "sky"], weights):
    print(la
...

In real models you don't loop in Python — you use matrix operations so the whole sequence is processed at once. The maths is identical: scores, softmax, weighted sum of Values. The @ symbol is matrix multiplication.

import torch
import torch.nn.functional as F

# Q, K, V for a 3-token sequence, each token a 4-dim vector
# (in a real model these come from learned linear layers)
torch.manual_seed(0)
Q = torch.randn(3, 4)
K = torch.randn(3, 4)
V = torch.randn(3, 4)

d_k = Q.size(-1)                         # 4
scores = Q @ K.transpose(-2, -1) / d_k ** 0.5   # (3, 3) score matrix
weights = F.softmax(scores, dim=-1)      # each row sums to 1.0
output = weights @ V                     # context-aware vectors, shape (3, 4)

print("scores shape:", tuple(scores.shape))
print("weights row 0 sums to:", round(weights[0].sum().item(), 4))
print("output shape:", tuple(output.shape))

# Expected output:
# scores shape: (3, 3)
# weights row 0 sums to: 1.0
# output shape: (3, 4)

Note the shapes: a 3-token input produces a 3×3 score matrix (every token scored against every token), and the output keeps the original shape so blocks can stack.

One attention calculation can only learn one kind of relationship. Multi-head attention runs several attention computations in parallel — each "head" gets its own learned projection of the data, so one head might track grammar while another tracks meaning. Their outputs are concatenated and mixed back together.

Attention treats the input as an unordered set. On its own it can't tell "dog bites man" from "man bites dog". Positional encoding fixes this by adding a unique pattern to each position before attention runs.

The original paper used sine and cosine waves of different frequencies, so each position gets a distinct fingerprint and nearby positions stay similar. Newer models often use learned or rotary positions, but the goal is the same: tell the model where each token sits.

A transformer block bolts four pieces together, and real models stack dozens of these blocks:

• Multi-head self-attention — mix in context from other tokens.
• Feed-forward network (FFN) — a small two-layer net applied to each position independently, to process what attention gathered.
• Residual connections — add the block's input back to its output, so information and gradients flow cleanly through deep stacks.
• Layer normalisation (LayerNorm) — rescale each vector to keep training stable.

Here's a complete block in PyTorch. Notice the input and output shapes are identical — that's what lets you stack blocks:

import torch
import torch.nn as nn

# One transformer block = attention + feed-forward,
# each wrapped in a residual connection and LayerNorm.
class TransformerBlock(nn.Module):
    def __init__(self, d_model=8, n_heads=2):
        super().__init__()
        self.attn = nn.MultiheadAttention(d_model, n_heads, batch_first=True)
        self.norm1 = nn.LayerNorm(d_model)
        self.ffn = nn.Sequential(
            nn.Linear(d_model, 4 * d_model),  # expand
            nn.ReLU(),
            nn.Linear(4 * d_model, d_model),  # project back
        )
        self.norm2 = nn.LayerNorm(d_model)

    def forward(self, x):
        a, _ = self.attn(x, x, x)        # self-attention: Q=K=V=x
        x = self.norm1(x + a)            # residual + layernorm
        f = self.ffn(x)
        x = self.norm2(x + f)            # residual + layernorm
        return x

block = TransformerBlock()
tokens = torch.randn(1, 5, 8)            # (batch=1, seq=5, d_model=8)
out = block(tokens)
print("input shape:", tuple(tokens.shape))
print("output shape:", tuple(out.shape))  # unchanged — blocks stack cleanly

# Expected output:
# input shape: (1, 5, 8)
# output shape: (1, 5, 8)

You almost never train a transformer from scratch. Pre-trained models from Hugging Face give you a working LLM instantly — a decoder generates, an encoder understands:

from transformers import pipeline

# A decoder-only model (GPT-2) GENERATES text...
gen = pipeline("text-generation", model="gpt2")
print(gen("The transformer architecture", max_new_tokens=8)[0]["generated_text"])

# ...while an encoder-only model (BERT) UNDERSTANDS text by filling blanks
fill = pipeline("fill-mask", model="bert-base-uncased")
print(fill("Paris is the [MASK] of France.")[0]["token_str"])

# Expected output (wording varies — models are probabilistic):
# The transformer architecture is a powerful tool for ...
# capital

Fine-tuning a pre-trained model beats training from scratch in almost every real project.