Lesson 24 • Advanced

Diffusion Models Explained

Understand how DALL-E and Stable Diffusion generate images from text — the forward noising process, reverse denoising, and classifier-free guidance.

✅ What You'll Learn

• Forward diffusion: gradually adding noise to data
• Reverse diffusion: learning to denoise step by step
• The U-Net architecture and noise prediction
• Text conditioning and classifier-free guidance

🌫️ From Noise to Art

🎯 Real-World Analogy: Imagine crumpling a piece of paper into a ball (adding noise). A diffusion model learns to un-crumple any ball of paper back into a beautiful drawing. During training, it watches millions of crumpling processes. At generation time, you hand it a random ball of paper and it carefully smooths it out — guided by your text description of what the drawing should be.

Diffusion models are the technology behind DALL-E 2, Stable Diffusion, Midjourney, and Imagen. They produce higher quality images than GANs with more stable training, and they naturally support text-to-image generation through cross-attention conditioning.

Try It: Forward Diffusion

Watch clean data gradually dissolve into pure noise

Try it Yourself »

Python

import numpy as np

# Forward Diffusion: Gradually Add Noise to Data
# This is the "destruction" phase — we learn to REVERSE it

np.random.seed(42)

def add_noise(x, t, total_steps, beta_start=0.0001, beta_end=0.02):
    """Add noise according to a noise schedule"""
    beta = beta_start + (beta_end - beta_start) * t / total_steps
    alpha = 1 - beta
    noise = np.random.randn(*x.shape)
    noisy = np.sqrt(alpha) * x + np.sqrt(beta) * noise
    return noisy, noise, beta

# Start with a simple 
...

Try It: Reverse Diffusion

See how the model removes noise step by step to generate data

Try it Yourself »

Python

import numpy as np

# Reverse Diffusion: The Model Learns to Denoise
# Start from noise, gradually remove it to create data

np.random.seed(42)

def predict_noise(x_noisy, t, total_steps):
    """Simplified noise prediction (in practice, this is a U-Net)"""
    # Real model: U-Net with time embedding predicts the noise
    # Here we simulate with a simple estimate
    estimated_noise = x_noisy * (t / total_steps) * 0.8
    return estimated_noise

def denoise_step(x_noisy, t, total_steps):
    ""
...

⚠️ Common Mistake: Confusing the noise schedule with the learning rate. The noise schedule (beta) controls how much noise is added at each timestep during training. It's fixed before training. The learning rate is a separate optimiser parameter. Most diffusion models use a linear or cosine noise schedule.

💡 Pro Tip: For practical use, start with Stable Diffusion via the diffusers library from Hugging Face. You can generate images in ~10 lines of Python. For fine-tuning on custom data, use DreamBooth or LoRA — they require as few as 5-10 training images.

📋 Quick Reference

Model	Key Innovation	Speed
DDPM	Original diffusion for images	Slow (1000 steps)
DDIM	Deterministic sampling	Faster (50 steps)
Latent Diffusion	Diffuse in latent space	Much faster
Stable Diffusion	Open-source latent diffusion	Consumer GPU
DALL-E 3	Caption-based training	API only

🎉 Lesson Complete!

You now understand the mechanics behind modern image generation. Next, dive into the architecture of Large Language Models!