Retrieval-Augmented Generation (RAG)

Reviewed & published by Brayan K

RAG grounds an AI's answers in your own documents. Instead of relying only on training knowledge, you retrieve the most relevant passages and inject them into the prompt, so the model answers from real source text.

Part of the free Prompt Engineering course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

The result: fewer made-up answers, plus fresh and private knowledge the model never trained on. This lesson explains chunking, embeddings, vector stores, and retrieval.

What You'll Learn in This Lesson

• What RAG is and why it grounds answers
• Chunking documents into retrievable pieces
• Embeddings and a vector store
• Top-k retrieval and injecting context
• How RAG reduces hallucination and adds fresh knowledge
• Citing sources, and RAG vs fine-tuning

1 Chunk, embed, store, retrieve

RAG has two phases. First you prepare your data; then, for each question, you retrieve and answer:

2 Grounding cuts hallucination

Without grounding, the model guesses from memory. With it, the model has real facts in front of it:

3 RAG vs fine-tuning

For adding up-to-date or private knowledge, RAG is usually the simpler choice, and the two can be combined.

Build the retriever yourself

The diagram above is easy to nod along to and hard to actually believe. So here is the whole pipeline in about forty lines of plain Python, with no libraries. Run it and watch the scores: the retriever finds the right policy for the first question and the wrong one for the second, even though the two questions mean the same thing. That failure is the single best argument for using a real embedding model, and you can see it happen rather than take our word for it.

# A tiny RAG retriever, end to end, with no libraries.
# The point is to SEE the mechanism the diagram above describes - and where it breaks.

import math
from collections import Counter

# 1) CHUNK - real systems split long documents; here each policy line is a chunk.
chunks = [
    "Refunds are available within 30 days of delivery for unopened items.",
    "Standard delivery takes 3 to 5 working days across mainland UK.",
    "Faulty goods can be returned at any time and we pay the postage.",
    "Gift cards are non-refundable once the code has been revealed.",
]

# 2) EMBED - a real system uses a neural embedding model. A bag-of-words vector
#    is enough to show the shape of retrieval: shared words -> higher score.
def embed(text):
    return Counter(text.lower().replace(".", "").replace(",", "").split())

def cosine(a, b):
    shared = set(a) & set(b)
    dot = sum(a[w] * b[w] for w in shared)
    mag = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))
    return dot / mag if mag else 0.0

store = [(c, embed(c)) for c in chunks]          # 3) STORE

def retrieve(question, k=2):
    q = embed(question)                          # 4) EMBED THE QUESTION
    ranked = sorted(store, key=lambda p: cosine(q, p[1]), reverse=True)  # 5) RETRIEVE
    for text, vec in ranked:
        print(f"  {cosine(q, vec):.3f}  {text}")
    return [t for t, _ in ranked[:k]]

print("Q1: 'What is the refund window for unopened items?'")
top = retrieve("What is the refund window for unopened items?")

# 6) INJECT the retrieved text into the prompt. This string goes to the model.
prompt = (
    "Answer using ONLY the context below. If it is not in the context, say you don't know.\n\n"
    "Context:\n" + "\n".join(f"- {t}" for t in top) + "\n\nQuestion: What is the refund window for unopened items?"
)
print()
print("--- prompt actually sent to the model ---")
print(prompt)

# Now the failure this toy embedding cannot avoid:
print()
print("Q2: 'How long do I have to send something back?'  (same meaning, no shared words)")
retrieve("How long do I have to send something back?")
print()
print("Both questions mean the same thing. Word overlap finds the first and misses")
print("the second -- which is exactly why real RAG uses a semantic embedding model")
print("instead of counting words.")

import math
from collections import Counter

# The knowledge base for a billing help desk.
chunks = [
    "Cancel your subscription any time from the account page.",
    "Annual plans are billed once a year and renew automatically.",
    "We email a receipt to the billing address after every payment.",
]

def embed(text):
    return Counter(text.lower().replace(".", "").split())

store = [(c, embed(c)) for c in chunks]

# 🎯 YOUR TURN - fill in the three blanks marked ___

def cosine(a, b):
    shared = set(a) & set(b)
    # 1) Multiply the counts of every shared word and add them up.
    dot = sum(a[w] * b[___] for w in shared)     # 👉 replace ___ with  w
    mag = math.sqrt(sum(v*v for v in a.values())) * math.sqrt(sum(v*v for v in b.values()))
    return dot / mag if mag else 0.0

question = "How do I cancel my subscription?"
q = embed(question)

# 2) Sort the store so the CLOSEST chunk comes first.
#    Each item in store is a (text, vector) pair.
ranked = sorted(store, key=lambda pair: cosine(q, pair[___]), reverse=True)
# 👉 replace ___ with  1   (index 1 is the vector; index 0 is the text)

# 3) Take the single best match.
best = ranked[___][0]                            # 👉 replace ___ with  0

print("Top match:", best)
print("Score:", round(cosine(q, ranked[0][1]), 3))

# ✅ Expected output:
# Top match: Cancel your subscription any time from the account page.
# Score: 0.136

Common Mistakes (and the fix)

• Chunks too large. — Smaller chunks retrieve more precisely.
• Not telling the model to use only the context. — Say to answer from the provided passages only.
• Skipping citations. — Ask it to name the source so users can verify.
• Reaching for fine-tuning to add facts. — RAG adds fresh knowledge without retraining.

⏱ Test Yourself — Timed Quiz

10 quick questions, 12 seconds each. Instant feedback — beat the clock!

🎉 Lesson Complete

• ✅ RAG grounds answers in your own documents
• ✅ Chunk, embed, and store your data in a vector store
• ✅ Retrieve the top-k relevant chunks and inject them into the prompt
• ✅ Grounding reduces hallucination; cite sources to verify
• ✅ RAG adds fresh / private knowledge without retraining

Practice quiz

Continue this course

• Previous: Tool & Function Calling
• Next: Prompting AI Agents — Multi-step planning, the ReAct loop, tools and stopping conditions

Frequently asked questions

What is RAG in plain English?

RAG stands for Retrieval-Augmented Generation. Instead of relying only on what the model learned during training, you keep your own documents in a searchable index. When a question comes in, you retrieve the most relevant passages and paste them into the prompt as context. The model then answers using that real source text, so its reply is grounded in your data, not just its memory.

Why does RAG reduce hallucination?

Hallucination is when a model confidently makes something up. By handing the model the actual relevant text from your documents, you give it real facts to work from rather than forcing it to guess. The answer is anchored to source material, and you can ask it to cite which passage it used so you can verify it.

What are chunks, embeddings, and a vector store?

Chunks are small pieces of your documents, so you can fetch just the relevant parts. An embedding turns a chunk of text into a list of numbers that captures its meaning. A vector store holds those embeddings and, given a question's embedding, quickly returns the closest-matching chunks, the top-k most relevant pieces, to inject into the prompt.

How is RAG different from fine-tuning?

Fine-tuning retrains the model on examples so new patterns are baked into its weights, which is powerful but slower and harder to update. RAG leaves the model untouched and instead supplies fresh, relevant context at the moment of each question. RAG is the easier way to add up-to-date or private knowledge, and you can change the documents anytime without retraining.