How AI Models Read Your Prompt

Reviewed & published by Brayan K

Under the hood, an AI chatbot is a text predictor. It reads your prompt as small chunks called tokens and then generates the most likely next text, word by word. Knowing this one fact makes you a far better prompter.

Part of the free Prompt Engineering course at LearnCodingFast — hands-on lessons with worked examples and the output they print, plus practice exercises and a quick quiz.

In this lesson you’ll meet four ideas in plain language: tokens, the context window, temperature (randomness), and the big one — the model predicts likely text rather than truly understanding like a human.

What You'll Learn in This Lesson

• How the model turns your words into tokens
• What the context window is and why it has limits
• What temperature (randomness) does to answers
• Why the model 'predicts likely text' rather than knows
• Why long chats can 'forget' earlier details
• How these facts change the way you write prompts

1 It predicts the most likely next text

The model was trained on huge amounts of text. When you prompt it, it works out — based on your words — what text is most likely to come next, and writes that. It is astonishingly good at this, which is why it feels like it understands.

Practical takeaway: a clear prompt makes your desired answer the most likely continuation. A confusing prompt makes a generic or off-target answer most likely instead.

2 Tokens and the context window

The model doesn’t read whole sentences at once. It reads tokens — chunks that are usually a word or part of a word. The context window is the maximum number of tokens it can hold in view at one time, including your message and the earlier conversation.

3 Temperature — how random the wording is

Temperature is a dial for randomness. The same prompt can produce slightly different replies because the model adds a little variety when choosing words.

More creative, varied, surprising. Great for brainstorming names or ideas — but less predictable.

More focused, consistent, repeatable. Better for facts, summaries, and anything you want done the same way each time.

Most chat apps pick a sensible temperature for you. If you can adjust it, lower it for precision and raise it for creativity.

Try it for real

The single most useful thing to understand here is that the model never sees your letters — it sees tokens. Run this and watch strawberry become three pieces. Once you have seen that, why it miscounts letters and why long context costs money both stop being mysterious.

# Why the model 'cannot count letters': it does not see letters, it sees tokens.
# This is a crude stand-in for a real tokenizer, but the lesson is the same.

import re

def tokenize(text):
    # Real tokenizers split on learned sub-word pieces. Splitting on word
    # boundaries and common suffixes is close enough to show the effect.
    rough = re.findall(r"[A-Za-z]+|[0-9]+|[^\sA-Za-z0-9]", text)
    out = []
    for w in rough:
        while len(w) > 5:                 # long words get chopped into pieces
            out.append(w[:4]); w = w[4:]
        out.append(w)
    return out

sentence = "Strawberry jam costs 3.50 at the corner shop."
toks = tokenize(sentence)
print("text  :", sentence)
print("tokens:", toks)
print("count :", len(toks), "tokens for", len(sentence), "characters")
print()

word = "strawberry"
print(f"'{word}' becomes: {tokenize(word)}")
print("The letter 'r' appears", word.count("r"), "times in the text,")
print("but the model is looking at those pieces, not at ten separate letters.")
print("That is why 'how many r's in strawberry' is hard for it and trivial for you,")
print("and why you should ask it to write code to count rather than count itself.")

# 🎯 YOUR TURN — count tokens, then see the budget problem for yourself.

import re

def tokenize(text):
    rough = re.findall(r"[A-Za-z]+|[0-9]+|[^\sA-Za-z0-9]", text)
    out = []
    for w in rough:
        while len(w) > 5:
            out.append(w[:4]); w = w[4:]
        out.append(w)
    return out

docs = [
    "Refunds within 30 days.",
    "Delivery takes 3 to 5 working days.",
    "Faulty goods are collected free of charge.",
]

BUDGET = 12   # pretend the model only has room for 12 tokens of context

# 1) Count the tokens in each document.
counts = [len(tokenize(d)) for d in ___]
# 👉 replace ___ with  docs

# 2) Keep adding documents while they still fit inside BUDGET.
used, kept = 0, []
for doc, n in zip(docs, counts):
    if used + n ___ BUDGET:
        # 👉 replace ___ with  <=
        kept.append(doc); used += n

print("token counts:", counts)
print("kept", len(kept), "of", len(docs), "documents using", used, "tokens")
for k in kept:
    print("  -", k)

# ✅ Expected output:
# token counts: [7, 10, 11]
# kept 1 of 3 documents using 7 tokens
#   - Refunds within 30 days.

Common Misunderstandings

• Thinking it 'knows' facts. — It predicts likely text and can be wrong; verify what matters.
• Assuming it remembers everything. — Long chats can lose earlier context; restate key details.
• Expecting identical answers each time. — Randomness means wording varies; ask for a fixed format.
• Ignoring length. — Very long inputs can crowd out important points.

⏱ Test Yourself — Timed Quiz

10 quick questions, 12 seconds each. Instant feedback — beat the clock!

🎉 Lesson Complete

• ✅ The model predicts the most likely next text from your words
• ✅ It reads text as tokens within a limited context window
• ✅ Long chats can 'forget' early details — restate them
• ✅ Temperature controls creativity vs consistency
• ✅ Clear prompts make your desired answer the likely one

Practice quiz

Continue this course

• Previous: What Is Prompt Engineering?
• Next: Anatomy of a Great Prompt — The five ingredients: role, task, context, format, and constraints

Frequently asked questions

How does an AI chatbot actually 'read' my prompt?

It breaks your text into small chunks called tokens, then predicts the most likely next token over and over to build a reply. It is not looking your answer up in a table. It is generating the most probable helpful continuation of your words, which is why clear wording matters so much.

What is a context window and why should I care?

The context window is the limited amount of text the model can pay attention to at once, including your prompt and the earlier conversation. If a chat gets very long, the earliest parts can fall out of view and be effectively forgotten, so it helps to restate key details rather than assume the model still remembers them.

What is 'temperature' in simple terms?

Temperature is a setting that controls how random or creative the wording is. Higher temperature gives more varied, surprising output, which is nice for brainstorming. Lower temperature gives more focused, consistent answers, which is better for facts, code, or anything where you want the same reliable result.

Why does the same prompt sometimes give different answers?

Because the model uses a bit of randomness when choosing each next word. With some temperature applied, two runs of the same prompt can word things differently. If you need consistency, ask for a precise format or use a lower-temperature setting where it is available.