Benchmarking with timeit

Reviewed & published by Brayan K

The timeit module measures how long small snippets of code take by running them many times, giving you reliable numbers instead of the noisy guesses a single timer produces.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

When you want to know which of two approaches is actually faster, timeit is the right tool — it handles the warm-up, repetition, and timing details so your comparison is trustworthy.

What You'll Learn in This Lesson

• Timing a snippet with timeit.timeit
• The number and setup arguments
• Why microbenchmarks beat manual time()
• Comparing two approaches the right way
• Using repeat and taking the minimum
• %timeit in Jupyter / IPython notebooks

1 timeit.timeit: number & setup

timeit.timeit(stmt, setup, number) runs stmt number times and returns the total seconds. The setup code runs once before timing — use it to import modules or build test data that shouldn't count toward the measurement:

import timeit

# stmt runs 'number' times; setup runs once, untimed
total = timeit.timeit(
    stmt="sum(range(100))",
    number=10000,
)
# 'total' is seconds for all 10,000 runs (varies per machine,
# so we don't print the raw number here).
print("ran the snippet 10,000 times")
print("got a positive duration:", total > 0)

import timeit

# A timer can take a callable instead of a string of code
def build_list():
    return [x * 2 for x in range(100)]

total = timeit.timeit(build_list, number=1000)

# We only assert it produced a sensible, positive measurement
print("ran 1000 times")
print("duration is positive:", total > 0)

# ✅ Expected output:
# ran 1000 times
# duration is positive: True

2 Comparing Two Approaches

The real point of timeit is to settle "which is faster?" arguments. Time both candidates with the same number, then compare the totals. A classic example: testing membership in a list vs a set:

import timeit

setup = "items = list(range(1000)); items_set = set(range(1000))"

list_time = timeit.timeit("999 in items", setup=setup, number=50000)
set_time  = timeit.timeit("999 in items_set", setup=setup, number=50000)

# Report the WINNER, not the raw times -> stable on any machine
winner = "set" if set_time < list_time else "list"
print("Faster for membership:", winner)   # Faster for membership: set

import timeit

# Two ways to build the same 500-character string.
plus_code = "s = ''\nfor i in range(500):\n    s += 'x'"
join_code = "s = ''.join('x' for i in range(500))"

plus_time = timeit.timeit(plus_code, number=2000)
join_time = timeit.timeit(join_code, number=2000)

# timeit gives you the seconds for ALL the runs, not one of them.
per_loop_plus = plus_time / 2000

# Both really do produce the same string — timing is the only difference.
s_plus = ""
for i in range(500):
    s_plus += "x"
s_join = "".join("x" for i in range(500))
print("Same string:", s_plus == s_join, "- length", len(s_join))

# Deliberately NOT printed: which one won, or how long either took.
# CPython optimises "s += 'x'" in place when the string has only one reference,
# so on some builds += beats join here and on others it does not. A tutorial
# that prints a winner is telling you about the machine it was written on.
print("Both measured:", plus_time > 0 and join_time > 0)
print("Per loop is the total divided by number:", round(per_loop_plus * 2000, 10) == round(plus_time, 10))
print("Run it yourself to see which wins on YOUR machine.")

# ✅ Expected output:
# Same string: True - length 500
# Both measured: True
# Per loop is the total divided by number: True
# Run it yourself to see which wins on YOUR machine.

3 repeat, the minimum, and %timeit

timeit.repeat runs the whole timing several times and returns a list. The official advice is to take the minimum, not the average — the fastest run is the one least disturbed by background processes:

import timeit

# repeat=5 -> five independent timings of 'number' runs each
runs = timeit.repeat("sum(range(50))", number=10000, repeat=5)

print("number of runs:", len(runs))            # 5
print("min is the recommended summary:",
      min(runs) == sorted(runs)[0])            # True

📓 In a Jupyter / IPython notebook:

%timeit sum(range(100))        # times one line
%%timeit                        # times a whole cell
total = 0
for i in range(100):
    total += i

The %timeit "magic" picks a sensible number and repeat for you and prints a nicely formatted result — it's timeit with zero boilerplate.

import timeit

# repeat() returns a list of timings; summarize with min()
runs = timeit.repeat("max([4, 9, 1, 7, 3])", number=20000, repeat=5)

print("runs collected:", len(runs))
print("all positive:", all(t > 0 for t in runs))
print("best == smallest:", min(runs) == sorted(runs)[0])

# ✅ Expected output:
# runs collected: 5
# all positive: True
# best == smallest: True

🎯 YOUR TURN: Fill in the Blanks

Fill in each ___ to benchmark two lookups and print the winner. (Hint: the function is timeit, and the keyword for run count is number.)

import timeit

setup = "d = {i: i for i in range(1000)}; pairs = list(d.items())"

# Time a dict key lookup vs scanning a list of pairs
dict_time = timeit.___("500 in d", setup=setup, ___=50000)
scan_time = timeit.timeit("any(k == 500 for k, _ in pairs)",
                          setup=setup, number=50000)

winner = "dict" if dict_time < scan_time else "scan"
print("Faster lookup:", winner)

# Expected output:
# Faster lookup: dict

Common Errors

❌ Referencing your own variables inside a string stmt

data = list(range(100))
timeit.timeit("sum(data)", number=1000)
# NameError: name 'data' is not defined  (stmt has its own namespace)

✅ Build the data in setup (or import it)

import timeit
t = timeit.timeit("sum(data)", setup="data = list(range(100))", number=1000)
print("ok:", t > 0)   # ok: True

❌ Comparing absolute times across machines or runs

# "It took 0.0023s on my laptop" -> meaningless to anyone else

✅ Compare relative results (winner or ratio)

import timeit
a = timeit.timeit("sum(range(100))", number=10000)
b = timeit.timeit("sum([x for x in range(100)])", number=10000)
print("range-sum is faster:", a < b)   # a relative, portable claim

❌ Forgetting the default number is one million

# This runs a slow snippet 1,000,000 times -> waits forever
timeit.timeit("time.sleep(0.1)", setup="import time")

✅ Set a small number for slow code

import timeit
t = timeit.timeit("time.sleep(0.01)", setup="import time", number=3)
print("finished quickly:", t > 0)   # finished quickly: True

🎯 Mini Challenge: Which Sum Wins?

Benchmark summing with a generator expression vs building a throwaway list first, run each five times with repeat, take the minimum of each, and print which approach wins. (Surprise: for a modest size the list version often edges it out, because the generator pays a small overhead per item — exactly the kind of myth-busting timeit is built for.)

import timeit

gen_runs = timeit.repeat(
    "sum(x * x for x in range(1000))",
    number=2000, repeat=5,
)
list_runs = timeit.repeat(
    "sum([x * x for x in range(1000)])",
    number=2000, repeat=5,
)

# Take the best (minimum) of each set of runs
gen_best = min(gen_runs)
list_best = min(list_runs)

# NOT printed: which one won. The two are close enough that the answer flips
# between CPython versions and between machines — this very lesson claimed
# "list-comprehension" and started printing "generator" on the same computer,
# on the same day. repeat() exists because a single measurement is noise; a
# tutorial that prints a winner is quoting noise back at you.
print("collected", len(gen_runs), "runs each")
print("best of 5 is the smallest:", gen_best == min(gen_runs))
print("both positive:", gen_best > 0 and list_best > 0)
print("Run it yourself, several times, before believing either.")

# ✅ Expected output:
# collected 5 runs each
# best of 5 is the smallest: True
# both positive: True
# Run it yourself, several times, before believing either.

📚 Keep Learning

• timeit — Measure execution time of small code snippets (official docs)
• cProfile & profile — for whole-program profiling
• time.perf_counter — the high-resolution clock timeit uses

Frequently Asked Questions

Lesson complete — you benchmark like a pro!

You can time snippets with timeit.timeit, prepare data in setup, compare two approaches by their relative result, run several timings with repeat and take the minimum, and reach for %timeit in notebooks — never trusting a single manual time() reading again.

Practice quiz

Continue this course

• Previous: Text Wrapping & Formatting (textwrap)
• Next: The warnings Module — Emit, filter and capture warnings — and turn them into errors
• Quick reference: Python cheat sheet

Frequently asked questions

What is the timeit module for?

timeit measures how long a small piece of code takes to run, accurately. It runs your snippet many times, disables the garbage collector during the measurement, and uses the best available clock, so it sidesteps the noise you'd get from a single manual time.time() reading. It's the standard way to compare two implementations.

Why is timeit better than wrapping code in time.time()?

A single time.time() call around fast code is dominated by noise — clock resolution, the cost of the calls themselves, and one-off scheduling hiccups. timeit runs the code thousands of times, takes the total, and reports a stable per-run figure. For microbenchmarks (operations measured in microseconds) this is the difference between a meaningful number and a coin flip.

What does the number argument do, and what is repeat?

number is how many times the snippet runs in a single timing (default 1,000,000). The total time is divided across those runs to smooth out noise. repeat runs that whole timing several times and returns a list, so you can take the minimum — the recommended summary, because the fastest run had the least interference from other processes.

Should I compare absolute nanoseconds between machines?

No. Absolute times depend on your CPU, OS, Python build, and current load, so they vary run to run and machine to machine. Compare RELATIVE results instead: which of two approaches is faster, or the ratio between them. That conclusion stays valid everywhere, while a raw nanosecond count does not.