Thread-Safe Queues (queue)

Reviewed & published by Brayan K

A queue.Queue is a thread-safe container for passing items between threads — producers put() work in and consumers get() it out, with all the locking handled for you.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

It's the backbone of the producer/consumer pattern: one or more threads generate work, others process it, and the queue safely connects them. No manual locks required.

What You'll Learn in This Lesson

• Putting and getting items with put() and get()
• Why Queue is thread-safe out of the box
• Coordinating with task_done() and join()
• Building a producer/consumer pipeline with threads
• Stopping workers with a poison pill
• LifoQueue (stack) and PriorityQueue ordering

1 The Basics: put() and get()

A FIFO queue.Queue hands items back in the order they went in. put(item) adds; get() removes and returns the oldest item; empty() and qsize() report state:

import queue

q = queue.Queue()
for item in ["task-a", "task-b", "task-c"]:
    q.put(item)

print("Size:", q.qsize())          # Size: 3
while not q.empty():
    print("Processing", q.get())   # FIFO: a, then b, then c
print("Done")

import queue

q = queue.Queue()
for item in ["task-a", "task-b", "task-c"]:
    q.put(item)

print("Size:", q.qsize())
while not q.empty():
    print("Processing", q.get())
print("Done")

# ✅ Expected output:
# Size: 3
# Processing task-a
# Processing task-b
# Processing task-c
# Done

2 Producer/Consumer with Threads

Here's the real power: a producer feeds items in while consumer threads pull them out. task_done() marks each item complete, join() waits until everything is processed, and a None "poison pill" tells each worker to stop:

import queue
import threading

q = queue.Queue()
results = []
lock = threading.Lock()

def consumer():
    while True:
        item = q.get()
        if item is None:          # poison pill -> stop
            q.task_done()
            break
        with lock:
            results.append(item * 10)
        q.task_done()             # mark this item handled

workers = [threading.Thread(target=consumer) for _ in range(3)]
for w in workers:
    w.start()

for n in range(1, 6):
    q.put(n)                      # produce work

q.join()                          # wait until queue is drained

for _ in workers:
    q.put(None)                   # one poison pill per worker
for w in workers:
    w.join()

print(sorted(results))
# [10, 20, 30, 40, 50]

import queue
import threading

q = queue.Queue()
results = []
lock = threading.Lock()

def consumer():
    while True:
        item = q.get()
        if item is None:
            q.task_done()
            break
        with lock:
            results.append(item * 10)
        q.task_done()

workers = [threading.Thread(target=consumer) for _ in range(3)]
for w in workers:
    w.start()

for n in range(1, 6):
    q.put(n)

q.join()

for _ in workers:
    q.put(None)
for w in workers:
    w.join()

# sort for deterministic output (workers finish in any order)
print(sorted(results))

# Expected output:
# [10, 20, 30, 40, 50]

3 LifoQueue and PriorityQueue

The queue module ships two ordered variants. A LifoQueue is a stack (last in, first out). A PriorityQueue serves the smallest item first — put (priority, value) tuples and the lowest priority number wins:

import queue

# LifoQueue = stack (last in, first out)
lifo = queue.LifoQueue()
for x in [1, 2, 3]:
    lifo.put(x)
print("LIFO order:", [lifo.get() for _ in range(3)])   # [3, 2, 1]

# PriorityQueue = smallest priority served first
pq = queue.PriorityQueue()
pq.put((2, "medium"))
pq.put((1, "high"))
pq.put((3, "low"))
order = [pq.get()[1] for _ in range(3)]
print("Priority order:", order)   # ['high', 'medium', 'low']

import queue

lifo = queue.LifoQueue()
for x in [1, 2, 3]:
    lifo.put(x)
print("LIFO order:", [lifo.get() for _ in range(3)])

pq = queue.PriorityQueue()
pq.put((2, "medium"))
pq.put((1, "high"))
pq.put((3, "low"))
order = [pq.get()[1] for _ in range(3)]
print("Priority order:", order)

# ✅ Expected output:
# LIFO order: [3, 2, 1]
# Priority order: ['high', 'medium', 'low']

🎯 Your Turn: Fill in the Blanks

Replace each ___ so the queue is filled, then fully drained into a list while marking each item done.

import queue

q = queue.Queue()
for word in ["red", "green", "blue"]:
    q.___(word)               # hint: add to the queue

collected = []
while not q.empty():
    collected.append(q.___())  # hint: remove from the queue
    q.task_done()

print(collected)

# Expected output:
# ['red', 'green', 'blue']

⚠️ Common Errors

❌ Calling get() on an empty queue with no thread to fill it

q = queue.Queue()
print(q.get())        # hangs forever — nothing to get

✅ Guard with empty(), or use a timeout

if not q.empty():
    print(q.get())
# or: q.get(timeout=1)  -> raises queue.Empty instead of hanging

❌ Forgetting task_done(), so join() never returns

item = q.get()
process(item)
# missing q.task_done() -> q.join() blocks forever

✅ Always pair get() with task_done()

item = q.get()
process(item)
q.task_done()

❌ Sending fewer poison pills than workers

If you have 3 worker threads, you must put 3 None sentinels — otherwise some workers never receive the stop signal and join() on them hangs.

🎯 Mini Challenge: Square Pipeline

Two worker threads pull numbers from a work queue, square them, and put the results on a second queue. Drain the result queue, sort it, and print the sum.

import queue
import threading

work = queue.Queue()
done = queue.Queue()

def worker():
    while True:
        n = work.get()
        if n is None:
            work.task_done()
            break
        done.put(n * n)
        work.task_done()

threads = [threading.Thread(target=worker) for _ in range(2)]
for t in threads:
    t.start()

for n in [1, 2, 3, 4, 5]:
    work.put(n)
work.join()

for _ in threads:
    work.put(None)
for t in threads:
    t.join()

squares = sorted(done.queue)
print("Squares:", squares)
print("Sum:", sum(squares))

# Expected output:
# Squares: [1, 4, 9, 16, 25]
# Sum: 55

📚 Keep Learning

• • queue — A synchronized queue class
• • threading — Thread-based parallelism
• • collections.deque — a fast double-ended queue

Frequently Asked Questions

Lesson complete — you can pipe work between threads!

You can put() and get() safely from many threads, coordinate completion with task_done() and join(), stop workers with a poison pill, and choose LifoQueue or PriorityQueue when ordering matters.

Practice quiz

Continue this course

• Previous: multiprocessing: True Parallelism
• Next: contextlib: contextmanager, suppress, ExitStack — Write your own with-statements and manage dynamic resources
• Quick reference: Python cheat sheet

Frequently asked questions

Why use queue.Queue instead of a plain list?

queue.Queue is thread-safe: multiple threads can put() and get() at the same time without corrupting the data or needing your own locks. A plain list is not safe for concurrent writes from several threads. Queue handles all the locking internally, so it's the right tool for passing work between threads.

What do task_done() and join() do?

They let a producer know when all the work it queued has been processed. Each consumer calls q.task_done() after finishing an item. The producer calls q.join(), which blocks until the number of task_done() calls equals the number of items put in. It's how you wait for a work queue to fully drain before moving on.

What's a 'poison pill'?

It's a sentinel value (commonly None) that you put on the queue to tell a consumer thread to stop. Worker threads usually loop forever on q.get(); when they pull a None, they break out of the loop and exit. You send one poison pill per worker so every thread gets the signal to shut down cleanly.

When would I use LifoQueue or PriorityQueue?

Use LifoQueue when you want stack behavior — the most recently added item comes out first (useful for depth-first traversal). Use PriorityQueue when items should come out in priority order rather than arrival order — you put (priority, item) tuples and the smallest priority is served first, perfect for task schedulers.