Strings, Bytes & Unicode Encodings

Reviewed & published by Brayan K

In Python a str is readable Unicode text while bytes is raw binary data, and you move between them with encode and decode using an encoding such as UTF-8 — the foundation for working correctly with files, networks, and every language's characters.

Part of the free Python course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Mixing up text and bytes is one of the most common sources of confusing errors. Once the difference clicks, encoding bugs stop being mysterious.

What You'll Learn in This Lesson

• How str (text) differs from bytes (binary)
• Converting with encode() and decode()
• Why UTF-8 is the encoding to use
• Reading and fixing UnicodeDecodeError
• Characters and code points with ord/chr
• Escape sequences and b"" literals

1 str vs bytes: encode & decode

A str is text you can read; bytes is raw data you store or send. To turn text into bytes you encode it; to turn bytes back into text you decode them. Always name the encoding:

text = "café"          # a str: 4 characters

# encode() turns text into bytes using an encoding
raw = text.encode("utf-8")
print(raw)              # b'caf\xc3\xa9'
print(type(raw))        # <class 'bytes'>
print(len(text), len(raw))   # 4 5  (é takes 2 bytes in UTF-8)

# decode() turns bytes back into text
back = raw.decode("utf-8")
print(back)             # café
print(back == text)     # True

# A b"" literal is bytes you write directly (ASCII only)
print(b"hello")         # b'hello'

# Round-trip text through bytes and inspect the size
messages = ["hi", "café", "日本語", "🎉"]

for m in messages:
    raw = m.encode("utf-8")
    print(f"{m!r}: {len(m)} chars, {len(raw)} bytes")

# ✅ Expected output:
# 'hi': 2 chars, 2 bytes
# 'café': 4 chars, 5 bytes
# '日本語': 3 chars, 9 bytes
# '🎉': 1 chars, 4 bytes

2 The UnicodeDecodeError

Decoding bytes with the wrong encoding raises UnicodeDecodeError. The cure is to decode with the same encoding the data was created in. When you can't, errors= lets you continue:

raw = "café".encode("utf-8")   # b'caf\xc3\xa9'

# Decoding UTF-8 bytes as ASCII fails on the multi-byte é
try:
    raw.decode("ascii")
except UnicodeDecodeError as e:
    print("Failed:", e.reason)      # Failed: ordinal not in range(128)

# Decode with the CORRECT encoding -> works
print(raw.decode("utf-8"))          # café

# errors='replace' substitutes a placeholder instead of crashing
print(raw.decode("ascii", errors="replace"))   # caf??

# errors='ignore' drops the undecodable bytes
print(raw.decode("ascii", errors="ignore"))    # caf

Prefer fixing the encoding over hiding the error. Use errors="replace" only for best-effort display of possibly-corrupt data, never for data you must keep intact.

3 ord, chr & Escape Sequences

ord() turns a character into its Unicode code point number, and chr() turns a number back into a character. Escape sequences like \n and \u let you write special characters in source code:

# ord(): character -> code point;  chr(): code point -> character
print(ord("A"))        # 65
print(chr(65))         # A
print(ord("🎉"))       # 127881
print(chr(127881))     # 🎉

# Escape sequences inside a normal string
print("line1\nline2")  # newline between them
print("tab\there")     # a tab gap
print("\u00e9")        # é   (a Unicode escape by code point)

# A tiny Caesar shift using ord/chr
def shift(ch, n):
    return chr(ord(ch) + n)
print(shift("a", 1), shift("y", 1))   # b z

# Build a Caesar cipher for lowercase letters with ord/chr
def caesar(text, shift):
    out = []
    for ch in text:
        if ch.isalpha():
            base = ord("a")
            rotated = (ord(ch) - base + shift) % 26 + base
            out.append(chr(rotated))
        else:
            out.append(ch)
    return "".join(out)

secret = caesar("hello world", 3)
print("Encoded:", secret)
print("Decoded:", caesar(secret, -3))

# ✅ Expected output:
# Encoded: khoor zruog
# Decoded: hello world

🎯 Your Turn: Fill in the Blanks

Replace each ___ so the text round-trips through UTF-8 bytes and comes back unchanged.

# 🎯 YOUR TURN: round-trip text through bytes
original = "Héllo, 世界!"

# 1. Turn the text into UTF-8 bytes
data = original.___("utf-8")
print("Bytes length:", len(data))

# 2. Turn the bytes back into text
restored = data.___("utf-8")
print("Restored:", restored)
print("Match:", restored == original)

# Expected output:
# Bytes length: 15
# Restored: Héllo, 世界!
# Match: True

Common Errors

❌ Mixing str and bytes

greeting = "Hi " + b"there"   # TypeError: can only concatenate str to str

✅ Convert to one type first

greeting = "Hi " + b"there".decode("utf-8")   # "Hi there"

❌ Decoding with the wrong encoding

raw = "naïve".encode("utf-8")
raw.decode("ascii")   # UnicodeDecodeError

✅ Decode with the matching encoding

raw.decode("utf-8")   # "naïve"

❌ Calling string methods on bytes

b"Hello".upper()   # works, but b"café"... b-literals can't hold é

✅ Decode to str for text operations

raw.decode("utf-8").upper()   # operate on real text

🎯 Mini Challenge: Byte Size Report

Write a report comparing how many bytes each name takes in UTF-8 — handy for understanding storage and network limits across languages.

# 🎯 Mini Challenge: report UTF-8 byte sizes
names = ["Alice", "José", "Łukasz", "さくら"]

for name in names:
    raw = name.encode("utf-8")
    ratio = len(raw) / len(name)
    print(f"{name}: {len(name)} chars -> {len(raw)} bytes ({ratio:.1f} bytes/char)")

# Expected output:
# Alice: 5 chars -> 5 bytes (1.0 bytes/char)
# José: 4 chars -> 5 bytes (1.2 bytes/char)
# Łukasz: 6 chars -> 7 bytes (1.2 bytes/char)
# さくら: 3 chars -> 9 bytes (3.0 bytes/char)

📚 Keep Learning

• Python docs: Unicode HOWTO (the definitive guide)
• Python docs: bytes and bytearray operations
• Python docs: standard encodings list

Frequently Asked Questions

Lesson complete — text and bytes no longer confuse you!

You know that str is text and bytes is binary, you convert with encode/decode using UTF-8, you can diagnose a UnicodeDecodeError, and you can map characters to code points with ord and chr.

Practice quiz

Continue this course

• Previous: Numbers: int, float, Decimal & Fraction
• Next: Iterators: __iter__ & __next__ — How the iterator protocol powers every for loop — write your own
• Quick reference: Python cheat sheet

Frequently asked questions

What is the difference between str and bytes?

A str is a sequence of human-readable Unicode characters — letters, emoji, symbols — meant for text. bytes is a sequence of raw 8-bit numbers (0-255), meant for binary data like files and network packets. You convert between them with encode (str to bytes) and decode (bytes to str), always choosing an encoding such as UTF-8.

What is UTF-8 and why is it the default?

UTF-8 is an encoding that represents every Unicode character as one to four bytes. ASCII characters stay one byte, so English text is compact, while accented letters, emoji, and other scripts use more bytes as needed. It's backward-compatible with ASCII and is the dominant encoding of the web, which is why Python defaults to it.

Why do I get a UnicodeDecodeError?

A UnicodeDecodeError happens when you decode bytes using the wrong encoding — for example reading a UTF-8 file as ASCII or Latin-1, so a multi-byte character can't be interpreted. The fix is to decode with the same encoding the data was written in. When you genuinely can't, errors='replace' or errors='ignore' lets decoding continue.

What do ord() and chr() do?

ord() takes a single character and returns its Unicode code point (an integer), and chr() does the reverse, turning a code point back into its character. ord('A') is 65 and chr(65) is 'A'. They're the bridge between characters and the numbers that represent them, useful for ciphers, sorting, and exploring Unicode.