Categorical Data

Reviewed & published by Brayan K

A categorical column is a pandas dtype that stores each repeated text value once and replaces every cell with a tiny integer code, saving memory and speeding up grouping.

Part of the free Pandas course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Learn to convert columns with astype("category"), measure the memory savings, build ordered categories that sort and compare correctly, and use the .cat accessor.

What You'll Learn in This Lesson

• Convert a column with astype("category")
• Measure the memory savings vs object
• Inspect labels with .cat.categories and .cat.codes
• Build ordered categories (Low < Med < High)
• Compare and sort by a custom order
• When category helps and when it does not

1 Converting to category — astype("category")

A column like status that only ever holds "active", "paused", or "closed" wastes space as an object column — it stores the full word in every single row. Convert it with df["status"].astype("category") and pandas keeps just one copy of each label, replacing every cell with a small integer code that points back at the label.

import pandas as pd

df = pd.DataFrame({"status": ["active", "paused", "active", "closed", "active"]})

df["status"] = df["status"].astype("category")
df["status"].cat.categories   # Index(['active', 'closed', 'paused'])
df["status"].cat.codes        # 0, 2, 0, 1, 0  (the integer per row)

import pandas as pd

# A column with lots of repeats is the perfect candidate
df = pd.DataFrame({"city": ["NYC", "LA", "NYC", "LA", "NYC", "LA"] * 500})

as_object = df["city"].memory_usage(deep=True)
df["city"] = df["city"].astype("category")
as_category = df["city"].memory_usage(deep=True)

# Add print(as_object, as_category) and look at the raw totals — that is the
# point of the lesson. They are deliberately not printed here, because deep=True
# measures real Python string objects and CPython 3.12 made every str 8 bytes
# smaller: this exact code reports 178,632 vs 3,359 on Python 3.11 and
# 154,632 vs 3,343 on 3.12. A tutorial that prints one of those as "the" answer
# is wrong for half its readers.
#
# The size of the win is what holds on every build:
print("category is at least 40x smaller:", as_category * 40 < as_object)
print("dtype is now:", df["city"].dtype)

# ✅ Expected output:
# category is at least 40x smaller: True
# dtype is now: category

2 The .cat Accessor — Managing Categories

Categorical columns get a dedicated namespace, .cat, just like text columns get .str and dates get .dt. Through it you read and reshape the category set: .cat.categories lists the labels, .cat.codes shows the integers, and methods such as .cat.add_categories() and .cat.remove_unused_categories() let you tidy the label list independently of the data.

s = pd.Series(["S", "M", "L", "M"]).astype("category")

s.cat.categories               # Index(['L', 'M', 'S'])
s.cat.codes                    # 2, 1, 0, 1
s = s.cat.add_categories(["XL"])           # add a label nobody uses yet
s = s.cat.remove_unused_categories()       # drop labels with no rows

import pandas as pd

s = pd.Series(["S", "M", "L", "M", "S"]).astype("category")

print(list(s.cat.categories))
# ['L', 'M', 'S']

print(s.cat.codes.tolist())

# Add a brand-new size that no row uses yet
s = s.cat.add_categories(["XL"])
print(list(s.cat.categories))
# ['L', 'M', 'S', 'XL']

# ✅ Expected output:
# ['L', 'M', 'S']
# [2, 1, 0, 1, 2]
# ['L', 'M', 'S', 'XL']

3 Ordered Categories — Low < Medium < High

Some categories have a natural ranking that the alphabet gets wrong: sorting ["Low", "Medium", "High"] as plain text gives High, Low, Medium, which is nonsense. An ordered category teaches pandas the real sequence. Then comparisons like df["size"] > "Low" mean what you expect, and sort_values follows your order rather than the alphabet.

import pandas as pd
from pandas.api.types import CategoricalDtype

order = CategoricalDtype(["Low", "Medium", "High"], ordered=True)
sizes = pd.Series(["High", "Low", "Medium", "High"]).astype(order)

sizes > "Low"            # True, False, True, True
sizes.sort_values()      # Low, Medium, High, High  (logical order!)

import pandas as pd
from pandas.api.types import CategoricalDtype

order = CategoricalDtype(["Low", "Medium", "High"], ordered=True)
sizes = pd.Series(["High", "Low", "Medium", "High"]).astype(order)

print((sizes > "Low").tolist())

print(sizes.sort_values().tolist())
# ['Low', 'Medium', 'High', 'High']

# ✅ Expected output:
# [True, False, True, True]
# ['Low', 'Medium', 'High', 'High']

import pandas as pd

s = pd.Series(["dog", "cat", "dog", "bird", "cat", "dog"])

# 🎯 YOUR TURN:
# Convert this repetitive column to the memory-saving dtype.
# Fill in the blank with the right dtype name (in quotes).

s = s.astype("___")
print(s.dtype)
print(list(s.cat.categories))

# Expected output:
# category
# ['bird', 'cat', 'dog']

! Common Errors (And How to Fix Them)

❌ Assigning a value that is not a category

Setting a brand-new label on a category column produces NaN or an error:

s = pd.Series(["S", "M"]).astype("category")
s.iloc[0] = "XL"   # ❌ "XL" is not a known category

✅ Fix: add the category first:

s = s.cat.add_categories(["XL"])
s.iloc[0] = "XL"   # ✅

❌ Sorting an unordered category and expecting rank order

s = pd.Series(["High", "Low"]).astype("category")
s.sort_values()   # ❌ alphabetical: High, Low

✅ Fix: declare the order:

from pandas.api.types import CategoricalDtype
t = CategoricalDtype(["Low", "High"], ordered=True)
s.astype(t).sort_values()   # ✅ Low, High

🎯 Mini Challenge: Rank Survey Responses

Turn free-text ratings into an ordered category and sort respondents.

• Build the ordered type Poor < Fair < Good < Great
• Apply it to the rating column
• Keep only respondents rated better than Fair
• Sort the whole table by rating

import pandas as pd
from pandas.api.types import CategoricalDtype

df = pd.DataFrame({
    "name":   ["Ann", "Ben", "Cam", "Dee"],
    "rating": ["Good", "Poor", "Great", "Fair"],
})

order = CategoricalDtype(["Poor", "Fair", "Good", "Great"], ordered=True)
df["rating"] = df["rating"].astype(order)

# better than Fair
print(df[df["rating"] > "Fair"]["name"].tolist())
# ['Ann', 'Cam']

# sorted worst -> best
print(df.sort_values("rating")["name"].tolist())

# ✅ Expected output:
# ['Ann', 'Cam']
# ['Ben', 'Dee', 'Ann', 'Cam']

❓ Frequently Asked Questions

Lesson complete — your repeated text is now cheap!

You can convert with astype("category"), measure the memory win, drive the .cat accessor, and build ordered categories that compare and sort by real rank.

Practice quiz

Continue this course

• Previous: Data Types & astype
• Next: Dropping & Deduplicating Rows — Remove rows and duplicates with drop, drop_duplicates, and duplicated
• Quick reference: Pandas cheat sheet

Frequently asked questions

When should I use the category dtype?

Use it when a text column has many rows but only a small number of distinct values that repeat — think country, status, size, or department. The fewer unique values relative to total rows, the bigger the memory and speed win. For columns where almost every value is unique (like names or IDs), category gives no benefit.

How does category save memory?

Pandas stores each distinct value once in a small lookup table and replaces every cell with a tiny integer code pointing at that table. A million-row column with three possible values stops storing a million strings and stores a million one-byte codes plus three strings, which can cut memory use by 90% or more.

What is an ordered category?

By default categories are unordered labels. An ordered category records a meaningful sequence — like Low < Medium < High — so comparisons such as df['size'] > 'Low' work correctly and sorting follows your order instead of the alphabet. Create one with pd.Categorical(values, categories=[...], ordered=True).

What is the .cat accessor for?

Once a column is categorical, the .cat accessor exposes category-specific tools: .cat.categories lists the labels, .cat.codes shows the underlying integers, and methods like .cat.add_categories(), .cat.remove_unused_categories(), and .cat.reorder_categories() let you manage the category set without touching the data.