Filtering with Boolean Masks

Reviewed & published by Brayan K

A boolean mask is a Series of True/False values that pandas uses to keep only the rows where the condition is True — it's how you ask questions like "show me everyone older than 30" and get back exactly those rows.

Part of the free Pandas course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

Learn to write single conditions, combine them safely with & and |, and use the powerful helpers .isin(), .between(), and ~.

What You'll Learn in This Lesson

• What a boolean mask is and how pandas applies it
• Filter rows with df[df["age"] > 30]
• Combine conditions with & (and) and | (or)
• Why every condition needs parentheses
• Membership filtering with .isin([...])
• Negation with ~ and ranges with .between(a, b)

1 A Single Condition — df[df["age"] > 30]

Filtering happens in two conceptual steps. First, a comparison like df["age"] > 30 produces a boolean mask — a Series of True/False, one per row. Second, you pass that mask back into the DataFrame with df[mask], and pandas keeps only the True rows.

import pandas as pd

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol", "Dan"],
    "age":  [25, 35, 30, 45],
})

mask = df["age"] > 30      # Series: False, True, False, True
adults = df[mask]          # keep only True rows
# same as one line: df[df["age"] > 30]

import pandas as pd

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol", "Dan"],
    "age":  [25, 35, 30, 45],
})

# Step 1: build the mask
mask = df["age"] > 30
print(mask)
# 0    False
# 1     True
# 2    False
# 3     True
# Name: age, dtype: bool

# Step 2: apply the mask
print(df[df["age"] > 30])

# ✅ Expected output:
# 0    False
# 1     True
# 2    False
# 3     True
# Name: age, dtype: bool
#   name  age
# 1  Bob   35
# 3  Dan   45

2 Combining Conditions — & (and) and | (or)

To combine conditions, use the bitwise operators & for AND and | for OR. Each individual condition must be wrapped in parentheses.

# AND: age over 30 AND city is "NYC"
df[(df["age"] > 30) & (df["city"] == "NYC")]

# OR: city is "NYC" OR city is "LA"
df[(df["city"] == "NYC") | (df["city"] == "LA")]

import pandas as pd

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol", "Dan", "Eve"],
    "age":  [25, 35, 30, 45, 28],
    "city": ["NYC", "NYC", "LA", "LA", "NYC"],
})

# AND -> both conditions must be True (note the parentheses!)
print(df[(df["age"] > 28) & (df["city"] == "NYC")])

# OR -> either condition can be True
print(df[(df["city"] == "LA") | (df["age"] < 26)])

# ✅ Expected output:
#   name  age city
# 1  Bob   35  NYC
#     name  age city
# 0  Alice   25  NYC
# 2  Carol   30   LA
# 3    Dan   45   LA

3 Powerful Helpers — .isin(), ~, and .between()

Three helpers make common filters short and readable:

• .isin([...]) — keep rows where a column matches any value in a list. Much cleaner than chaining many | conditions.
• ~ (tilde) — inverts a mask. ~mask flips every True to False, giving you "NOT".
• .between(a, b) — keep rows where a numeric column falls in a range. By default both ends are inclusive.

df[df["city"].isin(["NYC", "LA"])]      # city in the list
df[~df["city"].isin(["NYC", "LA"])]     # city NOT in the list
df[df["age"].between(30, 40)]           # 30 <= age <= 40

import pandas as pd

df = pd.DataFrame({
    "name": ["Alice", "Bob", "Carol", "Dan", "Eve"],
    "age":  [25, 35, 30, 45, 28],
    "city": ["NYC", "Boston", "LA", "LA", "NYC"],
})

# .isin -> membership in a list
print(df[df["city"].isin(["NYC", "LA"])])
#     name  age city
# 0  Alice   25  NYC
# 2  Carol   30   LA
# 3    Dan   45   LA
# 4    Eve   28  NYC

# ~ -> NOT in the list
print(df[~df["city"].isin(["NYC", "LA"])])
#   name  age    city
# 1  Bob   35  Boston

# .between -> inclusive range
print(df[df["age"].between(28, 35)])
#     name  age city
# 1    Bob   35  Boston
# 2  Carol   30      LA
# 4    Eve   28     NYC

# ✅ Expected output:
#     name  age city
# 0  Alice   25  NYC
# 2  Carol   30   LA
# 3    Dan   45   LA
# 4    Eve   28  NYC
#   name  age    city
# 1  Bob   35  Boston
#     name  age    city
# 1    Bob   35  Boston
# 2  Carol   30      LA
# 4    Eve   28     NYC

import pandas as pd

df = pd.DataFrame({
    "name":  ["Pen", "Notebook", "Eraser", "Marker", "Glue"],
    "price": [1.50, 3.00, 0.75, 2.25, 1.10],
    "dept":  ["writing", "paper", "writing", "writing", "craft"],
})

# 🎯 YOUR TURN:
# Keep items in the "writing" dept AND priced under 2.00.
# Remember: parentheses around EACH condition, and use & not "and".

result = df[(df["dept"] == "writing") ___ (df["price"] < 2.00)]
print(result)

# Expected output:
#      name  price     dept
# 0     Pen   1.50  writing
# 2  Eraser   0.75  writing

! Common Errors (And How to Fix Them)

❌ "truth value of a Series is ambiguous"

Caused by using and/or on Series:

df[(df["age"] > 30) and (df["city"] == "NYC")]  # ❌

✅ Fix: use the bitwise & operator:

df[(df["age"] > 30) & (df["city"] == "NYC")]   # ✅

df[df["age"] > 30 & df["city"] == "NYC"]       # ❌ precedence error

✅ Fix: wrap each condition:

🎯 Mini Challenge: Filter a Sales Table

Build a sales DataFrame, then answer three questions with masks.

• Rows where region is North OR South
• Rows where units is between 50 and 150
• Rows that are NOT in the West region

import pandas as pd

sales = pd.DataFrame({
    "rep":    ["A", "B", "C", "D"],
    "region": ["North", "West", "South", "East"],
    "units":  [120, 40, 90, 160],
})

# 1) North or South
print(sales[sales["region"].isin(["North", "South"])])

# 2) units between 50 and 150 (inclusive)
print(sales[sales["units"].between(50, 150)])

# 3) NOT West
print(sales[~(sales["region"] == "West")])

# ✅ Expected output:
#   rep region  units
# 0   A  North    120
# 2   C  South     90
#   rep region  units
# 0   A  North    120
# 2   C  South     90
#   rep region  units
# 0   A  North    120
# 2   C  South     90
# 3   D   East    160

❓ Frequently Asked Questions

Lesson complete — you can ask your data questions!

You can build boolean masks, combine them safely with & and | (parentheses always!), and reach for .isin(), ~, and .between() when they fit.

Practice quiz

Continue this course

• Previous: Selecting Columns and Rows
• Next: loc vs iloc: Label vs Position — Master the two indexers — label-based loc and position-based iloc
• Quick reference: Pandas cheat sheet › Selecting & Filtering

Frequently asked questions

Why do I get an error when combining two conditions with 'and'?

Pandas requires the bitwise operators & (and), | (or), and ~ (not) for combining boolean Series — the Python keywords and/or/not only work on single True/False values, not arrays. Using 'and' raises a ValueError about ambiguous truth value.

Why must each condition be wrapped in parentheses?

The bitwise operators & and | have higher precedence than comparison operators like > and ==. Without parentheses, pandas tries to evaluate the & before the comparisons, which causes an error. Always write (df['a'] > 1) & (df['b'] < 5).

What is the difference between .isin() and chaining many == comparisons?

.isin([...]) checks membership against a list of values in one clean call and is far more readable than chaining multiple (df['col'] == x) | (df['col'] == y) conditions. It also scales well to long lists of allowed values.