Handling Imbalanced Data

Reviewed & published by Brayan K

In imbalanced classification one class vastly outnumbers the other — fraud, disease, and churn are rare. The trap: a model that always predicts the majority class can score 99% accuracy while catching nothing. The fix is to measure with precision, recall, F1, PR-AUC and ROC-AUC, and to rebalance with resampling, class weights, and threshold tuning.

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

This lesson shows why accuracy lies, which metrics to trust, and the practical toolkit — oversampling (SMOTE), undersampling, class_weight='balanced', threshold tuning, stratified splits, the imbalanced-learn library, and an anomaly-detection framing for extreme imbalance.

What You'll Learn in This Lesson

• Why accuracy is misleading on skewed classes (the 99% trap)
• Precision, recall, F1, PR-AUC and ROC-AUC
• Resampling: oversampling (SMOTE) and undersampling
• class_weight='balanced' and threshold tuning
• Stratified splits and the imbalanced-learn library
• Anomaly-detection framing for extreme imbalance

1 The accuracy trap — why 99% means nothing

When 99% of cases are negative, a model that simply predicts “negative” every time is 99% accurate — and completely useless, because it never finds the rare positives you actually care about. Same data, two stories:

always predict "not fraud"
accuracy = 99%  ✅ (looks great)
fraud caught = 0  ❌ (caught none)

A single number hides total failure on the class that matters.

recall = 0%   (caught no fraud)
precision = n/a
F1 = 0
=> the model is useless, exposed

Precision/recall/F1 immediately reveal the model found nothing.

2 The metrics that matter

These are the numbers to report on imbalanced problems. Pick the ones that match the cost of a miss vs a false alarm:

Tip: for very rare positives, PR-AUC is usually more informative than ROC-AUC, because ROC-AUC can stay deceptively high when negatives dominate.

3 Rebalancing — resampling, weights, thresholds

Once you measure honestly, you can fix the imbalance. The main levers, roughly from simplest to most involved:

• class_weight='balanced' — make errors on the rare class cost more during training (often the easiest first try).
• Oversampling (SMOTE) — synthesise new minority examples by interpolating between real ones.
• Undersampling — drop majority examples to even the ratio (discards data).
• Threshold tuning — move the probability cutoff (e.g. 0.5 → 0.3) to trade precision for recall.

model.fit(X, y)        # 99/1 split
predict at threshold 0.5
=> misses almost all positives

SMOTE() or class_weight="balanced"
tune threshold on validation
=> recall up, measured by F1/PR-AUC

Run this to see the accuracy trap with your own eyes — 99% accuracy, zero positives caught:

# 1000 cases: 990 negative, 10 positive (e.g. fraud). 99% are negative.
y_true = [0] * 990 + [1] * 10        # the real labels

# A lazy "model" that ALWAYS predicts negative (0).
y_pred = [0] * 1000

correct = sum(t == p for t, p in zip(y_true, y_pred))
accuracy = correct / len(y_true)

# How many actual positives did it catch?
caught = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)

print("accuracy:", accuracy)         # looks amazing...
print("positives caught:", caught, "out of 10")
# Output:
# accuracy: 0.99
# positives caught: 0 out of 10
# 99% accuracy, yet it found NOTHING. That's why accuracy lies here.

Now compute the metrics that actually expose model quality on rare classes:

# Precision, recall, and F1 by hand — the metrics that DON'T lie on
# imbalanced data. (10 real positives; the model flags 8, 6 of them right.)

tp = 6     # true positives  (flagged positive AND really positive)
fp = 2     # false positives (flagged positive but actually negative)
fn = 4     # false negatives (missed real positives)

precision = tp / (tp + fp)   # of what we flagged, how much was right?
recall    = tp / (tp + fn)   # of all real positives, how many did we catch?
f1 = 2 * precision * recall / (precision + recall)   # harmonic mean

print("precision:", round(precision, 2))
print("recall:", round(recall, 2))
print("f1:", round(f1, 2))
# Output:
# precision: 0.75
# recall: 0.6
# f1: 0.67

# 🎯 YOUR TURN — replace each ___ using the hint beside it.
from collections import Counter

labels = ["ok"] * 950 + ["fraud"] * 50
print("Class counts:", dict(Counter(labels)))
print("Fraud share: {:.1%}".format(labels.count("fraud") / len(labels)))

tp, fp, fn, tn = 10, 5, 40, 945

accuracy = (tp + tn) / (tp + fp + fn + tn)
# 1) Of everything you FLAGGED, how much was really fraud?
precision = tp / (tp + ___)                     # 👉 replace ___ with fp
# 2) Of all the real fraud, how much did you CATCH?
recall = tp / (tp + ___)                        # 👉 replace ___ with fn
# 3) The harmonic mean, so one good number cannot hide one bad one.
f1 = ___ * precision * recall / (precision + recall)   # 👉 replace ___ with 2

print("Accuracy:  {:.3f}".format(accuracy))
print("Precision: {:.3f}".format(precision))
print("Recall:    {:.3f}".format(recall))
print("F1:        {:.3f}".format(f1))

# 4) A model that never flags anything still scores this. That is the
#    whole problem with accuracy on a skewed set.
always_ok = labels.___("ok") / len(labels)      # 👉 replace ___ with count
print("A model that always says 'ok': {:.3f} accuracy, 0.000 recall".format(always_ok))

# 5) Undersampling: cut the majority down to the minority's size.
majority = [l for l in labels if l == "ok"][:___]   # 👉 replace ___ with 50
minority = [l for l in labels if l == "fraud"]
print("After undersampling:", dict(Counter(majority + minority)))

weights = {"ok": len(labels) / (2 * labels.count("ok")), "fraud": len(labels) / (2 * labels.count("fraud"))}
print("Class weights:", {k: round(v, 2) for k, v in weights.items()})

# ✅ Expected output:
# Class counts: {'ok': 950, 'fraud': 50}
# Fraud share: 5.0%
# Accuracy:  0.955
# Precision: 0.667
# Recall:    0.200
# F1:        0.308
# A model that always says 'ok': 0.950 accuracy, 0.000 recall
# After undersampling: {'ok': 50, 'fraud': 50}
# Class weights: {'ok': 0.53, 'fraud': 10.0}

4 Tools, stratified splits, and extreme imbalance

A few more essentials for real projects:

Stratified splits
• train_test_split(..., stratify=y) keeps the class ratio in each split
• without it, a tiny class can vanish from train or test by chance

imbalanced-learn (imblearn)
• SMOTE, RandomOverSampler, RandomUnderSampler
• imblearn Pipeline resamples ONLY the training folds, safely

Extreme imbalance (fraud: a few in millions)
• reframe as ANOMALY / outlier detection
• model "normal" behaviour, then flag deviations
• examples: fraud, rare disease, churn, defect detection

When positives are vanishingly rare, standard classification struggles — treating the problem as anomaly detection (learn what “normal” looks like and flag the unusual) is often the stronger framing.

📋 Imbalanced-data checklist

1. Never trust accuracy alone -> report precision, recall, F1, PR-AUC.
2. Split with stratify=y so every class is present in each fold.
3. Try class_weight="balanced" first (cheap, no synthetic data).
4. If needed, resample: SMOTE (over) or random under-sampling.
   -> do it INSIDE an imblearn Pipeline, on training folds only.
5. Tune the decision threshold to your costs (recall vs precision).
6. Extreme imbalance? Reframe as anomaly / outlier detection.

Common Imbalanced-Data Mistakes (and the fix)

• Reporting accuracy. — On a 99/1 split it is meaningless; report precision, recall, F1, and PR-AUC instead.
• Resampling before the split. — Oversampling the whole dataset leaks test info; resample inside cross-validation only.
• Forgetting to stratify. — Use stratify=y so a rare class isn't missing from a split.
• Leaving the threshold at 0.5. — Tune the cutoff to trade recall against precision for your costs.
• Blindly oversampling. — SMOTE can overfit; undersampling discards data — compare against class weights.
• Ignoring extreme skew. — For a handful of positives in millions, try an anomaly-detection framing.

⏱ Test Yourself — Timed Quiz

10 quick questions, 12 seconds each. Instant feedback — beat the clock!

🎉 Lesson Complete

• ✅ Accuracy is misleading on skewed classes — 99% can mean zero positives caught
• ✅ Measure with precision, recall, F1, PR-AUC, and ROC-AUC
• ✅ Rebalance with SMOTE, undersampling, or class_weight='balanced'
• ✅ Use stratified splits and tune the decision threshold
• ✅ For extreme imbalance, reach for an anomaly-detection framing

Practice quiz

Continue this course

• Previous: LLM Agents & Tool Use
• Next: Residual Networks (ResNet), DenseNets & Modern CNN Design — Understand skip connections, dense blocks, and modern CNN innovations
• Quick reference: AI & Machine Learning cheat sheet › Training & Evaluation
• From the blog: Data Preprocessing for Machine Learning

Frequently asked questions

Why is accuracy a bad metric for imbalanced classification?

Because a model can score very high accuracy while being useless. If 99% of transactions are legitimate, a model that always predicts 'legit' is 99% accurate yet catches zero fraud. Accuracy rewards getting the easy majority class right and hides total failure on the rare class you actually care about. Use precision, recall, F1, PR-AUC, or ROC-AUC instead.

What is the difference between precision and recall?

Precision answers 'of the cases I flagged as positive, how many really were positive?' — it punishes false alarms. Recall answers 'of all the real positives, how many did I catch?' — it punishes misses. There is usually a trade-off: pushing recall up (catching more positives) tends to lower precision (more false alarms). F1 (their harmonic mean) and PR-AUC summarise the balance.

How do oversampling, undersampling, and class weights differ?

Oversampling adds more minority examples (SMOTE creates synthetic ones by interpolating between real minority points); undersampling removes majority examples to balance the ratio; class_weight='balanced' leaves the data alone but makes mistakes on the rare class cost more during training. Oversampling risks overfitting, undersampling discards data, and class weights are often the simplest first thing to try.

What is threshold tuning and why does it help?

Classifiers output a probability, then apply a default 0.5 cutoff to decide the class. With imbalance, 0.5 is rarely optimal. Lowering the threshold catches more positives (higher recall, lower precision); raising it does the opposite. By tuning the threshold on a validation set — and looking at the precision-recall curve — you pick the operating point that matches your costs, such as catching as much fraud as possible while tolerating some false alarms.