Object Detection: YOLO, SSD & Faster R-CNN

A classifier outputs one label for the whole image: "cat". A detector must output a list of objects, and for each one it predicts three things:

• Class — what it is ("cat", "car", "person").
• Bounding box — where it is, as four numbers.
• Confidence — how sure the model is (0 to 1).

A bounding box is the rectangle drawn around an object. The most common format is the two corners: [x1, y1, x2, y2] — the top-left corner(x1, y1) and the bottom-right corner(x2, y2), all in pixels. (YOLO often uses a centre format[cx, cy, w, h] instead — same rectangle, different numbers.)

# One detection from a model, in corner format:
detection = {
    "class": "cat",
    "box":   [50, 50, 200, 200],   # x1, y1, x2, y2 in pixels
    "conf":  0.93,                 # 93% confident
}

How do you score a predicted box against the true box? You use IoU (Intersection over Union): the area where the two boxes overlap, divided by the total area they cover together.

Here is the full, commented version. Read it, then run it — the output is at the bottom.

# Intersection over Union (IoU) — the core detection metric
# IoU = overlap area / combined area. Ranges 0 (no overlap) to 1 (perfect).
# Each box is [x1, y1, x2, y2] = top-left corner and bottom-right corner.

def compute_iou(box_a, box_b):
    # Coordinates of the overlapping rectangle
    x1 = max(box_a[0], box_b[0])      # leftmost right edge
    y1 = max(box_a[1], box_b[1])      # topmost bottom edge
    x2 = min(box_a[2], box_b[2])      # rightmost left edge
    y2 = min(box_a[3], box_b[3])

    # Width/height of overlap (0 if the boxes do not touch)
    overlap_w = max(0, x2 - x1)
    overlap_h = max(0, y2 - y1)
    intersection = overlap_w * overlap_h          # shared area

    # Area of each box, then the union (avoid double-counting overlap)
    area_a = (box_a[2] - box_a[0]) * (box_a[3] - box_a[1])
    area_b = (box_b[2] - box_b[0]) * (box_b[3] - box_b[1])
    union = area_a + area_b - intersection

    return intersection / union if union > 0 else 0.0

ground_truth = [50, 50, 200, 200]     # the real object's box
prediction   = [60, 60, 210, 210]     # what the model guessed

iou = compute_iou(ground_truth, prediction)
print("IoU:", round(iou, 3))
print("Match (IoU >= 0.5)?", iou >= 0.5)

# Expected output:
# IoU: 0.658
# Match (IoU >= 0.5)? True

The two max(0, ...) calls are important: if the boxes don't overlap, x2 - x1 goes negative, and clamping it to 0 keeps the intersection at 0 instead of producing a fake positive area.

Predicting box coordinates from scratch is hard. Instead, many detectors start from anchor boxes: a fixed set of reference rectangles in a range of shapes and sizes (a tall one for people, a wide one for cars, a square one, and so on). The model doesn't invent a box — it picks the closest anchor and nudges its size and position to fit the object.

Think of anchors as templates printed on tracing paper. The model slides the best-matching template over the object and tweaks it slightly, which is far easier than drawing freehand.

There are two broad families of detector, and they trade speed against accuracy:

A good default: reach for YOLO when you need speed or real-time, and Faster R-CNNwhen accuracy on hard images matters more than frame rate.

A raw detector fires dozens of overlapping boxes around each object. Non-Maximum Suppression (NMS)tidies them up with a simple rule: sort boxes by confidence, keep the most confident one, then throw away every other box that overlaps it too much (IoU above a threshold, usually 0.5). Repeat until none are left.

Here is NMS written out in plain Python, reusing the IoU function. Run it and watch four boxes become two.

# Non-Maximum Suppression (NMS): keep the best box, drop overlapping duplicates.
# A detector often fires several boxes for ONE object. NMS cleans that up.

def compute_iou(box_a, box_b):
    x1 = max(box_a[0], box_b[0]); y1 = max(box_a[1], box_b[1])
    x2 = min(box_a[2], box_b[2]); y2 = min(box_a[3], box_b[3])
    intersection = max(0, x2 - x1) * max(0, y2 - y1)
    area_a = (box_a[2] - box_a[0]) * (box_a[3] - box_a[1])
    area_b = (box_b[2] - box_b[0]) * (box_b[3] - box_b[1])
    union = area_a + area_b - intersection
    return intersection / union if union > 0 else 0.0

# Each detection: (box, confidence, class)
detections = [
    ([48, 48, 202, 202], 0.95, "cat"),
    ([50, 52, 198, 200], 0.87, "cat"),   # overlaps the cat above
    ([52, 46, 205, 198], 0.72, "cat"),   # also the same cat
    ([300, 100, 400, 250], 0.91, "dog"),
]

def nms(dets, iou_threshold=0.5):
    # 1) Sort by confidence, highest first
    dets = sorted(dets, key=lambda d: d[1], reverse=True)
    kept = []
    for box, conf, cls in dets:
        # 2) Keep this box only if it does not overlap a kept box of the same class
        duplicate = any(
            cls == k_cls and compute_iou(box, k_box) > iou_threshold
            for k_box, k_conf, k_cls in kept
        )
        if not duplicate:
            kept.append((box, conf, cls))
    return kept

print("Before NMS:", len(detections), "boxes")
result = nms(detections)
print("After NMS: ", len(result), "boxes")
for box, conf, cls in result:
    print(f"  {cls}: conf={conf:.2f} box={box}")

# Expected output:
# Before NMS: 4 boxes
# After NMS:  2 boxes
#   cat: conf=0.95 box=[48, 48, 202, 202]
#   dog: conf=0.91 box=[300, 100, 400, 250]

One number summarises a detector's quality: mAP (mean Average Precision). For each class you measure precision across recall levels to get its Average Precision, then average across all classes. A box counts as correct only if its IoU with the truth clears a threshold.

• mAP50 — uses a single IoU threshold of 0.50 (more forgiving).
• mAP50-95 — averages over IoU 0.50 to 0.95; the headline COCO score. Higher means tighter boxes.

This is read-only (it needs the library and a dataset), but it shows how you'd measure mAP in practice:

# Evaluating a detector: mAP (mean Average Precision) with Ultralytics.
from ultralytics import YOLO

model = YOLO("yolov8n.pt")
metrics = model.val(data="coco128.yaml")   # run validation on a labelled set

print(f"mAP50:    {metrics.box.map50:.3f}")   # IoU threshold 0.50 only
print(f"mAP50-95: {metrics.box.map:.3f}")      # averaged over IoU 0.50..0.95

# Expected output (approximate):
# mAP50:    0.61
# mAP50-95: 0.45

# Rule of thumb when picking a detector:
#   1-stage (YOLO, SSD)   -> fast, great for real-time / edge / video
#   2-stage (Faster R-CNN)-> slower, stronger on small + crowded objects
# mAP50-95 is the headline number on the COCO leaderboard: higher = tighter boxes.