K-Means & Clustering

Reviewed & published by Brayan K

Discover hidden groups in unlabelled data. Learn how k-means iterates to convergence, how to choose k, and when DBSCAN or hierarchical clustering is the better tool.

Part of the free AI & Machine Learning course at LearnCodingFast — hands-on lessons with examples you run in your browser, plus practice exercises and a quick quiz.

What You'll Learn in This Lesson

• What unsupervised learning means
• The two-step k-means algorithm, by hand
• Choosing k with the elbow and silhouette methods
• How DBSCAN finds shapes and outliers
• What a hierarchical dendrogram gives you
• Why scaling matters for distance-based clustering

🎯 Real-World Analogy: Seating a Party

Imagine guests arriving at a party with no name tags. You want to seat people who get along together. You place a few empty tables (the centroids), send each guest to the nearest table, then nudge each table toward the middle of the guests who chose it. Repeat, and the tables drift to the natural social groups.

That is exactly k-means: assign each point to its nearest centroid, move each centroid to the average of its points, and repeat until nobody changes tables.

1 Unsupervised Learning & k-means

In unsupervised learning there are no labels — only features. The goal is to find structure. Clustering is the most common task: group points so that members of a group are similar to each other and different from other groups.

k-means is the classic clustering algorithm. You pick k, the number of clusters, then repeat two steps until convergence:

• Assign: send each point to its nearest centroid.
• Update: move each centroid to the mean of the points assigned to it.
• Repeat until assignments stop changing (convergence).

The smart k-means++ initialisation spreads the starting centroids far apart, which avoids bad local optima you can get from naive random seeds.

2 One Iteration, Step by Step

Let's run a single k-means iteration by hand: assign each point to its nearest centroid, then move each centroid to the mean of its points. Watch how the centroids jump toward the data.

# One full iteration of k-means by hand (plain numpy).
# Step A: assign each point to its NEAREST centroid.
# Step B: move each centroid to the MEAN of its assigned points.
import numpy as np

points = np.array([[1.0, 1.0], [1.5, 2.0], [8.0, 8.0], [9.0, 8.5]])
centroids = np.array([[0.0, 0.0], [10.0, 10.0]])   # two starting centroids

# Step A — assign: distance from every point to every centroid
def assign(points, centroids):
    labels = []
    for p in points:
        dists = [np.linalg.norm(p - c) for c in centroids]
        labels.append(int(np.argmin(dists)))       # index of nearest centroid
    return np.array(labels)

labels = assign(points, centroids)
print("assignments:", labels)        # [0 0 1 1] -> first two near c0, last two near c1

# Step B — update: each centroid becomes the mean of its points
new_centroids = np.array([points[labels == k].mean(axis=0) for k in [0, 1]])
print("new centroids:\n", new_centroids)

# Expected output:
# assignments: [0 0 1 1]
# new centroids:
#  [[1.25 1.5 ]
#  [8.5  8.25]]

3 Choosing k: Elbow & Silhouette

The hardest part of k-means is choosing k. Two standard tools help:

• Elbow method: plot the total within-cluster distance (inertia) for several k. It always drops as k grows, but at the right k the drop suddenly flattens — the "elbow".
• Silhouette score: for each point, compare how close it is to its own cluster versus the nearest other cluster. The average ranges from -1 to 1; pick the k with the highest score.

🌍 Worked Example: The Real Tool (scikit-learn)

Here's k-means with scikit-learn's KMeans, after scaling. Study it (it isn't runnable in the in-browser sandbox). Notice there is no y — only features — and that silhouette_score gives you a quality number to compare different k.

# WORKED EXAMPLE — what the real tool looks like (scikit-learn).
# You don't run this here; study it so the library code feels familiar.
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import silhouette_score

# Unlabelled feature data (no y!). Two natural blobs.
X = [[1, 1], [1.5, 2], [1, 0.5], [8, 8], [9, 8.5], [8.5, 9]]

X_scaled = StandardScaler().fit_transform(X)   # scale: k-means uses distance

km = KMeans(n_clusters=2, init="k-means++", n_init=10, random_state=0)
labels = km.fit_predict(X_scaled)              # discovers the groups

print("cluster labels:", labels)               # e.g. [1 1 1 0 0 0]
print("silhouette:", round(silhouette_score(X_scaled, labels), 2))

# Expected output (label numbers may flip):
# cluster labels: [1 1 1 0 0 0]
# silhouette: 0.8x  (close to 1.0 = well separated)

Cluster label numbers are arbitrary — only the grouping matters, so the same clusters might be labelled 0/1 or 1/0 on different runs.

4 DBSCAN & Hierarchical Clustering

k-means assumes round, similarly-sized blobs and forces you to pick k. Two alternatives lift those limits:

• DBSCAN: groups points that sit in dense regions, finds arbitrarily-shaped clusters, and labels sparse points as noise/outliers. You don't choose k — you set eps (neighbourhood radius) and min_samples.
• Hierarchical (agglomerative): repeatedly merges the two closest clusters, building a dendrogram tree. Cut it at any height to get the number of clusters you want — no k up front.

🎯 Your Turn 1: Assign to Nearest Centroid

Fill in the blank so nearest() returns the index of the closest centroid. The expected output is in the comments.

# 🎯 YOUR TURN — finish the 'assign to nearest centroid' step.
import numpy as np

points = np.array([[2.0, 2.0], [3.0, 3.0], [9.0, 9.0]])
centroids = np.array([[1.0, 1.0], [10.0, 10.0]])

def nearest(point, centroids):
    dists = [np.linalg.norm(point - c) for c in centroids]
    return int(___)            # 👉 return the INDEX of the smallest distance (np.argmin)

for p in points:
    print(nearest(p, centroids))

# ✅ Expected output:
# 0
# 0
# 1

🎯 Your Turn 2: Update a Centroid

A centroid is just the mean position of its points. Fill in the blank to compute it.

# 🎯 YOUR TURN — recompute a centroid as the mean of its points.
import numpy as np

cluster_points = np.array([[2.0, 4.0], [4.0, 6.0], [6.0, 8.0]])

# A centroid is just the average position of the points in its cluster.
new_centroid = cluster_points.____(axis=0)   # 👉 use .mean(axis=0)
print(new_centroid)

# ✅ Expected output:
# [4. 6.]

🎯 Mini-Challenge: Find the Elbow

Given pretend inertia values for several k, compute the drop at each step and print where the elbow is. Only a comment outline is provided.

# 🎯 MINI-CHALLENGE: pick k with a tiny elbow check
# The within-cluster error (inertia) always drops as k grows. The "elbow"
# is where it stops dropping much. You are given pretend inertia values.
#
# 1. inertia = {1: 100.0, 2: 40.0, 3: 35.0, 4: 33.0}
# 2. For each k from 2 to 4, compute the DROP from the previous k
# 3. print() each drop
# 4. The elbow is the k AFTER the last big drop -> print "best k = 2"
#
# ✅ Expected: big drop 60 from 1->2, then small drops -> elbow at k = 2

# your code here

5 Common Errors (And How to Fix Them)

These are the classic clustering pitfalls. Watch for them.

❌ Forgetting to scale before k-means

An unscaled large-range feature dominates the Euclidean distance and warps the clusters.

# ❌ income (0-100000) swamps age (0-100)
KMeans(n_clusters=3).fit(X_raw)

KMeans(n_clusters=3).fit(StandardScaler().fit_transform(X))

❌ Picking k by minimising inertia alone

Inertia always falls as k rises, so this just pushes k toward the number of rows.

best_k = max k that lowers inertia   # ❌ always picks the largest k

✅ Fix: use the elbow or the silhouette score:

silhouette_score(X, labels)   # pick the k with the highest score

❌ Using k-means on non-spherical clusters

k-means assumes round blobs, so it splits rings, crescents, and elongated shapes badly.

# ❌ two interleaved moons -> k-means cuts them in half
KMeans(n_clusters=2).fit(moons)

✅ Fix: use DBSCAN (or spectral clustering) for odd shapes:

DBSCAN(eps=0.3, min_samples=5).fit(moons)

📋 Quick Reference

🎉 Lesson Complete!

You can now run k-means by hand, choose k with the elbow and silhouette, and reach for DBSCAN or hierarchical clustering when the clusters aren't simple blobs.

Practice quiz

Continue this course

• Previous: Support Vector Machines
• Next: Dimensionality Reduction with PCA — Principal components, explained variance and when to reduce dimensions
• Quick reference: AI & Machine Learning cheat sheet

Frequently asked questions

What is clustering and how is it different from classification?

Clustering is unsupervised: it groups data points that are similar to each other without ever being told the right answer. Classification is supervised — it learns from labelled examples. With clustering you have only the features and you let the algorithm discover the groups (clusters) on its own.

How does the k-means algorithm work?

You choose k, the number of clusters. The algorithm places k centroids, then repeats two steps until things stop moving: (1) assign each point to its nearest centroid, and (2) move each centroid to the average position of the points assigned to it. After a few iterations the centroids settle and the assignments stabilise.

How do I choose the right number of clusters, k?

Two common tools. The elbow method plots the total within-cluster distance for several values of k and looks for the 'elbow' — the point where adding more clusters barely helps. The silhouette score measures how well each point fits its cluster compared to the nearest other cluster; you pick the k with the highest average silhouette. Domain knowledge also matters.

When should I use DBSCAN instead of k-means?

Use DBSCAN when your clusters are not round blobs, when you do not know how many clusters there are, or when you want the algorithm to flag outliers. DBSCAN groups dense regions and labels sparse points as noise. Its trade-off is that it is sensitive to its two parameters (eps and min_samples) and struggles when clusters have very different densities.

What is hierarchical clustering?

Hierarchical (agglomerative) clustering builds a tree called a dendrogram. It starts with every point as its own cluster and repeatedly merges the two closest clusters until one big cluster remains. You can 'cut' the tree at any height to get whatever number of clusters you want, and you do not have to commit to k in advance.

Do I need to scale features before clustering?

For distance-based methods like k-means and DBSCAN, yes. They use Euclidean distance, so a feature with a large numeric range would dominate the grouping. Standardising every feature to a comparable scale lets all features contribute fairly to the clusters.