Back to Learning Area

Unsupervised Learning

Learning without an answer key — the more demanding but often more practical variant of machine learning.

  • Fundamentals
  • 8 min
  • Intermediate

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: AI learns without examples

What is unsupervised learning?

Sometimes the AI has no correct answers to learn from. It searches for patterns in the data itself. This is called unsupervised learning. Unsupervised learning means: learning without a teacher.

Clustering: finding groups

Clustering means: sorting similar things into groups. One method is called k-means. K-means automatically divides data points into groups. The human determines beforehand how many groups there should be.

Recognizing anomalies

One method is called anomaly detection. The AI first learns what is normal. Then it marks everything unusual. For example, it recognizes suspicious payments on a credit card.

Your streaming service groups movies into categories that no one taught it. Your bank blocks fraudulent transactions in milliseconds — even if no one ever showed the system this exact fraud pattern. How do machines discover hidden patterns in data when there are no answers to learn from?

After the previous articles on Supervised Learning, where models train on labeled data, we flip the approach. What happens when there are no labels, no correct answers, no teacher? Unsupervised Learning is how machines discover hidden structure in raw, unlabeled data — from customer segmentation to fraud detection. This article introduces three foundational techniques.

The Paradigm Without a Teacher

In Supervised Learning, the model has answers to learn from — the labels. In Unsupervised Learning, the algorithm must find patterns entirely on its own. This is not easier — it is fundamentally different.

Three pillars support this paradigm: clustering (finding groups), dimensionality reduction (finding the essence), and anomaly detection (finding outliers). Each technique tackles a different facet of the challenge, and each comes with trade-offs that the practitioner — not the machine — must navigate.

Clustering with k-Means

Definition

k-Means Clustering

Analogy

Imagine dumping a box of 200 mixed buttons on a table — different colors, sizes, materials — and asking a friend who has never seen buttons before to sort them into groups. Without any instructions about categories, the friend starts placing similar-looking buttons together: large wooden ones here, small shiny metal ones there. k-Means does exactly this in mathematical space — measuring distances between feature vectors instead of eyeballing appearance.

The Four-Step Algorithm

k-Means works in four clearly defined, repeating steps:

  1. 1. Choose k random starting positions as cluster centerscentroids
  2. 2. Assign each data point to the nearest centroid
  3. 3. Recalculate centroids as the mean of all assigned points
  4. 4. Repeat steps 2 and 3 until convergence is reached

Worked Example: Customer Segmentation

An online retailer analyzes 10,000 customers using three features: age, income, and purchase frequency. With k=4, k-Means discovers four customer profiles that were never provided as labels: budget-conscious students, young families with high purchase frequency, well-earning professionals, and affluent retirees. These groups emerge solely from geometric proximity in three-dimensional feature space — the algorithm has zero prior knowledge about life stages or purchasing behavior. Names like "budget-conscious students" are only assigned afterwards, by a person looking at the groups.

The k Problem: Who Decides the Number?

The biggest catch with k-Means: the machine cannot determine on its own how many natural groups exist in the data. You, the practitioner, must set the hyperparameter k in advance. Choose k too low, and distinct groups get merged. Choose k too high, and natural clusters get artificially split. The choice of k is a human decision, not a machine discovery.

A proven approach for selecting k is the Elbow Method. You compute the sum of squared distances from each point to its cluster center (Within-Cluster Sum of Squares, WCSS) for various k values — say from k=1 to k=10. Plotting WCSS against k typically produces a curve that drops steeply at first and then flattens out. The inflection point — the "elbow" — marks where additional clusters bring only marginal improvement. Imagine: at k=3 the curve drops sharply, at k=4 only slightly, at k=5 barely at all. Then k=3 or k=4 is a sensible choice. But be careful: the method is a heuristic, not a guarantee. Sometimes there is no clear bend, and you need domain knowledge.

Caution

Common Misconception

Myth: k-Means always finds the correct clusters.

Fact: The result heavily depends on the choice of k and the random initialization of centroids. Different starting positions can produce entirely different cluster assignments. The Elbow Method helps with choosing k but is a heuristic — not a guarantee. In practice, k-Means is therefore run several times from different starting points, keeping the result with the smallest sum of squared distances (WCSS); k-means++ also spreads out the starting points.

Interactive: k-Means step by step

Here k-Means runs on the practical example above: each point is a customer, placed by age (to the right) and income (upward) — two of the three features, so it fits on the screen. The large circles are the centroids; k = 4 is preset as in the example. Click “Next step” to walk through the four steps. Then try another k or “Other start points”: not every start ends in the same clusters.

Interactive: Hierarchical clustering & cut height

k-Means needs the number of clusters k up front. Hierarchical (agglomerative) clustering does not: it merges the points bottom-up into a tree — and only the height at which you cut that tree decides how many clusters come out. Try it.

Dimensionality Reduction with PCA

Definition

Principal Component Analysis (PCA)

Analogy

Imagine photographing a sculpture from many angles. Most photos from adjacent positions look nearly identical — they carry redundant information. PCA finds the few "best angles" that capture the most visual diversity of the sculpture, and discards the rest. You lose subtle details (the texture on the back), but the essential shape is preserved in far fewer images.

Connection to Eigenvectors

PCA builds on the vectors and matrices from Path I.C. The covariance matrix records how strongly each pair of features varies together. An eigenvector of this matrix is a direction that the matrix only stretches, never rotates — the stretch factor is called the eigenvalue. The algorithm computes the eigenvectors of the dataset's covariance matrix. These eigenvectors define the new axes in the space — the principal components. The corresponding eigenvalues indicate how much variance each axis captures. You keep the eigenvectors with the largest eigenvalues and discard the rest.

Dimensionality Reduction

784 → 50

MNIST: 784 pixel features compressed to ~50 principal components (94% reduction), while preserving digit distinguishability

Worked Example: MNIST Digits

The MNIST dataset contains 70,000 handwritten digits, each with 28×28 = 784 pixels. Many border pixels are almost always empty (pure background) — they carry no relevant information. PCA identifies that just about 50 principal components already capture about 82% of the variation (for 95% you would need around 150). The digits remain distinguishable, but the number of features drops by 94% – storage and computation time for subsequent classifiers drop considerably.

Why is the eigenvector with the largest eigenvalue the first principal component? Because the eigenvalues of the covariance matrix directly measure the variance along the corresponding eigenvector direction. The largest eigenvalue = the direction with the greatest data spread. The second principal component is orthogonal (perpendicular) to the first — it captures the second-largest spread not explained by the first. Because the axes are eigenvectors of the covariance matrix, the data's values along them are also uncorrelated: no information is double-counted. Linear algebra tells us: eigenvectors of a symmetric matrix can always be chosen orthogonal to each other — this is precisely why PCA works so elegantly.

Caution

Common Misconception

Myth: PCA reduces dimensions without losing information.

Fact: With real data, PCA practically always discards variance. The dropped components contained real data — just less of it than the ones you kept. The question is never "Do we lose information?" but "How much loss is acceptable for how much efficiency gain?" And if about 8% of the variance is missing, that does not mean 8% of the data points are missing: every point is kept, but each one loses its share along the discarded directions – and for individual points, that share can be exactly what matters.

Anomaly Detection

Definition

Anomaly Detection

Analogy

Think of a night security guard at a factory. Every night, the guard sees the same patterns: lights off at 10 PM, cleaning crew at 11 PM, silence until 6 AM. Nobody gave the guard a list of "suspicious events." Over weeks, the guard simply learns what normal looks like. When a door opens at 3 AM and footsteps head toward the server room — the guard flags it instantly. Not because they recognized a burglar, but because the event doesn't match the learned pattern of normality.

Two Approaches Compared

Distance-Based

Measures how far a point is from known clusters. Strength: Works well with spherical clusters. Weakness: Struggles with unevenly distributed data.

Density-Based

Measures how sparsely populated a point's neighborhood is. Strength: Detects anomalies even in irregularly shaped data. Weakness: Sensitive to the choice of neighborhood radius.

Class Distribution

99.9 %

In credit card fraud, more than 99.9% of all transactions are legitimate — the needle-in-a-haystack challenge

Worked Example: Credit Card Fraud

A credit card company monitors transactions. More than 99.9% are legitimate. The model learns each customer's purchase profile: German stores, 10–200 EUR range, daytime purchases. Then three rapid 5,000 EUR transactions come from a foreign country. These points are geometrically far from the learned profile and get flagged automatically — not because the system "knows" fraud, but because the transactions don't match the normal pattern.

Caution

Common Misconception

Myth: Unsupervised Learning is easier because you don't need labels.

Fact: The absence of labels makes evaluation extremely difficult. In Supervised Learning, you have ground truth to measure accuracy against. In anomaly detection, you often can't tell whether a flagged point is a true anomaly or a false positive — without human investigation.

Memory: Which Technique Matches Which Application?

Test your knowledge of unsupervised learning techniques. Find the matching pairs: each technique has a typical application. How quickly can you find all six pairs? DBSCAN, autoencoders and t-SNE do not appear in the article – the hints on the cards will help you.

0/ 6 Pairs
0Attempts
0:00Time

Note

Key Takeaways

  • k-Means groups data into k clusters — but k itself is a human decision, not a machine discovery.
  • PCA compresses data by keeping the axes of greatest variance — deliberately trading some information for massive efficiency gains.
  • Anomaly detection learns what "normal" looks like and flags everything that deviates — a paradigm that powers fraud detection, quality control, and cybersecurity.

Check

Knowledge Check: Unsupervised Learning

What must a practitioner specify before running k-Means that the algorithm cannot determine on its own?

Select one answer

Answer Key: 1) B · 2) B · 3) B · 4) D · 5) B · 6) B

Check

Checkpoint: Unsupervised Learning

Can you explain this in your own words? Tick what you've got.

0 of 3 done