k-Nearest Neighbors (k-NN)

Tune the neighborhood: a data point asks its k closest acquaintances - and they vote.

LevelSwitches the dataset. Each level isolates exactly one effect - from the gentle Blobs to the Spiral, where only a middling k wins.
Goal: Make every ring green 0 / 0 rings correct

Interactive classification field. Use the left and right arrow keys to change k. Use the up and down arrow keys to step through the test rings one by one and hear each ring's neighbor vote. Escape clears the selection. With a mouse or touch you can paint points, erase, and grab test rings.

0 / 0 rings correct. k is 1

Tap a ring: its neighbors cast their votes.
Controls
Neighbours
How many neighbors get to vote. A small k follows every outlier (jagged), a large k smooths - but also erases real structure.
How many neighbors vote. Odd values only, so there's no tie.

Two clean groups, but one stray red point flips a ring at k=1. Nudge k up a little (k=3) and the majority already smooths the boundary.

Rings correct0 / 0Stars0Streak0
ToolFour tools for the field: paint blue or red training points, erase one, or grab a test ring to see its k neighbors and their vote.
Distance metricHow distance is measured. Euclidean is the straight line (round regions), Manhattan the city-block path (diamond regions) - so different points count as "near".
Level
Blue (class A)Red (class B)Test ringmisclassifiedThe colored area is the decision boundary: k-NN classifies every point of the field by its neighbors. It has no training - it forms live from the points you paint.

Each ring is a test point that wants to be classified correctly. Color the field so every ring glows green - using three tools: slide k, paint points, and grab a test point to see its neighbors.

What happens on the field

This demo sorts points by their neighbours. Here is what the picture shows — the spatial idea behind the numbers.

What you see
A square field full of known points in two colours, blue and red. The tinted background splits the field into a blue region and a red one, with a curved boundary running between them. Points drawn as rings are test points still waiting for their verdict.
What happens
Grab a ring and short lines reach out to its k nearest neighbours; the point takes on whichever colour is in the majority there. Push k up and the boundary between the coloured regions grows smoother; pull k down and it turns jagged, hugging single outliers.
What you can do
Pick a level, set k with the slider, paint new points in blue or red or erase them, grab a ring to inspect it, and switch the distance measure.
What to watch for
k-NN computes no formula. It simply looks at what lies nearby and takes a vote. The boundary between the colours is drawn by the neighbours alone — and k decides how calm or jittery it runs.

Definition

k-NN in one sentence

Analogy

You move to a new town and want to know whether your street is quiet or lively. You simply ask the k nearest neighbors - and go with whatever the majority says.

That is exactly how k-NN classifies: for a new point it finds the k most similar examples and lets them vote. No training, no formula - just the neighborhood.

k-NN classifies a new point in four steps - with no training at all, purely by comparing it to the closest examples:

  1. Measure distances: compute the distance from the new point to every training point (e.g. Euclidean - straight line - or Manhattan - city-block path).
  2. Pick the k closest: sort by distance and keep the k nearest neighbors. k is the one true dial.
  3. Let them vote: count which class is most common among those k neighbors.
  4. Majority wins: the most common class becomes the prediction. An odd k avoids ties.

That is exactly what you see below: grab a ring and its k neighbors, their distance lines and the running vote light up. Slide k and the majority changes live.

Lazy learning

k-NN never trains. The colored boundary only appears the moment you classify - every point simply asks its nearest neighbors.

Majority vote

Grab a ring: its k nearest neighbors light up, the lines show the distances, and the majority wins. That is the entire decision.

The k lever = bias-variance

A small k smooths nothing and follows every outlier (variance). A large k smooths everything, even real structure (bias). You just felt it yourself: too small breaks here, too large breaks there.

Check

Quick quiz - does k-NN stick?

Not completed

Why is k-NN called a "lazy learner"?

Select one answer

Answer Key: 1) B · 2) C · 3) A · 4) B · 5) C

TheoryHistoryPseudocodeStep by StepFlow Diagram

The idea: ask your neighbors

k-Nearest-Neighbors (k-NN) is one of the simplest classification algorithms there is. A new, unknown point simply gets the class that is in the majority among its k nearest neighbors. There is no formula, no model, no weights to train. The algorithm relies solely on the assumption that points lying close together usually belong to the same class.

Lazy learning: no training

k-NN is a lazy learner: during the training phase, nothing happens except that all training points are stored. All the work is done only at prediction time. That is why training is extremely fast, but each individual prediction is comparatively expensive, because it requires computing the distance to every stored point.

How a prediction works

  1. Measure distances: Compute the distance from the new point to every stored point.
  2. Choose the k nearest: Sort by distance and take the k closest neighbors.
  3. Vote: Every neighbor casts a vote for its class; the majority wins.

The distance measure (metric)

What close means is determined by the metric. The Euclidean distance measures the straight-line distance (square root of the sum of squared differences), the Manhattan distance only the paths along the axes. The choice of metric visibly changes the shape of the decision boundary. Important: features should be brought to a similar scale beforehand, otherwise one axis with large numeric values dominates the distance.

The role of k

The parameter k controls how smooth the decision boundary becomes. A small k (say k=1) follows every detail and reacts sensitively to outliers and noise (high variance). A large k averages over many neighbors and smooths the boundary, but can blur fine structures (high bias). There is no universally best k; the right value depends on the data and is usually found by trial and error or cross-validation. An odd k avoids ties with two classes.

Properties

  • Non-parametric: no assumption about the shape of the data
  • Instance-based: the training data itself is the model
  • Intuitive: the decision can be read directly off the neighbors
  • Expensive at runtime: every prediction checks all stored points

Play with the demo! Grab a test ring and slide the k control: at k=1, a single outlier can tip the ring into the wrong class; at a larger k, the majority overrules the troublemaker. On the spiral plane you see the opposite — a k that is too large blurs the fine arms, and here only a medium k helps. Also switch between Euclidean and Manhattan and watch how the boundary changes shape.

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Dots Ask Their Neighbors

What does the demo show?

Blue and red dots lie on a field. Dots with a ring are test dots. A test dot asks its k nearest neighbors. You choose the number k yourself. The majority decides its color.

How to use the demo

Slide k higher or lower. Draw new dots in blue or red. Grab a ring. Then you see lines to its neighbors. Your goal: All rings light up green.

What you learn

With a small k, the border follows every outlier. Outlier means: A dot lies right in the middle of the wrong color area. With a large k, the border gets smoother. The method is called k-NN. It stands for k nearest neighbors.