Neural Network Playground
Build your own neural network
Interactive demo
Layers: 2
What you can explore in this playground
- How stacking simple neurons in layers turns into recognising complex patterns
- What happens when you change architecture, learning rate or activation
- Why deep networks crack spirals where a flat model fails
- When a network overfits — memorising training points instead of learning the pattern
How to read the playground
- Decision heatmap: Blue = class 0, orange = class 1. The more saturated the colour, the more confident the prediction. Where it goes pale, the net is uncertain.
- Data points: Squares = class 1, circles = class 0. Click or drag to paint new points (Shift = class 0). Try drawing your own patterns!
- Loss curve below: Falling = the net is learning. Spikes = learning rate too high. Plateau = the net is stuck. Loss is the numeric measure of how wrong the net currently is.
- Architecture diagram: Glowing neurons = active. Pulsing edges = gradient is flowing during training. Greyed ✕-neurons = dead ReLU (no more learning signal).
- Numbers in the control panel: Steps = number of training steps (one mini-batch each). Loss = current error (smaller is better, 0 = perfect). Accuracy = share of correctly classified points. Parameters = total number of learnable weights in the net.
What you see on the field
The numbers tell you how good the network is — but not what its decision looks like. Here is what the coloured picture on the field is showing you.
- What you see
- The field is dotted with points in two colours — two classes, often in tricky patterns like rings, spirals or scattered islands. The coloured background beneath them shows how the network is currently splitting the field into two regions; the seam between the colours is the boundary it has learned.
- What happens
- At the start this boundary is rough and usually dead straight, and it fits the points badly. As training runs it bends and curves step by step, until it wraps snugly around the coloured groups of points.
- What you can do
- Pick a pattern (spiral, rings, XOR, linear), start the training or step through it one batch at a time, and add your own points by clicking or dragging. Rebuild the architecture — more layers and neurons — and watch how the boundary changes.
- What to watch for
- A network of several layers can learn crooked, nested boundaries — not just straight lines. Strip the layers away and the boundary stays a single straight line that can never capture a spiral.
Definition
What is a neural network?
Analogy
Picture a chain of bouncers. The first one looks at two simple features and decides: in or out. The second bouncer never sees the raw data — only the decisions of the first layer. The third works on the answers of the second, and so on.
Out of plain yes/no building blocks, a surprisingly nuanced judgement emerges. Each layer builds on the patterns of the previous one — combining them, refining them. That is exactly why deep networks can separate curved blobs like a spiral, where a single perceptron fails.
Definition
Mathematically a neural network is nothing mysterious: many small equations chained together. A single neuron computes a weighted sum of its inputs, adds a bias, and pipes the result through a non-linear activation function. A layer does this for many neurons in parallel; a Multi-Layer Perceptron (MLP) stacks several such layers.
Per layer: a⁽ˡ⁾ = σ(W⁽ˡ⁾·a⁽ˡ⁻¹⁾ + b⁽ˡ⁾) — read as “activation of the previous layer times weight matrix W, add bias b, pipe through activation σ”. Training adjusts W and b via backpropagation and gradient descent so that the error (loss) over the training data becomes minimal. The forward pass computes the prediction; the backward pass distributes the error back through the network using the chain rule.
You have almost certainly used a neural network multiple times today — most likely without noticing:
- Language modelsChatGPT, Claude, Gemini & friends
Huge neural networks with hundreds of billions of parameters. They predict word by word what is most likely to come next — the same idea you train here, just a thousand times larger.
- Image processingSmartphone camera
Face detection, night mode, portrait blur, auto-snap — neural networks deciding in milliseconds, right on your device.
- RecommendationsSpotify, YouTube, Netflix
“You might also like” — networks compare your behaviour with that of millions of others and spot patterns you couldn't articulate yourself.
- LanguageVoice assistants & DeepL
Siri, Alexa, automatic subtitles, machine translation — speech recognition and translation today run almost exclusively on neural networks.
- Quiet helpersMedicine, banking, security
Early cancer detection on X-rays, fraud detection in banking, spam filters, quality control on factory floors — usually better than any rule-based system.
Every training step runs three phases. Out of thousands of repetitions emerges what we call “learning”:
- 1Forward passCompute a prediction
The input flows forward through every layer. Each neuron computes a weighted sum of its inputs plus a bias and pipes it through the activation function.
The output of the final layer is the network's prediction — here a probability between 0 and 1.
- 2Backward passDistribute the error
Prediction minus true label = loss. Using the chain rule, the gradient of the loss with respect to every single weight is propagated backwards through the network.
Each weight then knows in which direction to move to reduce the error.
- 3Gradient descentNudge the weights
Every parameter is updated with w ← w − η·∇w: a small step in the gradient direction, scaled by the learning rate η.
Over thousands of mini-batches the network gradually shifts towards a clean decision surface.
The depth of the network is the secret: each layer builds on the patterns of the previous one. The non-linear activations bend the input space so that even spirals, circles or XOR become separable — patterns a single perceptron cannot crack. The catch: more layers and neurons allow more complex boundaries, but they also need more data. Otherwise the network just memorises the training points (overfitting).
If you only click Start, you miss the actual learning. These four short tasks make the most important concepts tangible:
- 1What do hidden layers even do?
- Set it upSpiral + Default preset (2×4). Click Start and watch the network learn a nicely curved boundary.
- What you seeNow switch off all hidden layers (Linear preset, 0 layers). No matter which activation you pick — you only get a straight line.
- The lessonOnly the combination of layer + non-linearity unlocks curved decision surfaces. Without hidden layers the network mathematically collapses to a single line.
- 2The magic of feature engineering
- Set it upCircles + Linear preset (0 hidden). With only x₁ and x₂ as input, the perceptron fails — a ring cannot be separated by a straight line.
- What you seeNow also enable x₁² and x₂² in the input features. Suddenly it works!
- The lessonThe squared features project the data into a space where it becomes linearly separable. That is exactly what deep networks do automatically in their hidden layers — they learn suitable features themselves.
- 3When the learning rate is too high
- Set it upXOR + Shallow preset. Set the learning rate to 0.3 (maximum) and hit Start.
- What you seeLoss jumps wildly, the boundary twitches — the network overshoots its target every time. Set the learning rate back to 0.03, Reset and Start: clean, steady learning.
- The lessonThe right step size is a balancing act: too high = chaotic, too low = painfully slow. In practice you'll feel your way between 0.001 and 0.1.
- 4Watch overfitting happen live
- Set it upSpiral, noise slider to 0.30, a fat net (e.g. 3 layers × 8 neurons). Train for 1500 steps.
- What you seeAccuracy on the visible points climbs — but the boundary becomes jagged and bulges around individual outliers.
- The lessonThe network has memorised noisy points instead of learning the spiral. The same thing happens to large models with too little data — they look perfect on training data and fall flat in the real world.
The training recipe
A Multi-Layer Perceptron does not learn in a single leap, but in thousands of tiny steps. Each step runs four phases, always in the same order:
- Forward pass: the input flows through every layer — at the end stands a prediction
- Loss: compare prediction vs. label and turn the gap into a single number
- Backward pass: use the chain rule to derive the gradient of the loss with respect to every weight
- Update: nudge every weight a tiny step in the right direction
Why does this work?
At its core, backpropagation is just an efficient application of the chain rule from calculus. Instead of computing a derivative for each of the thousands of weights individually, we propagate the error backwards through the network once and read off every gradient along the way.
The non-linearity (ReLU, Sigmoid, Tanh) is the second secret. Without it, no matter how many layers you stack, the network is mathematically identical to a single linear layer — and cannot learn curved decision boundaries.
Properties
- Universal: with enough layers and data, almost any function can be approximated
- Gradient-based: the learning rate η is the critical knob — too high blows up training, too low makes it crawl
- Data-hungry: more capacity needs more data, otherwise the network overfits
- Scalable: the same algorithm trains a toy net with 50 parameters and a language model with billions
Try it in the demo! Crank the learning rate to 0.3 and watch the loss curve bounce. Switch ReLU to Sigmoid and observe how deep networks suddenly almost stop learning — the infamous vanishing-gradient problem.
1
# Mini-batch training of an MLP
2
init: W[l], b[l] for every layer randomly (He init)
3
4
repeat until convergence:
5
batch = random_sample(data, batch_size)
6
7
# 1) Forward pass — predict per example
8
for each (x, y) in batch:
9
a[0] = x
10
for l = 1 .. L:
11
z[l] = W[l] · a[l-1] + b[l]
12
a[l] = activation(z[l]) # σ on the output
13
14
# 2) Loss — prediction vs. label
15
loss = BCE(a[L], y) # binary cross-entropy
16
17
# 3) Backward pass — gradients in reverse
18
delta[L] = a[L] - y # sigmoid + BCE → simple
19
for l = L .. 1:
20
∇W[l] += delta[l] · a[l-1]ᵀ
21
∇b[l] += delta[l]
22
delta[l-1] = (W[l]ᵀ · delta[l]) ⊙ σ'(z[l-1])
23
24
# 4) Update — gradient descent
25
for l = 1 .. L:
26
W[l] -= η · ∇W[l] / batch_size
27
b[l] -= η · ∇b[l] / batch_size
✨ Initialisation
Every weight starts random — usually with a normal distribution whose spread depends on layer size (He or Xavier init). Identical starting values would leave the network symmetric and unable to learn. Biases typically start at 0.
init: W[l], b[l] for every layer randomly (He init)✨ Init
Fill weights and biases with small random values.
→ Forward
Input flows layer by layer: z first, then activation.
📉 Loss
Compare prediction with label, compute the error.
← Backward
Use the chain rule to derive gradients for every weight.
✏️ Update
Nudge each weight by η · ∇w in the right direction.
🔄 Repeat
Next mini-batch — until the loss is small enough.
Check
Test your knowledge
Why do we need a non-linear activation function between layers at all?
Select one answer
1. Why do we need a non-linear activation function between layers at all?
2. What is the main difference between ReLU and Sigmoid?
3. What typically happens when you have many parameters and very few training points?
4. What does backpropagation actually do?
5. When does feature-engineering — adding x₁², x₂² or x₁·x₂ as inputs — help the network the most?