Neural Network Playground

Build your own neural network

Picture a chain of bouncers. The first one looks at two simple features and decides: in or out. The second bouncer never sees the raw data — only the decisions of the first layer. The third works on the answers of the second, and so on.

Out of plain yes/no building blocks, a surprisingly nuanced judgement emerges. Each layer builds on the patterns of the previous one — combining them, refining them. That is exactly why deep networks can separate curved blobs like a spiral, where a single perceptron fails.

Analogy:

Definition:

Mathematically a neural network is nothing mysterious: many small equations chained together. A single neuron computes a weighted sum of its inputs, adds a bias, and pipes the result through a non-linear activation function. A layer does this for many neurons in parallel; a Multi-Layer Perceptron (MLP) stacks several such layers.

Per layer: a⁽ˡ⁾ = σ(W⁽ˡ⁾·a⁽ˡ⁻¹⁾ + b⁽ˡ⁾) — read as “activation of the previous layer times weight matrix W, add bias b, pipe through activation σ”. Training adjusts W and b via backpropagation and gradient descent so that the error (loss) over the training data becomes minimal. The forward pass computes the prediction; the backward pass distributes the error back through the network using the chain rule.

You have almost certainly used a neural network multiple times today — most likely without noticing:

Language modelsChatGPT, Claude, Gemini & friends
Huge neural networks with hundreds of billions of parameters. They predict word by word what is most likely to come next — the same idea you train here, just a thousand times larger.
Image processingSmartphone camera
Face detection, night mode, portrait blur, auto-snap — neural networks deciding in milliseconds, right on your device.
RecommendationsSpotify, YouTube, Netflix
“You might also like” — networks compare your behaviour with that of millions of others and spot patterns you couldn't articulate yourself.
LanguageVoice assistants & DeepL
Siri, Alexa, automatic subtitles, machine translation — speech recognition and translation today run almost exclusively on neural networks.
Quiet helpersMedicine, banking, security
Early cancer detection on X-rays, fraud detection in banking, spam filters, quality control on factory floors — usually better than any rule-based system.

All use the same core idea you are about to train yourself: many small compute units (“neurons”) stacked in layers, learning to recognise a pattern. What you train here in five seconds runs on thousands of GPUs for weeks inside ChatGPT — the maths behind it is surprisingly similar.

Every training step runs three phases. Out of thousands of repetitions emerges what we call “learning”:

1
Forward passCompute a prediction
The input flows forward through every layer. Each neuron computes a weighted sum of its inputs plus a bias and pipes it through the activation function.
The output of the final layer is the network's prediction — here a probability between 0 and 1.
2
Backward passDistribute the error
Prediction minus true label = loss. Using the chain rule, the gradient of the loss with respect to every single weight is propagated backwards through the network.
Each weight then knows in which direction to move to reduce the error.
3
Gradient descentNudge the weights
Every parameter is updated with w ← w − η·∇w: a small step in the gradient direction, scaled by the learning rate η.
Over thousands of mini-batches the network gradually shifts towards a clean decision surface.

The depth of the network is the secret: each layer builds on the patterns of the previous one. The non-linear activations bend the input space so that even spirals, circles or XOR become separable — patterns a single perceptron cannot crack. The catch: more layers and neurons allow more complex boundaries, but they also need more data. Otherwise the network just memorises the training points (overfitting).

What you can explore in this playground

How stacking simple neurons in layers turns into recognising complex patterns
What happens when you change architecture, learning rate or activation
Why deep networks crack spirals where a flat model fails
When a network overfits — memorising training points instead of learning the pattern

How to read the playground

Decision heatmap:Blue = class 0, orange = class 1. The more saturated the colour, the more confident the prediction. Where it goes pale, the net is uncertain.
Data points:Squares = class 1, circles = class 0. Click or drag to paint new points (Shift = class 0). Try drawing your own patterns!
Loss curve below:Falling = the net is learning. Spikes = learning rate too high. Plateau = the net is stuck. Loss is the numeric measure of how wrong the net currently is.
Architecture diagram:Glowing neurons = active. Pulsing edges = gradient is flowing during training. Greyed ✕-neurons = dead ReLU (no more learning signal).
Numbers in the control panel:Epoch = number of training steps. Loss = current error (smaller is better, 0 = perfect). Accuracy = share of correctly classified points. Parameters = total number of learnable weights in the net.

What you see on the field

The numbers tell you how good the network is — but not what its decision looks like. Here is what the coloured picture on the field is showing you.

Interactive demo

Loss: 0.500

Loss curve: if it dives, the net is learning fast. Spikes or plateaus = learning rate too high or net too small.

Tip: the spiral needs at least 2 layers with 4+ neurons each

Layers: 2

Dataset

The boss level: two intertwined arms. Needs depth (≥2 layers, ≥4 neurons each) and patience — around 1000 training steps.

Activation

Rectified Linear Unit: max(0, x). Positive values pass unchanged, negatives become 0. Fast, the default in deep networks. Caveat: individual neurons can “die” (✕ in the diagram) if the bias drifts too negative.

Learning rate 0.1

Step size for weight updates. Too high = loss jumps; too low = painfully slow.

Batch size 16

How many points are processed together in one training step. Bigger = steadier, smaller = noisier loss.

Noise 0.10

Scatter of the data points: crank it up and check whether the boundary stays stable — that's the real generalisation test.

Input features

Pre-computed inputs: x₁², x₁·x₂, sin(x₁). For shallow networks they decide what's separable at all. Deep nets learn them themselves.

Epoch0

Accuracy0%

Loss0.500

Parameters37

Score 0

S C X L

If you only click Start, you miss the actual learning. These four short tasks make the most important concepts tangible:

1What does an activation function even do?
1. Set it upSpiral + Default preset (2×4). Click Start and watch the network learn a nicely curved boundary.
2. What you seeNow switch off all hidden layers (Linear preset, 0 layers). No matter which activation you pick — you only get a straight line.
3. The lessonOnly the combination of layer + non-linearity unlocks curved decision surfaces. Without hidden layers the network mathematically collapses to a single line.
2The magic of feature engineering
1. Set it upCircles + Linear preset (0 hidden). With only x₁ and x₂ as input, the perceptron fails — a ring cannot be separated by a straight line.
2. What you seeNow also enable x₁² and x₂² in the input features. Suddenly it works!
3. The lessonThe squared features project the data into a space where it becomes linearly separable. That is exactly what deep networks do automatically in their hidden layers — they learn suitable features themselves.
3When the learning rate is too high
1. Set it upXOR + Shallow preset. Set the learning rate to 0.3 (maximum) and hit Start.
2. What you seeLoss jumps wildly, the boundary twitches — the network overshoots its target every time. Set the learning rate back to 0.03, Reset and Start: clean, steady learning.
3. The lessonThe right step size is a balancing act: too high = chaotic, too low = painfully slow. In practice you'll feel your way between 0.001 and 0.1.
4Watch overfitting happen live
1. Set it upSpiral, noise slider to 0.30, a fat net (e.g. 3 layers × 8 neurons). Train for 1500 epochs.
2. What you seeAccuracy on the visible points climbs — but the boundary becomes jagged and bulges around individual outliers.
3. The lessonThe network has memorised noisy points instead of learning the spiral. The same thing happens to large models with too little data — they look perfect on training data and fall flat in the real world.

Theory

Pseudocode

Step by step

Flow diagram

The training recipe

A Multi-Layer Perceptron does not learn in a single leap, but in thousands of tiny steps. Each step runs four phases, always in the same order:

Forward pass: the input flows through every layer — at the end stands a prediction
Loss: compare prediction vs. label and turn the gap into a single number
Backward pass: use the chain rule to derive the gradient of the loss with respect to every weight
Update: nudge every weight a tiny step in the right direction

Why does this work?

At its core, backpropagation is just an efficient application of the chain rule from calculus. Instead of computing a derivative for each of the thousands of weights individually, we propagate the error backwards through the network once and read off every gradient along the way.

The non-linearity (ReLU, Sigmoid, Tanh) is the second secret. Without it, no matter how many layers you stack, the network is mathematically identical to a single linear layer — and cannot learn curved decision boundaries.

Properties

Universal: with enough layers and data, almost any function can be approximated
Gradient-based: the learning rate η is the critical knob — too high blows up training, too low makes it crawl
Data-hungry: more capacity needs more data, otherwise the network overfits
Scalable: the same algorithm trains a toy net with 50 parameters and a language model with billions

Try it in the demo! Crank the learning rate to 0.3 and watch the loss curve bounce. Switch ReLU to Sigmoid and observe how deep networks suddenly almost stop learning — the infamous vanishing-gradient problem.

English

                      
                        
                          1
                          # Mini-batch training of an MLP
                        
                      
                        
                          2
                          init: W[l], b[l] for every layer randomly (He init)
                        
                      
                        
                          3
                           
                        
                      
                        
                          4
                          repeat until convergence:
                        
                      
                        
                          5
                              batch = random_sample(data, batch_size)
                        
                      
                        
                          6
                           
                        
                      
                        
                          7
                              # 1) Forward pass — predict per example
                        
                      
                        
                          8
                              for each (x, y) in batch:
                        
                      
                        
                          9
                                  a[0] = x
                        
                      
                        
                          10
                                  for l = 1 .. L:
                        
                      
                        
                          11
                                      z[l] = W[l] · a[l-1] + b[l]
                        
                      
                        
                          12
                                      a[l] = activation(z[l])    # σ on the output
                        
                      
                        
                          13
                           
                        
                      
                        
                          14
                              # 2) Loss — prediction vs. label
                        
                      
                        
                          15
                              loss = BCE(a[L], y)           # binary cross-entropy
                        
                      
                        
                          16
                           
                        
                      
                        
                          17
                              # 3) Backward pass — gradients in reverse
                        
                      
                        
                          18
                              delta[L] = a[L] - y          # sigmoid + BCE → simple
                        
                      
                        
                          19
                              for l = L .. 1:
                        
                      
                        
                          20
                                  ∇W[l] += delta[l] · a[l-1]ᵀ
                        
                      
                        
                          21
                                  ∇b[l] += delta[l]
                        
                      
                        
                          22
                                  delta[l-1] = (W[l]ᵀ · delta[l]) ⊙ σ'(z[l-1])
                        
                      
                        
                          23
                           
                        
                      
                        
                          24
                              # 4) Update — gradient descent
                        
                      
                        
                          25
                              for l = 1 .. L:
                        
                      
                        
                          26
                                  W[l] -= η · ∇W[l] / batch_size
                        
                      
                        
                          27
                                  b[l] -= η · ∇b[l] / batch_size

1 / 6

✨ Initialisation

Every weight starts random — usually with a normal distribution whose spread depends on layer size (He or Xavier init). Identical starting values would leave the network symmetric and unable to learn. Biases typically start at 0.

init: W[l], b[l] for every layer randomly (He init)

Why do we need a non-linear activation function between layers at all?

Without it a multi-layer network mathematically collapses to a single linear transformation.

It makes training faster.

It keeps the output between 0 and 1.

It is just a historical convention.

1. Why do we need a non-linear activation function between layers at all?

☐ A) Without it a multi-layer network mathematically collapses to a single linear transformation.
☐ B) It makes training faster.
☐ C) It keeps the output between 0 and 1.
☐ D) It is just a historical convention.

2. What is the main difference between ReLU and Sigmoid?

☐ A) ReLU is piece-wise linear and does not saturate on the upper end; sigmoid squashes everything into (0, 1) and saturates on both ends.
☐ B) ReLU only works on images, sigmoid only on text.
☐ C) Sigmoid is always faster than ReLU.
☐ D) There is no practical difference, just different notation.

3. What typically happens when you have many parameters and very few training points?

☐ A) Overfitting: the network memorises the training data but generalises badly to new points.
☐ B) The network stops learning entirely; the loss stays constant.
☐ C) The learning rate automatically halves itself.
☐ D) The loss is guaranteed to fall to 0 and stay there.

4. What does backpropagation actually do?

☐ A) It efficiently computes the gradients of the loss with respect to every weight using the chain rule.
☐ B) It initialises the weights of the network with random values.
☐ C) It picks the right dataset for training.
☐ D) It converts images or text into numeric inputs.

5. When does feature-engineering — adding x₁², x₂² or x₁·x₂ as inputs — help the network the most?

☐ A) For small networks with no hidden layers: the right features turn an 'unsolvable' problem into one that is linearly separable.
☐ B) Always — more features always means better results.
☐ C) Only when the dataset has at least 10,000 points.
☐ D) Only for very deep networks with 5+ layers.

Answer Key: 1) A · 2) A · 3) A · 4) A · 5) A

Neural Network Playground

What is a neural network?

Analogy:

Definition:

Where you encounter neural networks every day

How does training work?

Interactive demo

What you can explore in this playground

What you see on the field

Interactive demo

Layers: 2

Dataset

Four experiments you really should try