Perceptron (Neural Networks)
Declared dead in 1957, today the basic building block of every deep learning model. In action here.
What is a Perceptron?
Imagine a bouncer at an exclusive club. He looks at a checklist with various criteria: Is the person on the guest list? Are they wearing appropriate clothing? Do they have an invitation?
Each criterion has a different importance (weight). The guest list might count more than the clothing. The bouncer adds up all the weighted points, and if the sum exceeds a certain threshold, he says 'YES, come in!' - otherwise 'NO, stay out!'. That's exactly how a perceptron works.
Analogy:
Imagine a bouncer at an exclusive club. He looks at a checklist with various criteria: Is the person on the guest list? Are they wearing appropriate clothing? Do they have an invitation?
Each criterion has a different importance (weight). The guest list might count more than the clothing. The bouncer adds up all the weighted points, and if the sum exceeds a certain threshold, he says 'YES, come in!' - otherwise 'NO, stay out!'. That's exactly how a perceptron works.
Definition:
The perceptron is a binary linear classifier developed by Frank Rosenblatt in 1957. It takes multiple inputs, multiplies each by a weight, adds these products together with a bias term, and applies a threshold function.
Mathematically: y = σ(Σ wᵢxᵢ + b), where σ is the activation function (typically a step function returning 1 if input > 0, else 0).
Interactive Perceptron
Welcome! A perceptron is the simplest neural network - a single artificial neuron. It takes inputs (x₁, x₂), weighs them (w₁, w₂), adds a bias (b), and then decides: 1 or 0.
What happens on the field
A single artificial neuron learns to separate two groups of points — just by moving a boundary. Here is what you see while it does.
- What you see
- A square field with points in two kinds: empty circles (○) and filled circles (●). A straight line runs across the field — the dividing line. It splits the field into two halves, one for each kind.
- What happens
- With every learning step the line tilts and shifts a little. When a point sits on the wrong side, the line turns toward it. Step by step the line moves into the position where both kinds are cleanly separated.
- What you can do
- Drag the line by its two handles with the mouse, or let the neuron learn on its own: step by step or automatically. A slider sets the pace, and you can switch between the tasks AND, OR and XOR.
- What to watch for
- This is how the simplest artificial neuron learns — not by thinking, but only by moving a single boundary. For XOR it never works: no straight line can separate those points.
Classification Plane
Classification Plane: 1/4 Correctly Classified, Accuracy 25%. w₁=0.50, w₂=0.50, The Neuron bias=0.00.
The Neuron
- w₁, w₂
- b
- Σ
- σ
Look at the neuron on the right: inputs flow in, get weighted and summed up.
How It Works
The perceptron is the simplest neural network - a single artificial neuron. It takes inputs, weighs them, sums them up, and then decides: Yes (1) or No (0).
The Weights (w₁, w₂)
Weights determine how important each input is. Positive weights strengthen, negative weights weaken the influence. The combination of weights determines the direction of the decision line.
The Bias (b)
The bias shifts the decision line. A positive bias makes it easier to output 1; a negative one makes it harder. It's like a 'default setting' for the neuron.
The Activation Function (σ)
The activation function makes the final decision: If the weighted sum is greater than 0, the neuron outputs 1 (fires). If it's less than or equal to 0, it outputs 0 (doesn't fire).
Perceptron Explained
The first artificial neuron
The perceptron is the ancestor of all neural networks. It is a mathematical model of a single neuron, invented in 1957 by Frank Rosenblatt. Despite its simplicity, it contains all fundamental concepts of machine learning: inputs, weights, summation, and activation.
Mathematical Foundation
The perceptron calculates a weighted sum of its inputs: z = w₁x₁ + w₂x₂ + ... + wₙxₙ + b. Here, x are the input values, w are the weights (which determine how important each input is), and b is the bias (a shift constant). The output is 1 if z > 0, otherwise 0.
What can the perceptron do?
- Linear Classification: Separation of two classes by a straight line (or hyperplane)
- Logical Functions: AND and OR can be perfectly represented
- Simple Pattern Recognition: Can distinguish between two simple patterns
The Fundamental Limitation
The perceptron can only solve linearly separable problems. The XOR problem (Exclusive Or) is the classic counterexample: The points lie diagonally and cannot be separated by a single straight line. This realization led to the AI Winter in 1969.
Learning Rule
The perceptron learns through the perceptron learning algorithm: For each misclassified point, the weights are adjusted: w_new = w_old + η × (y - ŷ) × x, where η is the learning rate, y is the expected and ŷ is the predicted result.
Try it yourself! Move the decision boundary in the demo and observe how the classification changes.
1943: The McCulloch-Pitts Neuron
Warren McCulloch and Walter Pitts laid the foundation with their paper "A Logical Calculus of Ideas Immanent in Nervous Activity." They showed that networks of simple neurons can compute any logical function.
1957: The Perceptron is Born
Frank Rosenblatt at the Cornell Aeronautical Laboratory developed the perceptron. The Mark I Perceptron machine was a hardware device with 400 photocells that learned to recognize letters. The New York Times enthusiastically titled: "Embryo of an electronic computer that will walk, talk, see, and write."
1969: The Book "Perceptrons"
Marvin Minsky and Seymour Papert published their influential book "Perceptrons," which mathematically proved the fundamental limitations of the perceptron. The XOR impossibility was a shock to AI research. Funding was cut - the AI Winter began.
The Legacy of the Perceptron
Only with the development of multi-layer networks and the backpropagation algorithm in the 1980s was the XOR limitation overcome. Every modern neural network builds on the principles of the perceptron - it is the building block of all deep learning architectures.
1
# The Perceptron - Simplest neural network
2
function perceptron(inputs):
3
# Two input values: x₁ and x₂
4
weights = [w1, w2] # How important is each input?
5
bias = b # Shift value (threshold)
6
7
for each input i in inputs:
8
# Calculate weighted sum
9
sum = sum + weights[i] * inputs[i]
10
11
# Activation function (step function)
12
if sum + bias > 0:
13
output = 1 # Neuron "fires"
14
else:
15
output = 0 # Neuron "doesn't fire"
16
17
return output
18
19
# Learning rule: Adjust weights
20
function train(x, expected_output, learning_rate):
21
for each training_sample:
22
prediction = perceptron(x)
23
error = expected_output - prediction
24
# Adjust weights towards the error
25
weights[i] = weights[i] + learning_rate * error * x[i]
26
bias = bias + learning_rate * error
⚙️ Initialize Weights
Start with random weights (w₁, w₂) and a bias (b). The weights determine how strongly each input influences the decision. The bias shifts the decision boundary.
weights = [w1, w2] # How important is each input?
bias = b # Shift value (threshold)
📥 Input
The perceptron receives input values (x₁, x₂). In the demo, these are the coordinates of a point. Each input represents a feature.
⚖️ Weighting
Each input is multiplied by a weight. Positive weights strengthen, negative weights weaken the influence. The weights are adjusted during learning.
➕ Summation
All weighted inputs are added together with the bias: z = Σ(wᵢ × xᵢ) + b. This sum is the net input to the neuron.
⚡ Activation
The step function (Heaviside) produces a binary output: 1 if z > 0, otherwise 0. This corresponds to the decision: Class 1 or Class 0.
📤 Output
The final classification: 0 or 1. In the demo, this means: blue area (○) or orange area (●). A binary decision.
Test Your Knowledge
What does a perceptron do?
1. What does a perceptron do?
- ☐ A) It stores large amounts of data
- ☐ B) It generates random numbers
- ☐ C) It separates data into two classes using a straight line
- ☐ D) It compresses images
2. What do the weights (w₁, w₂) do in a perceptron?
- ☐ A) They measure the learning speed
- ☐ B) They determine how important each input is for the decision
- ☐ C) They store the training data
- ☐ D) They count the number of inputs
3. Why can't a single perceptron solve the XOR problem?
- ☐ A) Because XOR has too many inputs
- ☐ B) Because the perceptron calculates too slowly
- ☐ C) Because XOR only works with colors
- ☐ D) Because the same classes lie diagonally and no straight line can separate them
4. In the demo, you moved the decision line. What happens to the classification when you rotate the line?
- ☐ A) Points can switch from one class to another
- ☐ B) The colors of the points change
- ☐ C) The points move along with the line
- ☐ D) New points appear
5. What was the historical significance of the XOR discovery?
- ☐ A) It directly led to ChatGPT
- ☐ B) It immediately made perceptrons popular
- ☐ C) It led to the 'AI Winter' - research funding was drastically cut
- ☐ D) It had no impact
Related Content
Article
The Spark: Activation Functions
The one small kink in the line without which neural networks wouldn't learn anything at all.
The AI Winters
Why artificial intelligence was written off multiple times without caring.
The Heart of Learning
How a neural network sends its errors home — and makes everything along the way a little better.
The Birth of AI
The early years of AI — full of optimism that lasted nearly 20 years.
The Chain Rule: Differentiating Nested Functions
A high school math rule that made the AI revolution possible in the first place.
Computer Vision (CNNs): How Machines Learned to See
How machines learned to read images and could suddenly tell dogs from cats — most of the time.
The Path to the Valley: Gradient Descent
How gradient descent finds the lowest point in a landscape with millions of hills — most of the time.
The Network
What the middle layers of a neural network learn — and can't explain to anyone.
The Machine's Knobs — Parameters vs. Hyperparameters
The fine line between "learned by the machine" and "guessed by you".
Linear & Logistic Regression
The mathematical foundation that every deep learning course only gets to after three hours.
How AI Measures Its Mistakes: Loss Functions
Loss functions are a model's pain threshold — without them, no learning.
Data Tables & Transformations (Matrices)
The 2D grid of numbers where every ML library primarily thinks.
Programming vs. Training
How programming changed when people stopped writing every rule themselves.
How Good Is Your Model? Metrics That Actually Matter
Evaluating models without self-deception — metrics that do more than just look good.
When the Model Memorizes (Overfitting)
How to notice that the model didn't learn but memorized.
Partial Derivatives & The Gradient
The gradient: the vector that every optimization follows — downhill, hopefully.
The Artificial Neuron
Frank Rosenblatt's 1958 idea that suddenly became relevant again more than 50 years later.
Time & Sequences (RNNs)
The architecture that gave AI memory — before someone found a better way.
Supervised Learning — Learning with a Teacher
Supervised Learning: the ML paradigm where someone diligently labeled things beforehand.
Tensors: The Language of GPUs
Tensors: the term that sounds more intimidating than the thing actually is.
Spaces and Directions (Vectors)
Why AI constantly works with arrows in n-dimensional space.
Demo
Convolutional Filters — the building blocks of CNNs
Interactive pixel workshop: pick an image, craft a 3×3 kernel and watch how values turn into edges, blur or sharpening.
Decision Tree
Interactive decision-tree demo: set points, tune depth, watch splits appear live and experience overfitting.
Gradient Descent
Interactive demo to understand gradient descent: click a starting point on the loss landscape, watch the algorithm roll into the valley, and experiment with learning rate and optimizers.
Naive Bayes (Classification)
Learn about the probabilistic classifier that detects spam emails
Neuroevolution
Watch virtual cars learn to drive using neural networks and genetic algorithms - or take the wheel yourself and challenge the AI
Neural Network Playground
Click layers and neurons together, choose a dataset and activation function, and watch the network learn to separate the data in real time.
Supervised Learning
Join Sharlock Helmes in his cleverest case: learning to distinguish between genuine clues and Moriarty's sophisticated red herrings. Elementary, my dear algorithm!