Every machine-learning model starts the same way: multiply inputs by weights, add a bias, and check how wrong the prediction is. Linear regression does this for continuous values, logistic regression does it for classes — and together they form the blueprint that every neural network still follows today.

In the previous article, you learned what supervised learning is: data with labels, features, and a target value. Now you meet the first models that actually do something with them. The pattern you learn here — weighted sum, error measurement, non-linear activation — is exactly the skeleton of a neural network.

The Prediction Machine — Linear Regression

Imagine a real-estate agent who, after hundreds of deals, has developed a rough pricing formula: "3,000 euros per square metre." This formula condenses experience into a straight line. That is exactly what linear regression does: it finds the line that best fits the data.

Example

The agent uses unquantifiable context (neighbourhood feel, renovation state) that they cannot put into numbers. The model can only use features it has been given as input — everything else is invisible.

Analogy:

Imagine a real-estate agent who, after hundreds of deals, has developed a rough pricing formula: "3,000 euros per square metre." This formula condenses experience into a straight line. That is exactly what linear regression does: it finds the line that best fits the data.

Example

The agent uses unquantifiable context (neighbourhood feel, renovation state) that they cannot put into numbers. The model can only use features it has been given as input — everything else is invisible.

Definition:

A model that predicts a continuous value by computing a weighted sum of inputs plus a bias: ŷ = w₁·x₁ + w₂·x₂ + … + b. The model learns by adjusting weights and bias until predictions fit the data as closely as possible.

Formula: Linear Regression (univariate)

ŷ = w · x + b

w is the weight (the slope of the line), x is the input value, and b is the bias (the y-intercept). With multiple inputs, the formula becomes ŷ = w₁·x₁ + w₂·x₂ + … + b — instead of a line, it describes a plane or hyperplane.

A simple dataset: 50 m² → €150,000, 80 m² → €240,000, 120 m² → €360,000. The best-fit line: ŷ = 3,000 · x.

For a 100 m² apartment: ŷ = 3,000 · 100 = €300,000.

Suppose the actual price is €280,000. The residual (error) is: 300,000 − 280,000 = +€20,000. The model overestimates the price by €20,000.

The univariate formula y = mx + b is the teaching entry point. Real models almost always use multiple features simultaneously (multivariate). A house-price model might consider square metres, year built, floor level, and distance to city centre — each feature gets its own weight.

Interactive: How Training Cost Scales

Move the slider to see how computational cost of different solution methods grows with dataset size. Gradient descent scales linearly, the normal equation quadratically.

Input size n = 100

1010000

Prediction (1)1

Gradient Descent100

Normal Equation10.000

Moderate Input

At n=100, the difference becomes visible: O(n²) requires 10.000 operations, while O(n) needs only 100. O(log n) needs just 6.6 — that's 15x less than O(n).

Ratio to O(n)

Complexity	Operations	Factor vs. O(n)
Prediction (1)	1	100x faster
Gradient Descent	100	1x (Reference)
Normal Equation	10.000	100x slower

Keeping Score — Mean Squared Error

Imagine an archery target where the bull's eye is the true value and each arrow is a prediction. Arrows that land far from the centre count disproportionately because the "penalty area" (the square) grows with distance. An arrow that misses by twice as far does not cost twice as many penalty points — it costs four times as many.

Example

With outliers, a single wild arrow can dominate the entire score. In archery that is just bad luck, but with noisy data it can mislead the model.

Analogy:

Imagine an archery target where the bull's eye is the true value and each arrow is a prediction. Arrows that land far from the centre count disproportionately because the "penalty area" (the square) grows with distance. An arrow that misses by twice as far does not cost twice as many penalty points — it costs four times as many.

Example

With outliers, a single wild arrow can dominate the entire score. In archery that is just bad luck, but with noisy data it can mislead the model.

Definition:

MSE = (1/n) · ∑(yᵢ − ŷᵢ)². MSE averages the squared differences between actual and predicted values. It is not a property of the data but a scoring rule the model is judged by. Its best possible value is 0 (perfect predictions).

Formula: Mean Squared Error

MSE = (1/n) · ∑(y_i − ŷ_i)²

Why square the errors? Three reasons: (1) Without squaring, positive and negative errors cancel out — an error of +10 and one of −10 would average to zero. (2) Large errors are punished disproportionately: an error of 10 gives 100, but an error of 50 gives 2,500 — not 5 times more, but 25 times. (3) The mean makes MSE comparable across datasets of different sizes.

Three houses with prediction errors: +€10,000, −€5,000, and −€50,000.

Squared: 100M, 25M, and 2,500M. MSE = (100 + 25 + 2,500) / 3 = 875M.

The third error (−50,000) dominates the entire MSE — that is by design. Squaring ensures the model cannot ignore large outliers.

MSE (or related functions) is actually the navigation signal: the model adjusts its weights in the direction that shrinks this value. Without a loss function, the model has no compass — it would not know which direction to adjust its weights.

Imagine the squaring function as a round bowl: a ball always rolls smoothly and unambiguously to the lowest point — there is no doubt which way is downhill. Absolute values (|y − ŷ|) are like a V-shaped valley with a sharp kink at the bottom where the optimisation algorithm can stumble. That is why the smooth squaring function is preferred. Additionally, logistic regression uses Log Loss (Binary Cross-Entropy) rather than MSE as its loss function, because MSE produces too flat a signal with probabilities.

Interactive: Beat the Line

From Numbers to Decisions — Logistic Regression

Imagine a nightclub bouncer with an internal scoring system: appearance, behaviour, ID. He mentally calculates a score and converts it into a probability of entry. Unlike a real bouncer, the model has a precise, adjustable threshold that can be shifted for different contexts (e.g., stricter for medical diagnoses).

Example

A real bouncer decides by gut feeling and experience — their threshold is fuzzy and context-dependent. The model has a mathematically exact threshold that can be deliberately shifted. Also, the bouncer can read context like "guest is in a bad mood" — the model only sees the features it receives.

Analogy:

Imagine a nightclub bouncer with an internal scoring system: appearance, behaviour, ID. He mentally calculates a score and converts it into a probability of entry. Unlike a real bouncer, the model has a precise, adjustable threshold that can be shifted for different contexts (e.g., stricter for medical diagnoses).

Example

A real bouncer decides by gut feeling and experience — their threshold is fuzzy and context-dependent. The model has a mathematically exact threshold that can be deliberately shifted. Also, the bouncer can read context like "guest is in a bad mood" — the model only sees the features it receives.

Definition:

A classification model that computes the same weighted sum as linear regression, then passes it through the sigmoid function σ(z) = 1/(1+e⁻ᶻ) to produce a probability between 0 and 1. A threshold (typically 0.5) converts that probability into a class label.

Formula: Sigmoid Function

σ(z) = 1 / (1 + e^−z)

This z is exactly the result of the weighted sum from the previous step — the bouncer’s internally calculated score. The sigmoid function squeezes this number into the range between 0 and 1. For large positive z it approaches 1, for large negative z it approaches 0, and at z = 0 it gives exactly 0.5. This turns an unbounded weighted sum into a probability.

A spam classifier with three features: suspicious words, unknown sender, and urgency language.

Weighted sum z = 1.8. Sigmoid(1.8) ≈ 0.858.

Since 0.858 > 0.5, the model classifies the email as "Spam" with about 86% model-confidence. A hospital might lower the threshold to 0.3 to catch more at-risk patients — better to warn once too often than to miss a case.

The name comes from its statistical ancestry: logistic regression "regresses" on probabilities via the logit link. In practice, it is a classifier. Joseph Berkson coined the term "logit" in 1944 — the confusing name has stuck ever since.

Linear Regression

Output: continuous number (e.g., price in €). Use case: Regression (predicting values). Loss function: MSE.

Logistic Regression

Output: probability between 0 and 1. Use case: Classification (assigning classes). Loss function: Log Loss.

Interactive: The Sigmoid Formula Step by Step

Click on each term of the sigmoid formula to understand its role. The example shows a concrete spam classification with z = 2.5.

Sigmoid Formula

σ(z)1(1 + e⁻ᶿ)

σ(z) — Result

The output of the sigmoid function: a probability between 0 and 1. When σ(z) > 0.5, the model classifies as positive (e.g., spam); when σ(z) < 0.5, as negative.

Concrete Example

Given: Spam detection z = w·x + b = 0.5·5 + 0 = 2.5

Computation: z = w·x + b (Linear score) , σ(z) = 1/(1+e⁻ᶿ) (Sigmoid function)

1Compute exponential term: e⁻²⋅⁵ ≈ e⁻²⋅⁵ ≈ 0.082

2Compute denominator: 1 + 0.082 = 1 + 0.082 = 1.082

3Result: 1/1.082 ≈ 0.924 → Spam (> 0.5) 1/1.082 ≈ 0.924

The Big Picture — Bridge to the Artificial Neuron

What you learned in this article is more than three separate concepts. It is a chain that exactly matches the structure of an artificial neuron:

1

Weighted Sum Inputs × Weights + Bias → a number (just like linear regression)

2

Activation Function Pass the number through a non-linear function (e.g., sigmoid)

3

Loss Signal Compare prediction to true value and measure the error (e.g., MSE)

4

Weight Update Adjust weights in the direction that shrinks the loss

You already understand 80% of an artificial neuron — you just don't know it yet. In the next path (I.F: Deep Learning), you will meet the neuron formally — and recognise that it is exactly this chain, just with more layers.

Adrien-Marie Legendre published the method of least squares in 1805 — the mathematical foundation on which MSE rests. Francis Galton made "regression" famous as an analysis tool in the 1880s: he observed that extremely tall parents tended to have children of more average height — a "regression to the mean." The name stuck, even though it now means something entirely different. Joseph Berkson introduced the logit in 1944, inadvertently naming a classification method "regression" — a naming confusion that persists to this day.

A linear model predicts by multiplying each input by a weight and adding a bias — the result is a straight line (or hyperplane).
MSE measures prediction quality by averaging squared errors, deliberately punishing large misses harder than small ones.
Logistic regression wraps the same weighted sum in a sigmoid so the output stays between 0 and 1 — turning a regression machine into a classifier.
Weights + bias + activation function + loss = the recipe card for every neural network.

Comprehension Check

Explain the formula ŷ = w · x + b: What is w, what is b, and what happens with multiple inputs?
Calculate the MSE for errors +4, −2, and +6.
A logistic model outputs σ(z) = 0.42. The hospital uses a threshold of 0.3. Is the patient classified as at-risk?

In the equation ŷ = w · x + b — what does the term b represent?

The slope of the line

The prediction error

The bias (y-intercept)

The learning rate

1. In the equation ŷ = w · x + b — what does the term b represent?

☐ A) The slope of the line
☐ B) The prediction error
☐ C) The bias (y-intercept)
☐ D) The learning rate

2. A linear model predicts house prices: ŷ = 2,500 · x + 10,000. What does it predict for a 60 m² apartment?

☐ A) €150,000
☐ B) €160,000
☐ C) €140,000
☐ D) €170,000

3. Three predictions have errors of +4, −2, and +6. What is the MSE?

☐ A) 8/3
☐ B) 12/3
☐ C) 56/3
☐ D) 36/3

4. A logistic regression model outputs σ(z) = 0.72. A hospital uses a threshold of 0.60 to flag at-risk patients. What classification results — and what happens if the threshold is lowered to 0.40?

☐ A) Not at-risk with both thresholds
☐ B) At-risk with both thresholds
☐ C) Not at-risk at 0.60, at-risk at 0.40
☐ D) At-risk at 0.60, not at-risk at 0.40

5. Why does MSE square the errors instead of using absolute values?

☐ A) Squaring prevents positive and negative errors from cancelling and penalises large errors more heavily
☐ B) Squaring is computationally cheaper
☐ C) Squaring makes all errors equal
☐ D) Squaring is required by law in statistics

6. Your team needs to predict whether a customer will cancel their contract (yes/no). A colleague suggests linear regression. Why is logistic regression better suited?

☐ A) Linear regression cannot calculate errors
☐ B) Logistic regression is faster
☐ C) Linear regression outputs unbounded numbers that cannot be interpreted as probabilities — logistic regression produces values between 0 and 1 through the sigmoid function
☐ D) Linear regression needs more data

Answer Key: 1) C · 2) B · 3) C · 4) B · 5) A · 6) C

Linear & Logistic Regression

The Prediction Machine — Linear Regression

Linear Regression

Example

Analogy:

Example

Definition:

Example: House Prices

Misconception: Linear regression only works with one input

Interactive: How Training Cost Scales

Ratio to O(n)

Keeping Score — Mean Squared Error

Mean Squared Error (MSE)

Example

Analogy:

Example

Definition:

Example: Three Houses, Three Errors

Misconception: MSE is just a number you look at after training

Deep Dive: Why Not Just Absolute Values?

Interactive: Beat the Line

From Numbers to Decisions — Logistic Regression

Logistic Regression

Example

Analogy:

Example

Definition:

Example: Spam Classifier

Misconception: Logistic regression is a regression algorithm

Interactive: The Sigmoid Formula Step by Step

Concrete Example

The Big Picture — Bridge to the Artificial Neuron

Deep Dive: Historical Origins

Key Takeaways

Comprehension Check

Quiz: Linear & Logistic Regression

In the equation ŷ = w · x + b — what does the term b represent?

The Prediction Machine — Linear Regression

Linear Regression

Example

Analogy:

Example

Definition:

Example: House Prices

Misconception: Linear regression only works with one input

Interactive: How Training Cost Scales

Ratio to O(n)

Keeping Score — Mean Squared Error

Mean Squared Error (MSE)

Example

Analogy:

Example

Definition:

Example: Three Houses, Three Errors

Misconception: MSE is just a number you look at after training

Deep Dive: Why Not Just Absolute Values?

Interactive: Beat the Line

From Numbers to Decisions — Logistic Regression

Logistic Regression

Example

Analogy:

Example

Definition:

Example: Spam Classifier

Misconception: Logistic regression is a regression algorithm

Interactive: The Sigmoid Formula Step by Step

Concrete Example

The Big Picture — Bridge to the Artificial Neuron

Deep Dive: Historical Origins

Key Takeaways

Comprehension Check

Quiz: Linear & Logistic Regression

In the equation ŷ = w · x + b — what does the term b represent?

Related Content

Article

Supervised Learning — Learning with a Teacher

How Good Is Your Model? Metrics That Actually Matter

The Machine's Knobs — Parameters vs. Hyperparameters

The Raw Material: Data Engineering for Machine Learning

Bayes & Conditional Probability

Measures of Central Tendency: Where Is the Middle?

Measuring Change: Derivatives

Distributions: The Shape of Data

The Path to the Valley: Gradient Descent

The Network

How AI Measures Its Mistakes: Loss Functions

Programming vs. Training

When the Model Memorizes (Overfitting)

The Artificial Neuron

Probability & Expected Value

Spread of Data

Spaces and Directions (Vectors)

Demo

Decision Tree

Naive Bayes (Classification)

Perceptron (Neural Networks)

Supervised Learning

Glossary

Timeline