Back to Learning Area

Programming vs. Training

How programming changed when people stopped writing every rule themselves.

  • Fundamentals
  • 8 min
  • Beginner

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Programming vs. learning

What is machine learning?

In the past, a human wrote rules for the computer. The computer followed these rules. Machine learning is different. Machine learning means: the computer learns the rules itself. For this, it needs many examples.

The difference from programming

In classic programming, the human writes all the rules. This costs a lot of time. The human must program new exceptions separately. In machine learning, the human only shows examples. The system finds the rules itself.

Machine learning in practice

An example: a spam filter. You could write rules: if the word win appears, then spam. But spammers bypass this with tricks. An AI learns from thousands of real spam emails. The AI then finds for itself what spam is.

For decades, programming meant dictating rules to a computer one by one: If the customer spends more than 100 euros, grant a discount. If the email contains the word "prize", flag it as spam. Then researchers asked a radical question: What if the computer could figure out the rules on its own?

This idea — that machines learn from examples instead of following instructions — transformed AI from a frozen research field into one of the most influential technologies of the 21st century. In this article, you will learn three things: why inverting the roles of rules and data changed everything, how the four building blocks of machine learning work together, and why the breakthrough happened in 2012 of all years.

The Inversion: From Rules to Data

In classical programming, a developer writes explicit rules that a computer executes. The programmer must understand the problem down to every detail and encode every edge case as an if-then statement. In machine learning, this process is completely reversed: the developer provides the computer with example data and the desired outcomes, and the machine discovers the underlying rules on its own.

Classical Programming

Human writes rules. The computer executes those rules on new data and produces answers. The human must anticipate every edge case.

Machine Learning

Human provides data and the expected answers. Training produces a model. This model generates answers for new data. The machine discovers the rules itself.

Definition

Machine Learning — When Computers Learn from Experience

Analogy

Imagine you want to teach someone to tell apples from pears. The classical approach: write a rulebook — "if round and red, then apple; if pear-shaped and green, then pear." These rules break on the first unusual apple. The ML approach: show 1,000 labeled photos and say "figure out the pattern yourself."

The apple-pear analogy has limits: it suggests ML works effortlessly. In reality, ML requires large amounts of labeled data, and it can learn spurious patterns (such as background color instead of fruit shape). ML finds statistical patterns, not causal relationships.

A concrete example: a rule-based spam filter blocks emails containing "click here" or "you won." Spammers bypass it with typos like "cl1ck h3re." An ML model trained on 100,000 labeled emails learns subtle statistical patterns — word combinations, formatting cues, metadata signals — that no human could ever write as an explicit rulebook.

Caution

Common Misconception: "ML Replaces Programming"

Machine learning does not replace classical programming — it extends it.

Data preprocessing, server infrastructure, user interfaces, and the ML pipeline itself remain tasks of classical software engineering. ML is a tool in the programmer's toolbox, not a replacement for the entire toolbox. The same goes for AI coding assistants that now write code themselves: they are ML tools that people program with. People decide what gets built and review the result.

Tom Mitchell, one of the pioneers of machine learning, provided a precise formal definition in 1997 that remains the gold standard today: a program learns if its performance P on a task T improves with experience E. The three letters mean:

E (Experience) = The training data the system learns from. For spam detection, this would be 100,000 emails labeled "spam" or "not spam."

T (Task) = The task the system should solve. For example: correctly classifying new emails as spam or not-spam.

P (Performance) = The measure used to evaluate success. For example: the percentage of correctly classified emails.

This formalization matters because it separates "learning" from "memorizing": a system that reproduces all training data but fails on new data has not learned according to Mitchell — because P does not improve on new data.

The Four Building Blocks of Machine Learning

Every ML system follows a four-stage pipeline. Think of preparing for an exam: the practice problems are your training data, your understanding is the model, studying is the training, and the actual exam is inference.

  1. 1. Training Data
  2. 2. Model
  3. 3. Training
  4. 4. Inference

Training data consists of the historical examples the system learns from — no ML system can learn without them; in supervised learning, which is what this is about, they are labeled. These could be 10,000 images of cats and dogs, each with the correct label attached. The quality of this data matters more than its quantity — a model trained on poorly labeled data learns the wrong patterns.

The model is a mathematical function with adjustable parameters — the weights. Initially, these weights are usually set at random, and the model produces nothing but nonsense. That changes in the next step.

During training, the model is shown data, makes a prediction, the error (loss) is measured, and the weights are adjusted via gradient descent. This cycle repeats hundreds to millions of times — just like studying for an exam, where you solve problems, recognize mistakes, and improve your understanding.

Inference is the moment of truth: the trained model is applied to new, unseen data. Like the exam itself, where you must solve problems you have never seen before.

These four steps interlock seamlessly — and this is exactly where the mathematics from previous paths comes in: the weight adjustment via gradient descent comes from calculus (derivatives), the computation inside the model from linear algebra. ML absolutely requires mathematics.

Example: House Price Prediction in Pseudocode

# Training data: House size -> Price
X = [50, 80, 120, 200]
Y = [150000, 240000, 360000, 600000]

# Model: Price = w * Size (one weight)
w = 0  # Initial value

# Training: 100 iterations
for _ in range(100):
    for x, y in zip(X, Y):
        prediction = w * x
        error = y - prediction
        w = w + 0.00001 * error * x

# Inference: New house, 150m²
print(w * 150)  # Estimated price

Don't worry, you don't need to calculate these formulas yourself — the point is simply to see how the machine corrects itself step by step. In this minimal example, the model starts with w=0 (it knows nothing). After 100 passes, w approaches roughly 3,000, because the prices in the training data are exactly 3,000 times the floor area in square meters. The inference line uses the learned weight to estimate the price of a 150m² house.

Why 2012? The Three Factors

Machine learning theory had existed since the 1950s. Neural networks had been around for decades. Backpropagation was known since 1986. So why did everything explode in 2012? Because three factors were available simultaneously in sufficient measure for the first time.

ImageNet Dataset

14 M

Labeled images, launched in 2009 with 3.2M

NVIDIA CUDA

2007

GPU computing made programmable for ML

AlexNet Accuracy

84.7 %

ImageNet 2012 (top-5) — leap from ~74% to 84.7%

Imagine cooking an elaborate dish: the algorithms are the recipe (known at its core since the 1980s, decisively improved around 2010). The data are the ingredients (only available in mass with the internet era). The compute power is the professional kitchen (GPUs). In the 1980s, researchers had recipes but lacked both sufficient ingredients and a powerful enough kitchen. By 2012, everything was available simultaneously.

The turning point was the ImageNet competition in 2012. In the competition's two previous years, traditional image recognition methods had topped out at roughly 74% accuracy (correct answer among the five top guesses, "top-5"). Then a team entered with a deep neural network called AlexNet — and achieved 84.7%. This gain of more than ten percentage points was more than four times the progress from 2010 to 2011. AlexNet convinced the entire field that deep learning works.

AlexNet was a deep neural network with eight layers — built specifically for image recognition. The key trick: a more efficient mathematical function (ReLU) replaced the previous ones (sigmoid, tanh), dramatically accelerating training.

The network was trained on two NVIDIA GTX 580 GPUs — graphics cards originally designed for gaming. Training took five to six days. Without GPUs, the same computation would have taken weeks or months on CPUs.

The impact was immense: within a year, the vast majority of teams in the ImageNet competition switched from handcrafted features (SIFT, HOG) to deep learning. The paradigm shift was not gradual — it was abrupt.

Caution

Common Misconception: "ML Suddenly Appeared in 2012"

Backpropagation has been widely known since 1986, neural network research from the 1950s. 2012 was not an invention but a convergence — the infrastructure finally caught up with the theory. ReLU (2010) and Dropout (2012) made deep networks trainable, ImageNet (2009) provided the data, and CUDA (2007) the compute power. Believing AI is an invention of the 2010s ignores decades of scientific work.

Interactive: 5 ML Myths Busted

Test your knowledge: click on a card to reveal the misconception. Each myth is based on a common misunderstanding about machine learning covered in this article.

Note

Key Takeaways

  1. Classical programming: Human writes rules, computer executes. Machine learning: Human provides data and outcomes, computer discovers the rules itself.
  2. Every ML system rests on four pillars: training data (examples), model (parameterized function), training (loss minimization via gradient descent), and inference (application to new data).
  3. The 2012 breakthrough required three ingredients simultaneously: massive datasets, GPU compute power, and stable training algorithms — the ingredients, the kitchen, and the recipe.

Now you understand WHY machine learning works. The next article introduces the first concrete learning paradigm: Supervised Learning — learning with a teacher.

Check

Quiz: Programming vs. Training

Not completed

In classical programming, a developer provides rules and data, and the computer produces answers. What does the developer provide instead in machine learning with a teacher (supervised learning)?

Select one answer

Answer Key: 1) B · 2) B · 3) D · 4) B · 5) B

Check

Did you understand everything?

Can you explain this in your own words? Tick what you've got.

0 of 3 done