Programming vs. Training
How programming changed when people stopped writing every rule themselves.
For decades, programming meant dictating rules to a computer one by one: If the customer spends more than 100 euros, grant a discount. If the email contains the word "prize", flag it as spam. Then researchers asked a radical question: What if the computer could figure out the rules on its own?
This idea — that machines learn from examples instead of following instructions — transformed AI from a frozen research field into one of the most influential technologies of the 21st century. In this article, you will learn three things: why inverting the roles of rules and data changed everything, how the four building blocks of machine learning work together, and why the breakthrough happened in 2012 of all years.
The Inversion: From Rules to Data
In classical programming, a developer writes explicit rules that a computer executes. The programmer must understand the problem down to every detail and encode every edge case as an if-then statement. In machine learning, this process is completely reversed: the developer provides the computer with example data and the desired outcomes, and the machine discovers the underlying rules on its own.
Human writes rules. The computer executes those rules on new data and produces answers. The human must anticipate every edge case.
Human provides data and the expected answers. Training produces a model. This model generates answers for new data. The machine discovers the rules itself.
Definition
Machine Learning — When Computers Learn from Experience
Analogy
Definition
The apple-pear analogy has limits: it suggests ML works effortlessly. In reality, ML requires large amounts of labeled data, and it can learn spurious patterns (such as background color instead of fruit shape). ML finds statistical patterns, not causal relationships.
A concrete example: a rule-based spam filter blocks emails containing "click here" or "you won." Spammers bypass it with typos like "cl1ck h3re." An ML model trained on 100,000 labeled emails learns subtle statistical patterns — word combinations, formatting cues, metadata signals — that no human could ever write as an explicit rulebook.
Caution
Common Misconception: "ML Replaces Programming"
Machine learning does not replace classical programming — it extends it.
Data preprocessing, server infrastructure, user interfaces, and the ML pipeline itself remain tasks of classical software engineering. ML is a tool in the programmer's toolbox, not a replacement for the entire toolbox. The same goes for AI coding assistants that now write code themselves: they are ML tools that people program with. People decide what gets built and review the result.
Tom Mitchell, one of the pioneers of machine learning, provided a precise formal definition in 1997 that remains the gold standard today: a program learns if its performance P on a task T improves with experience E. The three letters mean:
E (Experience) = The training data the system learns from. For spam detection, this would be 100,000 emails labeled "spam" or "not spam."
T (Task) = The task the system should solve. For example: correctly classifying new emails as spam or not-spam.
P (Performance) = The measure used to evaluate success. For example: the percentage of correctly classified emails.
This formalization matters because it separates "learning" from "memorizing": a system that reproduces all training data but fails on new data has not learned according to Mitchell — because P does not improve on new data.
The Four Building Blocks of Machine Learning
Every ML system follows a four-stage pipeline. Think of preparing for an exam: the practice problems are your training data, your understanding is the model, studying is the training, and the actual exam is inference.
- 1. Training Data
- 2. Model
- 3. Training
- 4. Inference
Training data consists of the historical examples the system learns from — no ML system can learn without them; in supervised learning, which is what this is about, they are labeled. These could be 10,000 images of cats and dogs, each with the correct label attached. The quality of this data matters more than its quantity — a model trained on poorly labeled data learns the wrong patterns.
The model is a mathematical function with adjustable parameters — the weights. Initially, these weights are usually set at random, and the model produces nothing but nonsense. That changes in the next step.
During training, the model is shown data, makes a prediction, the error (loss) is measured, and the weights are adjusted via gradient descent. This cycle repeats hundreds to millions of times — just like studying for an exam, where you solve problems, recognize mistakes, and improve your understanding.
Inference is the moment of truth: the trained model is applied to new, unseen data. Like the exam itself, where you must solve problems you have never seen before.
These four steps interlock seamlessly — and this is exactly where the mathematics from previous paths comes in: the weight adjustment via gradient descent comes from calculus (derivatives), the computation inside the model from linear algebra. ML absolutely requires mathematics.
Example: House Price Prediction in Pseudocode
# Training data: House size -> Price
X = [50, 80, 120, 200]
Y = [150000, 240000, 360000, 600000]
# Model: Price = w * Size (one weight)
w = 0 # Initial value
# Training: 100 iterations
for _ in range(100):
for x, y in zip(X, Y):
prediction = w * x
error = y - prediction
w = w + 0.00001 * error * x
# Inference: New house, 150m²
print(w * 150) # Estimated priceDon't worry, you don't need to calculate these formulas yourself — the point is simply to see how the machine corrects itself step by step. In this minimal example, the model starts with w=0 (it knows nothing). After 100 passes, w approaches roughly 3,000, because the prices in the training data are exactly 3,000 times the floor area in square meters. The inference line uses the learned weight to estimate the price of a 150m² house.
Why 2012? The Three Factors
Machine learning theory had existed since the 1950s. Neural networks had been around for decades. Backpropagation was known since 1986. So why did everything explode in 2012? Because three factors were available simultaneously in sufficient measure for the first time.
ImageNet Dataset
14 M
Labeled images, launched in 2009 with 3.2M
NVIDIA CUDA
2007
GPU computing made programmable for ML
AlexNet Accuracy
84.7 %
ImageNet 2012 (top-5) — leap from ~74% to 84.7%
Imagine cooking an elaborate dish: the algorithms are the recipe (known at its core since the 1980s, decisively improved around 2010). The data are the ingredients (only available in mass with the internet era). The compute power is the professional kitchen (GPUs). In the 1980s, researchers had recipes but lacked both sufficient ingredients and a powerful enough kitchen. By 2012, everything was available simultaneously.
The turning point was the ImageNet competition in 2012. In the competition's two previous years, traditional image recognition methods had topped out at roughly 74% accuracy (correct answer among the five top guesses, "top-5"). Then a team entered with a deep neural network called AlexNet — and achieved 84.7%. This gain of more than ten percentage points was more than four times the progress from 2010 to 2011. AlexNet convinced the entire field that deep learning works.
AlexNet Success
The turning point for deep learning and modern AI. On September 30, 2012, the team submitted AlexNet to the ImageNet Challenge; when the results were published in October 2012, AlexNet had won by such a wide margin that computer vision was fundamentally changed. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton from the University of Toronto developed a CNN architecture that beat its competition by a notable 10.9 percentage points — an improvement considered extraordinary in the scientific community. With 60 million parameters and innovative techniques such as ReLU activations and dropout layers, AlexNet demonstrated the practical superiority of deep learning impressively. That was the moment when an interesting theory became a dominant technology. Yann LeCun called it 'an undeniable turning point in the history of computer vision.' The GPU-based implementation paved the way for modern AI development.
AlexNet was a deep neural network with eight layers — built specifically for image recognition. The key trick: a more efficient mathematical function (ReLU) replaced the previous ones (sigmoid, tanh), dramatically accelerating training.
The network was trained on two NVIDIA GTX 580 GPUs — graphics cards originally designed for gaming. Training took five to six days. Without GPUs, the same computation would have taken weeks or months on CPUs.
The impact was immense: within a year, the vast majority of teams in the ImageNet competition switched from handcrafted features (SIFT, HOG) to deep learning. The paradigm shift was not gradual — it was abrupt.
Caution
Common Misconception: "ML Suddenly Appeared in 2012"
Backpropagation has been widely known since 1986, neural network research from the 1950s. 2012 was not an invention but a convergence — the infrastructure finally caught up with the theory. ReLU (2010) and Dropout (2012) made deep networks trainable, ImageNet (2009) provided the data, and CUDA (2007) the compute power. Believing AI is an invention of the 2010s ignores decades of scientific work.
Interactive: 5 ML Myths Busted
Test your knowledge: click on a card to reveal the misconception. Each myth is based on a common misunderstanding about machine learning covered in this article.
Note
Key Takeaways
- Classical programming: Human writes rules, computer executes. Machine learning: Human provides data and outcomes, computer discovers the rules itself.
- Every ML system rests on four pillars: training data (examples), model (parameterized function), training (loss minimization via gradient descent), and inference (application to new data).
- The 2012 breakthrough required three ingredients simultaneously: massive datasets, GPU compute power, and stable training algorithms — the ingredients, the kitchen, and the recipe.
Now you understand WHY machine learning works. The next article introduces the first concrete learning paradigm: Supervised Learning — learning with a teacher.
Check
Quiz: Programming vs. Training
In classical programming, a developer provides rules and data, and the computer produces answers. What does the developer provide instead in machine learning with a teacher (supervised learning)?
Select one answer
1. In classical programming, a developer provides rules and data, and the computer produces answers. What does the developer provide instead in machine learning with a teacher (supervised learning)?
2. A company wants to build a system that detects defective products on a conveyor belt using camera images. Why is an ML approach more suitable than writing explicit rules?
3. You have a dataset of 500 apartment listings with size (m²) and price (EUR). You initialize a model with weight w=0 and run 50 training iterations. After training, your model predicts reasonable prices. A colleague asks you to predict the price of a new 90m² apartment. Which phase of the ML pipeline are you now in?
4. A researcher in 1990 had access to the backpropagation algorithm and a small dataset of 500 images. Why could this researcher NOT achieve results comparable to AlexNet (2012)?
5. A friend claims: "Machine learning will replace all programmers because computers can now write their own rules." Which response best corrects this misconception?
Check
Did you understand everything?
Can you explain this in your own words? Tick what you've got.
0 of 3 done