Neuroevolution: Cars Learn to Drive
Learning without a teacher — it works when you don't mourn the bad attempts.
What is Neuroevolution?
Imagine a driving school where nobody teaches the students how to drive. Instead, hundreds of students get into cars simultaneously and just try their best.
After each round, the students who got the farthest are allowed to continue. Their driving strategies are combined and slightly modified for the next generation of students:
The best drivers pass on their instincts to the next generation
By combining strategies from two good drivers, even better approaches emerge
Small random changes sometimes lead to surprisingly clever maneuvers
Analogy:
Imagine a driving school where nobody teaches the students how to drive. Instead, hundreds of students get into cars simultaneously and just try their best.
After each round, the students who got the farthest are allowed to continue. Their driving strategies are combined and slightly modified for the next generation of students:
The best drivers pass on their instincts to the next generation
By combining strategies from two good drivers, even better approaches emerge
Small random changes sometimes lead to surprisingly clever maneuvers
Definition:
Neuroevolution combines artificial neural networks with evolutionary algorithms. Instead of training networks through backpropagation, their weights (and sometimes topologies) are optimized through an evolutionary process:
Neural Network: Each car is controlled by a small neural network that receives sensor inputs and outputs steering and acceleration commands
Fitness Function: Evaluates how well a car performs based on distance traveled, checkpoints reached, and time taken
Evolutionary Selection: The best-performing networks are selected as parents for the next generation
Weight Evolution: Instead of gradient descent, the connection weights of the neural networks are evolved through crossover and mutation
How the Demo Works
Overview
This interactive demo simulates neuroevolution: a population of cars, each controlled by its own neural network, learns to navigate a track. Over successive generations, the cars improve their driving through evolutionary selection, crossover, and mutation of their neural network weights.
Key Concepts
- Sensors:Each car has distance sensors that detect how far away the track walls are. These measurements serve as the inputs to the neural network.
- Neural Network:A small feedforward network that takes sensor readings and current speed as inputs and outputs steering angle and acceleration values.
- Fitness:A car's fitness counts the checkpoints it reaches, plus a small bonus for how close it is to the next one. Time does not count directly – but fast cars reach more checkpoints within a generation's time.
- Selection:After each generation, the cars with the highest fitness are selected as parents. Their neural network weights form the basis for the next generation.
- Mutation:Small random changes are applied to the neural network weights of offspring. This introduces variation and helps discover new driving strategies.
- Crossover:The weights of two parent networks are combined to produce a child network. This merges successful strategies from different parents.
- Elitism:The top-performing cars are copied unchanged into the next generation. This ensures the best solutions are never lost.
Interactive Neuroevolution
Generation 0. Best fitness 0. Diversity 0 percent.
- Generation
- 0
- Best Fitness
- 0
- Alive
- 0 / 30
- Elapsed
- 20.0s
What happens on the track
This demo teaches cars to drive — not by programming them, but through trial and inheritance. Here is what you see on screen.
- What you see
- A race track seen from above, with many cars driving at once. Each car feels its way forward with thin sensor whiskers that measure the walls. A small diagram in the bottom right shows the neural network of the car you are watching.
- What happens
- At first almost every car drives straight into a wall. The few that get furthest pass their brain on to the next generation — slightly mutated. Over many generations, more and more cars take the corners cleanly. In the champions race, the best cars from every generation compete at once.
- What you can do
- Start and pause the evolution, change the speed, follow a single car or click on it. In manual mode you drive a lap yourself and compare your time with the AI.
- What to watch for
- Nobody programmed the cars with rules for driving. The driving behaviour emerges from chance, selection and inheritance alone — generation after generation.
No champions yet. Run the evolution — every 5 generations the best brain is saved.
Neuroevolution Step by Step
- Create Population
A population of cars is created, each equipped with a randomly initialized neural network. These networks have random connection weights, so the cars initially drive erratically in all directions.
- Measure Fitness
Each car drives the track until it crashes or times out. Its fitness counts the checkpoints it passes, plus a small bonus for being close to the next one. Time does not count directly – but faster cars pass more checkpoints before time runs out.
- Selection
The cars with the highest fitness scores are selected as parents for the next generation. This mimics natural selection: only the fittest survive and reproduce.
- Crossover
The neural network weights of two parent cars are combined to create offspring. Some weights come from one parent, others from the second. This merges successful driving strategies.
- Mutation
Small random changes are applied to the weights of the offspring's neural networks. This introduces diversity and can produce novel behaviors that neither parent exhibited.
- Repeat
Steps 2 through 5 are repeated for each new generation. Over time, the population converges toward neural networks that drive the track effectively, sometimes discovering surprisingly creative strategies.
Deep Dive
The Sensor System
Each car is equipped with five distance sensors pointing in different directions: front, left 45 degrees, right 45 degrees, left 90 degrees, and right 90 degrees. These sensors cast rays from the car toward the track walls and measure the distance to the nearest wall. Short distances mean the wall is close, signaling the car to steer away. The sensor readings are the primary inputs to the neural network, giving it spatial awareness of the track ahead and to the sides.
The Neural Network
The neural network is a simple feedforward network with an input layer (sensor readings plus current speed and steering angle), one hidden layer, and an output layer with two neurons: one for steering and one for acceleration. Each connection has a weight that determines how strongly one neuron influences another. During evolution, these weights are what gets optimized - the network architecture stays fixed. The visualization shows active connections in brighter colors and thicker lines when they carry stronger signals.
The Fitness Function
Fitness is the measure of how well a car performs on the track. It counts the checkpoints the car passes (invisible lines across the track) and adds a small bonus for being close to the next one. There is no time bonus: fast cars simply pass more checkpoints before the generation ends. Cars that crash early receive low fitness scores. This fitness value determines which cars are selected as parents for the next generation - creating evolutionary pressure toward better driving.
Selection Pressure
Selection pressure determines how strongly fitness differences influence reproduction. High selection pressure means only the very best cars get to reproduce, leading to fast convergence but risking the loss of genetic diversity. Low selection pressure allows more cars to contribute to the next generation, maintaining diversity but slowing improvement. The balance is crucial: too much pressure leads to premature convergence on a suboptimal strategy, while too little pressure makes evolution inefficiently slow. In this demo, tournament selection is used - a small group of cars is randomly chosen, and the best one from that group becomes a parent.
Quiz: Understanding Neuroevolution
What do the car's sensors measure?
1. What do the car's sensors measure?
- ☐ A) The speed of other cars nearby
- ☐ B) The distance to the track walls
- ☐ C) The color of the road surface
- ☐ D) The fuel level of the car
2. What determines a car's fitness in this simulation?
- ☐ A) How fast the neural network computes
- ☐ B) The number of neurons in the network
- ☐ C) The number of checkpoints passed and time taken
- ☐ D) How smoothly the car steers
3. What does mutation do to the neural networks?
- ☐ A) Applies small random changes to the connection weights
- ☐ B) Completely rewires the network architecture
- ☐ C) Removes the worst-performing neurons
- ☐ D) Doubles the number of hidden layers
4. Why is elitism used in the evolutionary process?
- ☐ A) To make the simulation run faster
- ☐ B) To ensure the best solutions are preserved across generations
- ☐ C) To reduce the population size over time
- ☐ D) To prevent crossover from happening
5. What is neuroevolution?
- ☐ A) Training neural networks using backpropagation
- ☐ B) A type of reinforcement learning with rewards
- ☐ C) Evolving neural network weights through an evolutionary process
- ☐ D) Manually tuning neural network parameters
Neuroevolution Explained
Learning without a teacher, without a gradient
Neuroevolution trains neural networks using the principles of biological evolution: reproduction, selection and mutation. Instead of computing a gradient from an error and adjusting the weights via backpropagation, a whole population of networks is tried out. The most successful get to reproduce, the weak ones disappear. Over many generations, ever better networks emerge — without a gradient ever being computed.
The network in the car
In this demo, each network steers a car. The inputs are five distance sensors (rays that measure how far away the nearest wall is) plus speed and steering angle. A hidden layer with ReLU activation processes them, and two outputs steer the car: steering (via tanh, from left to right) and gas (via sigmoid, from zero to full). The network structure is the same and fixed for all cars.
Fitness instead of a loss function
Because there are no predefined correct answers, neuroevolution needs no differentiable loss function, just a fitness — a single number that measures success. Here, what counts is how far a car gets: every checkpoint passed and every completed lap increases the fitness. That is enough to act as evolution's compass; it never needs to know why a car drives better, only that it drives better.
The evolutionary cycle
- Selection (tournament): For each parent, three random networks are drawn and the fittest wins. Good networks get more offspring, weak ones keep a remaining chance.
- Elitism: The best networks (default: 2) are carried over unchanged, so a good driving style that has been found is never lost.
- Crossover: The child inherits each individual weight by a coin flip from one of the two parents.
- Mutation: Each weight is randomly nudged slightly with a small probability — the source of genuine novelty.
Weights, not structure
Important for classification: this demo evolves only the weights of a network with a fixed structure (classical neuroevolution). More advanced methods like NEAT additionally grow the topology — they add new neurons and connections over the course of evolution. Both are neuroevolution; the difference lies in whether only the weights evolve or the network's blueprint as well.
Strengths and limits
- No gradient needed: works even when the reward is non-differentiable or only known at the end.
- Highly parallelizable: all networks in a generation can be evaluated independently.
- Metaheuristic: there is no guarantee of finding the best possible network — usually a good, not the optimal, solution emerges.
- Noisy and slow: the fitness is only a sample, and without diversity the population can get stuck in a local optimum early.
Play with the demo! Turn the mutation rate up and watch how restless the population becomes; set the elite count to 0 and observe how good driving styles can get lost again. Watch the diversity indicator — if it drops toward zero, all networks resemble each other and evolution barely progresses anymore.
1980s: Network weights via genetic algorithm
Soon after the rediscovery of backpropagation, researchers experimented with an alternative. In 1989, Montana and Davis showed in Training Feedforward Neural Networks Using Genetic Algorithms that a network's weights can also be set purely through selection and mutation — without any gradient.
1994: Evolving the topology as well
Angeline, Saunders and Pollack presented GNARL, a method that evolved not only the weights but also the structure of recurrent networks — neurons and connections could appear and disappear. This gave birth to the idea of leaving the network's blueprint itself to evolution.
2002: NEAT
Kenneth Stanley and Risto Miikkulainen published NEAT (NeuroEvolution of Augmenting Topologies). It starts with minimal networks, lets them grow over generations, and protects new structures through speciation long enough for them to be optimized. NEAT became the most-cited neuroevolution method.
2017: Neuroevolution in the deep learning era
Neuroevolution returned at large scale. OpenAI showed in Evolution Strategies as a Scalable Alternative to Reinforcement Learning that evolution strategies can keep pace with classical reinforcement learning. At the same time, Uber AI evolved deep networks with millions of parameters under the term Deep Neuroevolution to master Atari games.
This demo
The demo shows the classic, easy-to-understand variant: a network with a fixed structure whose weights, through tournament selection, elitism, crossover and mutation, produce ever better drivers over many generations.
1
# Neuroevolution — training neural networks without backpropagation
2
# Topology fixed: 7 inputs -> 8 hidden (ReLU) -> 2 outputs
3
population = P random networks # only the weights, fixed structure
4
5
repeat every generation:
6
# 1. Evaluate: let each network drive one round
7
for each network in population:
8
while car is driving and time remains:
9
inputs = 5 sensor distances + speed + steering angle
10
steering, gas = feedforward(network, inputs)
11
move car; touches a wall -> car dies
12
# more checkpoints and laps yield more fitness
13
fitness = (laps * checkpoints + checkpoint) * 100
14
15
# 2. Rank: sort population by fitness descending
16
sort population by fitness
17
18
# 3. Elitism: keep the best E networks unchanged
19
next = copy the top E networks
20
21
# 4. Breed offspring until the population is full again
22
while count(next) < P:
23
# tournament selection: k random ones, the fittest wins
24
parent_a = tournament(population, k=3)
25
parent_b = tournament(population, k=3)
26
# crossover: each weight 50/50 from one parent
27
child = crossover(parent_a, parent_b)
28
# mutation: randomly nudge individual weights
29
child = mutate(child, rate, strength)
30
add child to next
31
population = next
32
33
# No gradient, no backprop — only selection, recombination, mutation
34
35
# Forward pass: sensor inputs -> control commands
36
function feedforward(network, inputs):
37
# hidden layer with ReLU activation
38
hidden = relu(inputs * weights_ih)
39
# outputs: steering via tanh, gas via sigmoid
40
steering = tanh(hidden * weights_ho_steer)
41
gas = sigmoid(hidden * weights_ho_gas)
42
return (steering, gas)
43
44
# Tournament selection: the best of k random ones
45
function tournament(population, k):
46
pick k random networks from the population
47
return the network with the highest fitness
48
49
# Mutation: adjust each weight with probability rate
50
function mutate(network, rate, strength):
51
for each weight w in network:
52
if random() < rate:
53
w = w + random(-1, 1) * strength
54
return the mutated network
🧬 Generating random networks
At the start there is a whole population of neural networks — in this demo, 30 cars by default. All weights are random, and the network structure is the same for all (7 inputs, 8 hidden neurons, 2 outputs). Generation 1 therefore drives aimlessly into the walls.
# Topology fixed: 7 inputs -> 8 hidden (ReLU) -> 2 outputs
population = P random networks # only the weights, fixed structure
🧬 Start
Create a population of random networks with a fixed structure.
🏎️ Evaluate
Each network drives; checkpoints and laps yield its fitness.
🏆 Select
Sort by fitness, keep the elite, and choose parents via tournament.
🔀 Reproduce
Crossover combines the parents, mutation brings in something new.
🔁 Next generation
The new population starts — over many rounds, the driving improves.
Related Content
Article
The Spark: Activation Functions
The one small kink in the line without which neural networks wouldn't learn anything at all.
The Heart of Learning
How a neural network sends its errors home — and makes everything along the way a little better.
The Path to the Valley: Gradient Descent
How gradient descent finds the lowest point in a landscape with millions of hills — most of the time.
The Network
What the middle layers of a neural network learn — and can't explain to anyone.
The Machine's Knobs — Parameters vs. Hyperparameters
The fine line between "learned by the machine" and "guessed by you".
How AI Measures Its Mistakes: Loss Functions
Loss functions are a model's pain threshold — without them, no learning.
Data Tables & Transformations (Matrices)
The 2D grid of numbers where every ML library primarily thinks.
The Artificial Neuron
Frank Rosenblatt's 1958 idea that suddenly became relevant again more than 50 years later.
Supervised Learning — Learning with a Teacher
Supervised Learning: the ML paradigm where someone diligently labeled things beforehand.
Demo
Evolution (Optimization)
Interactive demonstration of evolutionary optimization with mutation, selection, and crossover operators
Gradient Descent
Interactive demo to understand gradient descent: click a starting point on the loss landscape, watch the algorithm roll into the valley, and experiment with learning rate and optimizers.
Neural Network Playground
Click layers and neurons together, choose a dataset and activation function, and watch the network learn to separate the data in real time.
Perceptron (Neural Networks)
Discover the first artificial neuron - the Big Bang of machine learning from 1957.
Q-Learning
Interactive demonstration of the Q-Learning algorithm with an intelligent agent in the Temple of Learning
Travelling Salesman: Algorithms in Competition
Place cities, draw your own route, and pit Greedy, Simulated Annealing and a Genetic Algorithm against you.