Q-Learning (Reinforcement Learning)

The method by which AI does what Pavlov would have tried if he had loved tables.

Nobody shows the explorer the way. Watch it work out the route from reward and penalty alone.

The Temple of Learning

Q-Learning training environment with grid world consisting of treasures, traps, and walls Grid with 5 columns and 5 rows containing 1 treasures and 1 traps
Use arrow keys to navigate the grid and spacebar to select cells
Izaac Jenson Data Crystal (+100 reward) Trap (-10 reward) Wall (impassable) Start position
SimulationSettingsStatisticsAgent Thoughts
Controls
Let's Go
Explorer Tools
Steps0Episode0Total Reward0.0Exploration30%

What happens on the grid

This demo shows how a figure finds its way to the goal purely through reward and punishment — not from instructions. Here is what you see on the board.

What you see
A grid of tiles seen from above. A start flag, a diamond as the goal, plus holes as traps and walls as obstacles. The figure stands on one tile. The tiles color in like a heat map.
What happens
From the goal outward, the warm color spreads across the tiles: whatever lies near the diamond is marked as valuable. In each tile a small arrow points in the currently best direction. Little by little the arrows line up into one continuous path from the start flag to the goal.
What you can do
Start or pause the training, step through it move by move, or change the speed. The explorer tools let you show the heat map, the arrows, the Q-values and the path. Click a tile to inspect it more closely.
What to watch for
Nobody shows the figure the way. It learns on its own, from reward and punishment, which direction pays off. Watch how the warm color and the arrows grow into a route by themselves.

Definition

Q-Learning Definition

Analogy

Imagine entering a huge department store for the first time, looking for the electronics section. You don't know the layout or the signage. What do you do? You simply start walking, try different directions, remember what worked, and become more targeted with each visit. After a few visits, you know not only the direct route but also alternative paths. That's exactly how our digital treasure hunter learns in the demo above: He starts completely clueless, gathers experience through trial and error, and develops a clever strategy over time. The special thing: He doesn't just learn "what is good," but also "what might be good in the future" – just like you remember that certain aisles in the store lead to interesting areas, even if they're not your direct destination.

Izaac Jenson enters the temple without a map or prior knowledge of its layout. Through careful exploration and learning from mistakes, he creates a "quality table" (Q-table) that captures the value of each possible action from each position. Watch as our algorithm archaeologist gradually discovers the optimal path to the Data Crystal while avoiding ancient traps.

Important Learning Parameters
  • Learning Rate (α): How quickly the agent updates its knowledge. Higher values mean faster learning but less stability.
  • Discount Factor (γ): How much the agent values future rewards compared to immediate ones. Values closer to 1 favor long-term planning.
  • Exploration Rate (ε): The probability of trying random actions instead of the current best choice. Essential for discovering new paths.
Understanding the Visualization

The demo provides rich visual feedback for understanding the learning process:

  • Q-Values: Color-coded arrows show the agent's learned preferences for each direction from each cell.
  • Color Coding: Green shows positive values (good actions), red shows negative values (actions leading to traps), and blue represents the treasure.
  • Agent Behavior: Observe the character's thought process in real-time as he makes decisions and updates his knowledge.
Experiment Tips
  • Start with default parameters to see basic learning, then adjust one parameter at a time.
  • Try high exploration rates for more diverse behavior, then lower them for more focused learning.
  • Reset the environment and change parameters to see how different settings affect learning speed and quality.
  • Use step-by-step mode to closely observe individual decision moments.

TheoryPseudo-CodeStep by StepFlow Diagram

After watching our digital treasure hunter in action, you might be interested in what exactly happens in his "brain." Q-Learning is far more sophisticated than it appears at first glance.

The secret lies in balance. Our agent must constantly switch between two strategies: trying new things (exploration) and using proven paths (exploitation). Too much curiosity leads to chaotic behavior, too little prevents discovering better routes.

Why is Q-Learning so special?

Unlike supervised learning, where we spoon-feed the algorithm the right answers, our Q-Learning agent must figure everything out himself. He gets only minimal information:

  • His current state (where am I right now?)
  • Possible actions (what can I do?)
  • Rewards or penalties after each action (was that good or bad?)

From these simple ingredients, the agent develops complex strategies. The fascinating part: He learns not only what is good, but also what might be good in the future.

The digital notebook: The Q-Table

Did you notice in the demo how our agent initially acts chaotically but becomes more targeted over time? This is due to his Q-table – a kind of digital notebook where he notes for each situation and action: "How good was this decision?"

When you display the Q-values in the demo, you see these numbers in real-time. Positive values mean "good idea," negative ones "better not." The more often an action in a situation leads to success, the higher its Q-value becomes.

The magic of future planning

The really clever thing about Q-Learning: It doesn't just think about the next step, but about the entire future. A path that seems laborious at the moment could lead to great later possibilities.

In the demo, you see this when the agent apparently takes "detours" – he has learned that certain positions are strategically valuable, even if they don't lead directly to the goal.

Why does this work so well?

Q-Learning combines three fundamental learning principles that you just observed in the demo:

  • Trial and Error: Like a child, the agent learns through experimentation
  • Reinforcement: Successful actions are preferred, bad ones avoided
  • Long-term planning: Not just the next step counts, but the overall goal

The result you see above: From chaotic initial movements emerges an elegant, targeted strategy over time. And all without explicit programming of the solution path – the agent discovered it himself.

Feel free to experiment further! Change the parameters in the configuration and observe how the learning behavior changes. Each setting tells its own story about the balance between curiosity and efficiency.

Check

Q-Learning Knowledge Test

Not completed

What best describes the exploration-exploitation dilemma in Q-Learning?

Select one answer

Answer Key: 1) B · 2) C · 3) C

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Q-learning: learning through trial and error

What is shown?

Here, an agent learns through trial and error. Agent means: a program that makes decisions. The agent navigates through a grid. It searches for treasures and avoids traps.

How does the demo work?

Press start training. Watch how the agent learns. It makes many mistakes at first. Over time, it finds the best path. You can adjust the speed.

What you learn from this?

This is called Q-learning. Q-learning means: learning through rewards and punishments. Good decisions are rewarded. Bad ones are punished. This is how the AI learns by itself.