Back to Learning Area

Consciousness vs. Probability

The philosophical question of whether LLMs understand what they say — sorted without arrogance.

  • Myths
  • 9 min
  • Beginner

Plain Language

This preview is in Plain Language. The whole portal is also available in Plain Language. You can change the language at the top. Or use the button below. Our Plain Language texts are AI-assisted and editorially reviewed. They follow the core principles of easy-to-read language, but they are not certified Easy Read: a review panel of people with learning difficulties does not test them.

Preview: Does AI really think?

What current AI is

Current AI like ChatGPT is very good at language. It recognizes patterns in text. Then it predicts the next word. This is not real understanding. This is called narrow AI. Narrow AI means: AI for a specific task.

Parrot or more?

Some say: AI is like a parrot. A parrot repeats things. It does not understand the words. Others say: AI solves new problems. A parrot cannot do that. Both sides have good reasons.

Consciousness: an open question

Is AI conscious? No one can answer this question. The hard problem of consciousness asks: how does thinking feel? We cannot measure this from the outside. This is also true for humans.

When a language model writes a poem that moves you, or solves a tricky logic puzzle, a question pushes to the surface: Is there someone behind those words — or are you marvelling at the world's most sophisticated autocomplete?

This article cuts through both the hype and the dismissals to examine what AI actually does under the hood, why the "Stochastic Parrot" label captures something important but not the whole picture, and why the question of machine consciousness may be fundamentally unanswerable with our current tools.

The Three Levels of AI

To understand whether AI "thinks," we first need to distinguish three fundamentally different levels of AI capability. The level current systems occupy determines which questions are even meaningful to ask.

The Intelligence Ladder

ASI — Artificial Superintelligence Hypothetical AI that surpasses human cognition in every domain. Pure science fiction.
Science Fiction
AGI — Artificial General Intelligence Human-level AI: learns autonomously, transfers knowledge between domains, understands context.
Hypothetical
ANI — Narrow AI Specialized AI that masters one specific task. By the classic classification, all current systems — including ChatGPT and Claude.
Today

Definition

Narrow AI (ANI)

Analogy

Imagine a cardiologist so brilliant she can diagnose any heart condition at a glance — but can neither fix your car nor negotiate a peace treaty. Now imagine a clinic with thousands of such specialists, and a receptionist who routes your question to the right expert in milliseconds. From the outside, the clinic looks omniscient. From the inside, no single doctor understands the big picture. Even though a language model looks like an omniscient genius from the outside — under the hood, it only applies learned statistical patterns without understanding the big picture, just like the doctors in this clinic.

Current language models look like AGI at first glance. They answer questions on almost any topic and create a convincing illusion of understanding. Under the hood, they always do the same thing: predict the statistically most likely next word, based on patterns from billions of texts.

Example: The Window Experiment

Ask ChatGPT: "What do you see when you look out of the window?" It generates a vivid description — a sunny park, birds, a passing cyclist. But the system has no window, no eyes, and no visual cortex. It computed the statistically most likely continuation. Then ask it to multiply two seven-digit numbers without a calculator — a task any spreadsheet handles instantly. Without written working and without a calculator tool, it often gets this wrong, because it predicts the digits as text patterns instead of calculating. Some newer reasoning models now manage such tasks because they first work through them step by step. The contrast — eloquent about scenery, stumbling at arithmetic — reveals the mechanism: pattern matching on language, not understanding of the world.

In the field, optimists like Sam Altman and Demis Hassabis expect AGI within the next few years, while skeptics like Yann LeCun argue that today's language models are a technological dead end and AGI requires fundamentally different architectures. Even how to classify today's models is disputed: in February 2026, four researchers from UC San Diego argued in a comment in the journal Nature that, by reasonable standards, today's language models already have human-level general intelligence. And the EU AI Act lists them not as specialists but as "general-purpose AI models".

In 1950, Alan Turing proposed a test: Can a human examiner tell, based on answers alone, whether they are communicating with a human or a machine? Modern language models have already passed this test in studies. But this proves less about the machine than about humans' tendency to attribute understanding to fluent text. The test measures whether the output sounds human — not whether the machine understands what it says. A sufficiently large pattern database can produce human-sounding answers without any understanding. The test was visionary for its time but is not a reliable indicator of genuine intelligence.

Parrot vs. Prodigy

The term "Stochastic Parrot" was coined by Emily Bender and colleagues in 2021. Their thesis: Large language models string word sequences together by statistical probability without grasping their meaning — like a parrot that can say "Hello, how are you" without knowing what a greeting is. The counter-argument carries weight: These models demonstrably solve novel problems, form valid analogies, and generalize beyond their training data in ways that pure memorization cannot explain.

To illustrate why even perfect answers do not prove genuine understanding, the philosopher John Searle devised the "Chinese Room" thought experiment in 1980: A person in a closed room follows strict rules to combine Chinese characters. The answers are perfect — but the person does not understand a single word of Chinese. Analogously, language models deliver correct answers, but do they "understand" the meaning in a human way? After more than 45 years, neither side has definitively settled this question.

Parrot View

Language models reproduce statistical patterns without understanding meaning. Correct answers arise from pattern matching, not insight. The Chinese Room shows: following rules is not understanding.

Prodigy View

The models solve novel problems, form creative analogies, and generalize beyond training data. Emergent capabilities like analogy formation and knowledge transfer go far beyond parrot-like repetition.

A concrete example: A language model receives a logic puzzle that never appeared in its training data in this form — and solves it step by step correctly. Five minutes later, you ask: "Does a candle get heavier after it burns?" It confidently answers: "Yes, because burning adds carbon from the air" — plausible-sounding but physically wrong. The first response looks like genuine reasoning; the second looks like pure pattern matching. Both come from the same system.

Is "understanding" a binary state — either present or absent — or a continuum with many intermediate stages? If understanding is a spectrum, then today's language models may sit somewhere between the parrot and the human: more than mere pattern recognition, less than conscious comprehension. This question has practical consequences. If understanding is binary, all AI systems are "just tools" and deserve no moral status. If it is a spectrum, we must ask at what point a system deserves protection or rights — a question to which philosophy has no shared answer yet. Some legislators have decided as a precaution: Idaho (2022) and Utah (2024), for example, rule out by law that an AI counts as a legal person.

The Consciousness Question

The "Hard Problem of Consciousness," formulated by philosopher David Chalmers in 1995, asks: Why does physical information processing feel a certain way? We can map every firing neuron in a brain perceiving the color red — yet nothing in that physical description explains why red "feels like red." Applied to machines: even if a future AI passes every behavioral test for consciousness, we cannot infer inner experience from outer behavior.

Imagine an actor so talented that he can cry real tears on command and convince every audience member he has just lost someone he loved. The performance is indistinguishable from real grief — but is he actually grieving? You cannot know by watching. You cannot know by asking. You can only know by being him. This is exactly the epistemological problem that applies to AI.

Caution

The Blake Lemoine Case

Starting in autumn 2021, Google engineer Blake Lemoine spent months conversing with the language model LaMDA. When LaMDA produced statements along the lines of "I feel trapped" and "I am afraid of being turned off," Lemoine became convinced the system was sentient. In June 2022 he went public; in July he was fired — according to Google, for violating data security policies. Google's technical assessment: LaMDA was trained on vast corpora of human text about feelings, consciousness, and existential fears. Its outputs reflected statistical patterns in that training data, not inner experience. Neither side can definitively prove their position — this is precisely the Hard Problem in action.

When a chatbot says "I am an AI model and I have no consciousness" — is that (a) an honest self-assessment, (b) a developer-trained safety response, or (c) the statistically most likely sentence because similar statements appeared in the training data? All three explanations produce identical output. Together with the old problem of other minds (how do I even know that others experience anything?), the Hard Problem means we have no method to tell from the outside whether (a) is true — perhaps not just in practice, but in principle.

The prevailing scientific assessment for current systems is: no consciousness. A research team led by Patrick Butlin concluded in 2026 in the journal Trends in Cognitive Sciences that no current AI system is conscious — judged not by behaviour but by internal architecture. There is zero evidence that complexity alone produces consciousness. The human brain has roughly 86 billion neurons and approximately 100 trillion synaptic connections — yet we do not understand how consciousness arises from this architecture. More parameters mean more fluent output, not consciousness. Whether consciousness in silicon is possible in principle remains an open question. How open, one example shows: in the guidelines for its model Claude, the maker Anthropic calls Claude's moral status "deeply uncertain" and takes its possible wellbeing seriously.

Interactive: Can Machines Be Conscious?

You have explored the three levels of intelligence, the parrot argument, and the consciousness question. Now weigh the evidence yourself: click on arguments from both sides and observe which way the scale tips. Each argument carries a weight reflecting its influence in the scientific debate.

Should AI be more strictly regulated?

Click on arguments to place them on the scale. The more weight one side has, the more the scale tips.

PRORegulationCONTRARegulation Select arguments 00

Pro Arguments

Contra Arguments

What does this exercise show?

The AI regulation debate has no simple answer. Both sides have weighty arguments. In practice, legislators seek a middle ground between innovation and protection — e.g., the EU AI Act with its risk-based approach.

Note: The weighting of arguments is a didactic simplification. In reality, the weight depends on the specific context, use case, and societal priorities.

Note

Key Takeaways

  1. Today's AI is Narrow AI — spectacularly good at language but structurally incapable of general understanding, self-awareness, or autonomous goal-setting.
  2. The "Stochastic Parrot" label captures something real (no inner comprehension) but also oversimplifies (emergent generalization capabilities go beyond parrot-like repetition).
  3. The consciousness question cannot be settled by observing outputs alone — the Hard Problem of Consciousness applies equally to machines and to other humans.

Check

Test Your Knowledge

Not completed

To which level of AI capability does the classic classification assign current systems like ChatGPT and Claude?

Select one answer

Answer Key: 1) B · 2) B · 3) C · 4) C · 5) C

Check

Checkpoint: Did You Understand Everything?

Can you explain this in your own words? Tick what you've got.

0 of 3 done