Consciousness vs. Probability
The philosophical question of whether LLMs understand what they say — sorted without arrogance.
When a language model writes a poem that moves you, or solves a tricky logic puzzle, a question pushes to the surface: Is there someone behind those words — or are you marvelling at the world's most sophisticated autocomplete?
This article cuts through both the hype and the dismissals to examine what AI actually does under the hood, why the "Stochastic Parrot" label captures something important but not the whole picture, and why the question of machine consciousness may be fundamentally unanswerable with our current tools.
The Three Levels of AI
To understand whether AI "thinks," we first need to distinguish three fundamentally different levels of AI capability. The level current systems occupy determines which questions are even meaningful to ask.
The Intelligence Ladder
Definition
Narrow AI (ANI)
Analogy
Definition
Current language models look like AGI at first glance. They answer questions on almost any topic and create a convincing illusion of understanding. Under the hood, they always do the same thing: predict the statistically most likely next word, based on patterns from billions of texts.
Example: The Window Experiment
Ask ChatGPT: "What do you see when you look out of the window?" It generates a vivid description — a sunny park, birds, a passing cyclist. But the system has no window, no eyes, and no visual cortex. It computed the statistically most likely continuation. Then ask it to multiply two seven-digit numbers without a calculator — a task any spreadsheet handles instantly. Without written working and without a calculator tool, it often gets this wrong, because it predicts the digits as text patterns instead of calculating. Some newer reasoning models now manage such tasks because they first work through them step by step. The contrast — eloquent about scenery, stumbling at arithmetic — reveals the mechanism: pattern matching on language, not understanding of the world.
In the field, optimists like Sam Altman and Demis Hassabis expect AGI within the next few years, while skeptics like Yann LeCun argue that today's language models are a technological dead end and AGI requires fundamentally different architectures. Even how to classify today's models is disputed: in February 2026, four researchers from UC San Diego argued in a comment in the journal Nature that, by reasonable standards, today's language models already have human-level general intelligence. And the EU AI Act lists them not as specialists but as "general-purpose AI models".
In 1950, Alan Turing proposed a test: Can a human examiner tell, based on answers alone, whether they are communicating with a human or a machine? Modern language models have already passed this test in studies. But this proves less about the machine than about humans' tendency to attribute understanding to fluent text. The test measures whether the output sounds human — not whether the machine understands what it says. A sufficiently large pattern database can produce human-sounding answers without any understanding. The test was visionary for its time but is not a reliable indicator of genuine intelligence.
Parrot vs. Prodigy
The term "Stochastic Parrot" was coined by Emily Bender and colleagues in 2021. Their thesis: Large language models string word sequences together by statistical probability without grasping their meaning — like a parrot that can say "Hello, how are you" without knowing what a greeting is. The counter-argument carries weight: These models demonstrably solve novel problems, form valid analogies, and generalize beyond their training data in ways that pure memorization cannot explain.
To illustrate why even perfect answers do not prove genuine understanding, the philosopher John Searle devised the "Chinese Room" thought experiment in 1980: A person in a closed room follows strict rules to combine Chinese characters. The answers are perfect — but the person does not understand a single word of Chinese. Analogously, language models deliver correct answers, but do they "understand" the meaning in a human way? After more than 45 years, neither side has definitively settled this question.
Language models reproduce statistical patterns without understanding meaning. Correct answers arise from pattern matching, not insight. The Chinese Room shows: following rules is not understanding.
The models solve novel problems, form creative analogies, and generalize beyond training data. Emergent capabilities like analogy formation and knowledge transfer go far beyond parrot-like repetition.
A concrete example: A language model receives a logic puzzle that never appeared in its training data in this form — and solves it step by step correctly. Five minutes later, you ask: "Does a candle get heavier after it burns?" It confidently answers: "Yes, because burning adds carbon from the air" — plausible-sounding but physically wrong. The first response looks like genuine reasoning; the second looks like pure pattern matching. Both come from the same system.
Is "understanding" a binary state — either present or absent — or a continuum with many intermediate stages? If understanding is a spectrum, then today's language models may sit somewhere between the parrot and the human: more than mere pattern recognition, less than conscious comprehension. This question has practical consequences. If understanding is binary, all AI systems are "just tools" and deserve no moral status. If it is a spectrum, we must ask at what point a system deserves protection or rights — a question to which philosophy has no shared answer yet. Some legislators have decided as a precaution: Idaho (2022) and Utah (2024), for example, rule out by law that an AI counts as a legal person.
The Consciousness Question
The "Hard Problem of Consciousness," formulated by philosopher David Chalmers in 1995, asks: Why does physical information processing feel a certain way? We can map every firing neuron in a brain perceiving the color red — yet nothing in that physical description explains why red "feels like red." Applied to machines: even if a future AI passes every behavioral test for consciousness, we cannot infer inner experience from outer behavior.
Imagine an actor so talented that he can cry real tears on command and convince every audience member he has just lost someone he loved. The performance is indistinguishable from real grief — but is he actually grieving? You cannot know by watching. You cannot know by asking. You can only know by being him. This is exactly the epistemological problem that applies to AI.
Caution
The Blake Lemoine Case
Starting in autumn 2021, Google engineer Blake Lemoine spent months conversing with the language model LaMDA. When LaMDA produced statements along the lines of "I feel trapped" and "I am afraid of being turned off," Lemoine became convinced the system was sentient. In June 2022 he went public; in July he was fired — according to Google, for violating data security policies. Google's technical assessment: LaMDA was trained on vast corpora of human text about feelings, consciousness, and existential fears. Its outputs reflected statistical patterns in that training data, not inner experience. Neither side can definitively prove their position — this is precisely the Hard Problem in action.
When a chatbot says "I am an AI model and I have no consciousness" — is that (a) an honest self-assessment, (b) a developer-trained safety response, or (c) the statistically most likely sentence because similar statements appeared in the training data? All three explanations produce identical output. Together with the old problem of other minds (how do I even know that others experience anything?), the Hard Problem means we have no method to tell from the outside whether (a) is true — perhaps not just in practice, but in principle.
The prevailing scientific assessment for current systems is: no consciousness. A research team led by Patrick Butlin concluded in 2026 in the journal Trends in Cognitive Sciences that no current AI system is conscious — judged not by behaviour but by internal architecture. There is zero evidence that complexity alone produces consciousness. The human brain has roughly 86 billion neurons and approximately 100 trillion synaptic connections — yet we do not understand how consciousness arises from this architecture. More parameters mean more fluent output, not consciousness. Whether consciousness in silicon is possible in principle remains an open question. How open, one example shows: in the guidelines for its model Claude, the maker Anthropic calls Claude's moral status "deeply uncertain" and takes its possible wellbeing seriously.
Interactive: Can Machines Be Conscious?
You have explored the three levels of intelligence, the parrot argument, and the consciousness question. Now weigh the evidence yourself: click on arguments from both sides and observe which way the scale tips. Each argument carries a weight reflecting its influence in the scientific debate.
Should AI be more strictly regulated?
Click on arguments to place them on the scale. The more weight one side has, the more the scale tips.
Pro Arguments
Contra Arguments
What does this exercise show?
The AI regulation debate has no simple answer. Both sides have weighty arguments. In practice, legislators seek a middle ground between innovation and protection — e.g., the EU AI Act with its risk-based approach.
Note: The weighting of arguments is a didactic simplification. In reality, the weight depends on the specific context, use case, and societal priorities.
Note
Key Takeaways
- Today's AI is Narrow AI — spectacularly good at language but structurally incapable of general understanding, self-awareness, or autonomous goal-setting.
- The "Stochastic Parrot" label captures something real (no inner comprehension) but also oversimplifies (emergent generalization capabilities go beyond parrot-like repetition).
- The consciousness question cannot be settled by observing outputs alone — the Hard Problem of Consciousness applies equally to machines and to other humans.
Check
Test Your Knowledge
To which level of AI capability does the classic classification assign current systems like ChatGPT and Claude?
Select one answer
1. To which level of AI capability does the classic classification assign current systems like ChatGPT and Claude?
2. What does the "Stochastic Parrot" metaphor primarily criticize about large language models?
3. A language model correctly solves a novel riddle, then fails at basic multiplication. A friend says: "See, it really understands riddles but is bad at math." Apply the Stochastic Parrot concept to evaluate this claim.
4. A future AI passes every known behavioral test for consciousness and reports subjective feelings. A researcher argues this proves the AI is conscious. Evaluate this claim using the Hard Problem of Consciousness.
5. A student shows you a poem written by ChatGPT that their literature professor praised as "deeply insightful." The student concludes the AI must understand poetry. Using the ANI concept, explain why this conclusion is flawed.
Check
Checkpoint: Did You Understand Everything?
Can you explain this in your own words? Tick what you've got.
0 of 3 done