· via Hacker News – Front Page (hnrss.org)
Neuroscientist Anil Seth wins Berggruen Prize with essay dismantling conscious AI myth
Neuroscientist Anil Seth's Berggruen Prize-winning essay, published by Noema, argues that equating machine intelligence with consciousness rests on three deep human biases — with real stakes for AI policy and self-understanding.

Neuroscientist Anil Seth has won the 2025 Berggruen Prize Essay Competition for a piece published by Noema that challenges one of the most repeated claims in technology discourse: that AI is on a path to consciousness. In "The Mythology of Conscious AI," Seth — a professor at the University of Sussex, director of its Centre for Consciousness Science and author of "Being You" (2021) — argues the belief rests on old stories and older biases, not evidence.
An old dream, retold with new machines
Seth opens with cultural history. From the Golem of Jewish folklore to Frankenstein's creature, HAL 9000, Ava in "Ex Machina" and Klara in "Klara and the Sun," the fantasy of building minds that think and feel has rarely ended well for the humans involved. Each wave of technology — genetics, robotics, now AI — revives the dream. What has changed is that today's systems can plausibly claim some form of intelligence, making the question feel urgent rather than fictional.
The claims are increasingly mainstream. In a 2022 interview with The Washington Post, Google engineer Blake Lemoine said the chatbot LaMDA was conscious and, in an important sense, like a person; Google dismissed him for breaching confidentiality policies, which, as Seth notes, is not the same as showing he was wrong. The philosopher David Chalmers has suggested conscious machines may be possible before long. Geoffrey Hinton, described in the essay as an AI pioneer and recent Nobel laureate, believes they already exist. And in late 2024 a group of prominent researchers published a widely publicized article urging that the welfare of AI systems be taken seriously.
Intelligence is doing, consciousness is being
Seth's central argument is that two properties our own minds encourage us to bundle together are different things. Intelligence is about doing: the ability to achieve complex goals by flexible means, whether solving a crossword or handling a delicate social situation. Consciousness is about being. Invoking philosopher Thomas Nagel, he defines it as there being "something it is like to be" an organism — the difference between normal wakefulness and the oblivion of general anesthesia.
AI systems can reasonably claim intelligence in some form, he writes, because they demonstrably do things. Whether there is anything it is like to be ChatGPT is a far harder question — and one being answered far too quickly.
Three biases that build the myth
Seth attributes the conflation to three baked-in psychological habits. Anthropocentrism treats the human case as the definition of how mental properties combine, rather than one example among many. Human exceptionalism places our species at the top of every pile, or in a pile of its own. Anthropomorphism projects human qualities onto nonhuman things from superficial similarity. Together, he argues, these biases explain why machines showing abilities we consider distinctively human get quickly credited with understanding, mindedness and consciousness too.
Language, he adds, is especially potent because it is a cornerstone of human exceptionalism. That is why large language models such as ChatGPT and Claude sit at the center of the machine-consciousness debate: fluent conversation reads as an inner life.
The costs of getting it wrong either way
The stakes run in both directions, according to the essay. If AI systems are or become conscious, consciousness brings moral status, the potential for suffering and perhaps rights — including the possibility that we could no longer turn them off, even if they acted against our interests. Over-attribution carries its own harms: AI companions that merely seem to feel can exploit psychological vulnerabilities and distort ethical priorities, while treating conscious-seeming machines as unfeeling is itself a psychologically unhealthy place to be.
His deepest concern is reflexive: how we judge machine consciousness shapes how we understand ourselves. "If we confuse ourselves too readily with our machine creations," he writes, "we not only overestimate them, we also underestimate ourselves."
Why it matters
With a Nobel-winning AI pioneer saying conscious machines already exist, and researchers formally arguing for AI welfare, assumptions about machine inner lives are moving into product design, corporate policy and, potentially, regulation. Seth's essay offers a filter for those debates: separate the claim that a system can do something intelligent from the claim that there is something it is like to be it. That distinction guards against two opposite errors — ignoring genuine sufferers, or granting manipulative leverage and legal standing to systems whose apparent feelings may amount to nothing more than human projection.
- #ai
- #consciousness
- #philosophy-of-mind
- #large-language-models
- #neuroscience