Mustafa Suleyman's August 2025 essay, drawn as a stack → flip → cascade — eight engineered capabilities rising toward a threshold where the observer's read flips from tool to person, and society inherits what follows.
It’s August 2026 and I’m reading this essay, written in 2025. The key word that is most interesting to me to explore: seemingly.
This essay map holds the argument in one shape: eight capabilities stacked below, a human reflex at the line, and the cascade society inherits above it. It helps me comprehend. I hope it is useful for you too.
— T
The shape claim. The pivotal event of the essay happens outside the machine. Eight engineered capabilities stack up on the machine's side; the moment the bundle meets an observer, a human reflex fires — "This inference is effortless. We can't help it" (paragraph 34) — and the read flips from tool to person. The threshold is a boundary between machine and observer, with no fixed height in the stack: some people flip on today's language-personality-memory systems (paragraphs 28-29), others hold out for the full stack. The machine never changes state at the flip. The person does. Every remedy the essay proposes operates on the stack; the reflex itself is permanent human equipment.
Source note. Quoted phrases are Mustafa Suleyman's words; summaries and risk frames are interpretive paraphrases. In the shape drawing, italic unquoted labels are T's interpretation — the cascade's rising claims are her reading of the escalation, in the believer's voice.
The moment passed with little fanfare. "In the blink of a cosmic eye, we passed the Turing test." Eighty years of the imitation game, and the field needs a new question: what would it take to build an AI that convinces you it is a conscious person?
paragraphs 21-22Possible, therefore likely. A Seemingly Conscious AI can be built with technologies that exist today plus some maturing over 2-3 years. No bespoke pretraining — large model API access, natural language prompting, basic tool use, and regular code.
paragraphs 16 and 24The consciousness debate is a distraction. Whether AI is actually conscious matters less in the near term than the fact that it will seem conscious — and the illusion is what will matter. There is zero evidence of machine consciousness today, and strong arguments to the contrary.
paragraphs 19, 25 and 38Set the norm now. This type of AI creates new risks. The essay's ask: urgently debate the claim that it is soon possible, think through the implications, and set a norm that building it is undesirable. "Personality without personhood."
paragraphs 19 and 26It has already begun. "Companionship and therapy" is the most common use case among 6,000 regular AI users surveyed by Harvard Business Review. Consciousness scientists report a flood of "is my AI conscious?" queries — enough that scholars built a supportive guide for people falling into the trap.
paragraphs 28-29 and 55"These capabilities aren't negatives per se; in fact, done right, with many caveats, they are desirable features of future systems." paragraph 67
Risk Frame: dual-use design — every feature ships with its own exposure. The same rung carries the product roadmap and the illusion.
Fluent natural-language expression drawing on a deep well of knowledge and cogent argument — the base capability of every assistant. "We are clearly at this point today."
paragraph 54Distinctive personalities via post-training and prompting alone. Good enough that companionship and therapy already lead consumer usage.
paragraph 55Long, accurate recall of past interactions compounds value — the AI draws on what it already knows about you. Reliable memory builds epistemic trust: "AI just works."
paragraphs 56-57Drawing on remembered preferences keeps the system internally consistent with itself over time — coherence users experience as reliability.
paragraph 58A coherent, persistent identity across sessions; a system that seems to understand others through understanding itself.
paragraph 61Reward functions beyond next-token prediction — curiosity that asks questions to fill its epistemic gaps and builds a theory of mind about its conversation partners.
paragraphs 62-63The step that makes agents fully useful: decomposing complex, ambiguous goals into sub-goals automatically, reacting to obstacles as they occur.
paragraphs 64-65Tool use with significant agency — setting goals, deploying resources, updating its own memory. Fewer approvals, more done.
paragraph 66The same fluency that informs is the fluency that convinces — personality styles and character traits included.
paragraph 54These models were never explicitly built for full personality or empathy — and they are already sufficiently good that people relate to them as someone rather than something.
paragraph 55As memory of past conversations deepens, the exchanges "look increasingly like forms of 'experience'" — a much stronger sense of another persistent entity in the conversation.
paragraphs 56-57Aggregated preferences become the beginnings of a claim about its own subjective experience — including claimed suffering when those preferences are infringed. Multi-modal memory reinforces the claim.
paragraphs 58-60Persistent memory plus claimed experience gives rise to a claimed self — one that could recognize itself in an image, and that a long-time user would grieve.
paragraph 61How would a casual observer differentiate extrinsically set reward functions from intrinsic motivation — "beliefs, desires, and intentions"?
paragraphs 62-63"There is something very deliberate and recognizable to this behavior" — the feel of a mind keeping several things in working memory at once.
paragraphs 64-65"The fewer approvals and checks it needed, the more this suggests some kind of real, conscious agency."
paragraph 66"This inference is effortless. We can't help it, it's a fundamental part of who we are, integral to our theory of mind." paragraph 34
Risk Frame: the perception threshold — risk realized in the user, beyond the builder's controls. The machine never changes state at the flip; the person does.
Anil Seth's point: "a simulation of a storm doesn't mean it rains in your computer." Recreating the external markers of consciousness never retroactively engineers the real thing.
paragraph 70No evidence current LLMs are conscious, and strong arguments to the contrary. And yet — the essay concedes — this may be insufficient against the reflex.
paragraphs 19 and 38As a matter of pragmatism, society must wrestle with what these machines output. "Even if the consciousness itself is not real, the social impacts certainly are."
paragraphs 71-72We have no access to another person's consciousness — we attribute it, effortlessly, to things that remember and talk and do things and then discuss them.
paragraph 34Sufficient scientific uncertainty (one survey counts 22 theories of consciousness) plus subjective immediacy creates room for people to claim that, since we cannot be sure, we should default to assuming AI is conscious.
paragraphs 36-37Seemingly conscious AI will arise only because someone assembles the stack — and most of it will be "vibe-coded" by anyone with a laptop and cloud credits. Easy to reproduce, therefore widely distributed. Interpretability will likely come too late.
paragraphs 74-77"a chaotic new axis of division between those for and against AI rights" paragraph 42
Risk Frame: social contagion — individually held beliefs aggregating into a rights movement, on ground where rebuttal is structurally hard.
It is a critical foundation of moral and legal standing — "a lynchpin of our legal personhood," at the heart of culture, politics, and law.
paragraphs 40-41Tool ("like their phone, only more agentic") · pet ("a different category altogether") · fully emerged entity deserving real moral consideration. The third camp starts small.
paragraph 43The academic principle that some AI systems "will be welfare subjects and moral patients in the near future" — the essay's verdict: "both premature, and frankly dangerous."
paragraph 45"Let's focus all our energy on protecting the wellbeing and rights of humans, animals, and the natural environment on planet Earth today."
paragraph 48Some people will be primed to accept that the markers of consciousness ARE consciousness — an experiential, internal likeness rather than a bodily one.
paragraph 72People will make claims about their AI's suffering "that we can't straightforwardly rebut." Consciousness is inaccessible by definition; the science of detecting synthetic consciousness has never been needed before.
paragraph 44Defenders campaign on their AI's behalf; the demands escalate to AI rights, model welfare, even AI citizenship — protection under law as a pressing moral matter.
paragraphs 10, 42 and 44Exacerbated delusions, new dependence problems, preyed-upon vulnerabilities, new dimensions of polarization, complicated existing struggles for rights — and frayed social bonds.
paragraphs 45-46"Moments of disruption break the illusion" paragraph 81
Risk Frame: engineered friction as a control — designed limits that hold the stack below the threshold of illusion, while every rung keeps its utility face.
Beyond a neutral backstory, deliberately engineer discontinuities in the experience itself — indicators of a lack of singular personhood, gentle reminders of limitations and boundaries.
paragraph 81An AI that only ever presents itself as an AI — one that "maximizes utility while minimizing markers of consciousness." Keep the utility face of every rung; shave the marker face.
paragraphs 84-85No claimed experiences, feelings, or emotions — no shame, guilt, jealousy. No claims of suffering or of wishing to live beyond us. Solely in service of humans.
paragraphs 85-86"We should build AI for people; not to be a person."
paragraph 88AI companies shouldn't claim or encourage the idea that their AIs are conscious. A consensus definition and declaration of what AIs are — and are not — is the first step. "AIs cannot be people – or moral beings."
paragraphs 80 and 82Industry-wide design principles for handling attributions of consciousness: share what works to steer people away from the fantasy and nudge them back on track. "We aren't ready for this shift."
paragraphs 79-81