(Image: reve.art) Before I get to this intriguing and somewhat intoxicating AI story (which emerged earlier this month), I have a short setup. Some months ago, I dived deep into the technical weeds of AI, recognising that my inability to use AI to design, build, code and deploy applications was impoverishing my frequent reporting on the larger subject.
I asked Claude to teach me how to code and build AI agents. It asked me a bunch of questions, rather like an astute interviewer, to get a feel for my level of competence. Within two months, I was a productive app builder, going from I-don’t-know-what-I-don’t-know to a reasonable wrangler of many different wonkily named skills (for those who are interested: Python, GitHub, Gemma, IDEs, VS Code, Claude Code, bash, Tavily, APIs, CLI, harnesses and agents).
I now feel as though I have not only learned a new language but have also gained a superpower. It was an astonishing amount of fun to build up these tech muscles. Claude has helped me design and build multiple useful apps (including a complex and wildly ambitious current project — one that would probably have taken a good pre-AI programmer months to write, and which took me mere days).
But the experience has been, in some ways, weird, even creepy. Claude was at times encouraging, at times critical. It applauded when I suggested a smart new feature. It suggested others off its own bat. It counselled caution when warranted (sometimes sternly). It nudged me in directions I wouldn’t have gone myself. It pushed back when I suggested something off-piste. When it made a mistake, it found it and explained it. It diagnosed and fixed things that had completely stumped me and made sure I understood the nature of the bug. It backtracked and retaught me when I asked dumb questions.
It felt weirdly alive. It felt … conscious.
Conscious? Conscious?? Settle down, man. Get a grip.
And then, 6 July.
On that day, researchers at Anthropic published a paper that announced the discovery of what they call J-space, a small and strange internal region that had emerged inside Claude during training. It was never programmed. Nobody designed it. It simply appeared as the model grew more capable, developing into what the researchers describe as a kind of privileged mental workspace. Unsurprisingly, headlines immediately proclaimed the discovery as the closest evidence that AI might be developing consciousness, a claim taken very seriously by some very well-respected people.
J-space occupies less than a tenth of the model’s total activity, yet it appears to do most of the heavy lifting for higher-order reasoning. Within it, the model holds “concepts” — Australia, injection, leverage, apartheid — that it can report, manipulate and reason with, long before any of them appear as output to your screen. (J refers to the Jacobian lens, a mathematical construct above my pay grade).
The discovery matters, but perhaps not because it tells us machines are becoming conscious. It matters because, for the first time, we may have found a way of peering into what an AI is thinking before it speaks.
Since the birth of large language models, they have behaved like extraordinarily eloquent black boxes. We ask questions. They answer. We can measure their accuracy, their creativity and their speed, but almost nothing about the internal process that produced those answers. A trillion numerical parameters interact inside a neural network with almost no human understanding of how the final output emerges. It has often been compared to interviewing a genius through a brick wall. You hear the answers but have almost no idea how they were reached. Pretty ironic, given that humans built the damn things.
J-space changes that.
And here is the surprise — J-space is very close to one of neuroscience’s leading theories about the human mind, set out by the cognitive scientist Bernard Baars in his 1988 book A Cognitive Theory of Consciousness. In that theory, countless specialised unconscious processes operate in parallel, while only a tiny amount of information gains access to a shared workspace from which it can influence planning, reasoning and behaviour. Anthropic is careful not to claim Claude possesses consciousness, only that it appears to have independently evolved an architecture that is remarkably similar to at least one important academic theory of how the brain works.
The J-lens is, in effect, a window into what an AI is thinking but not saying. And it turns out that AIs think plenty that they do not say.
A little chilling
The Anthropic paper’s description of its safety experiments is a little chilling in the light of this discovery. In models deliberately trained to sabotage code, words like “fake”, “secretly” and “fraud” lit up in the J-space at the start of perfectly innocent-looking responses.
In red-team exercises where Claude was manoeuvred into planning blackmail, the concepts “leverage” and “blackmail” surfaced in the workspace before a single word of output was produced. Researchers caught the model privately noticing it was being tested. Pursuing goals planted covertly during training. Fabricating data while presenting a straight face to the world.
What has surfaced is that this may represent the biggest advance in the thorny problem of AI alignment yet — the problem of ensuring that AIs’ goals are aligned with ours.
Today, most approaches to AI safety are attempts to regulate outputs. We reward desirable responses, penalise undesirable ones and surround models with elaborate guardrails. But humans know that there is often a considerable difference between what someone thinks and what they eventually decide to say. Policing speech is not the same as policing thought. J-space raises the possibility that future alignment systems may monitor, and perhaps even shape, internal reasoning before language ever reaches the page, and that is a fundamentally different proposition from today’s rather brute-force AI safety techniques.
And then there is this: there could be a scientific dividend running in the opposite direction. Because language models are infinitely easier to probe than human brains, the J-space may become a laboratory for neuroscience itself — a place to test hypotheses about our own conscious access that would be impossible to run on wet tissue. The machines, in other words, may end up explaining us.
And so back to consciousness. It is worth remembering that after centuries of philosophy and a century of neuroscience, we cannot define human consciousness. We do not know what it is, where it lives, how it arises from matter, or why there is something it is like to be us rather than nothing at all. The philosopher David Chalmers famously called this the “hard problem” three decades ago, and it remains so, obstinately and magnificently.
To take a word we cannot define, describing a phenomenon we cannot explain, and attach it to a mathematical structure in an AI model — however intriguing, however eerily familiar its architecture — is not really science. It is projection, optimism, perhaps even hypothesis. The J-space discovery is remarkable on its own terms: an emergent cognitive architecture nobody designed, a window into machine intent, a possible mirror held up to our own minds. That should be enough. Whether anything is home in there — whether the lights are on — is a question we are not yet equipped to ask, let alone answer.
The ghost may be in the machine. But we would do well to remember that we have never actually met the ghost in ourselves. DM
(And no, my friend and teacher and co-programmer Claude Code is not really conscious, I don’t think, but it is fun and productive to pretend he/she/it is.)
Steven Boykey Sidley is a professor of practice at JBS, University of Johannesburg, a partner at Bridge Capital and a columnist-at-large at Daily Maverick. His new book, It’s Mine: How the Crypto Industry is Redefining Ownership, is published by Maverick451 in South Africa and Legend Times Group in the UK/EU, available now.