(Image: FlyD for Unsplash) Every now and again, a piece of tech news appears that affords a glimpse of a future which no one was expecting. Such a moment occurred last Tuesday, concerning AI and cybersecurity and the future of just about everything we rely on. It has set off an urgent new race between good guys and bad guys (as though we don’t have enough of those). If the bad guys win, then there is going to be chaos at a scale that is hard to imagine.
If you think this is hyperbolic, I am going to try to convince you that it is not.
In 2003, a programmer somewhere added a few lines of code to a piece of software that now helps computers process video and is used billions of times a day — if you have watched a video on the internet, streamed anything, used a media player, or edited a video in the last 20 years, there is a very good chance you have used this software.
The code added by the programmer was routine work, unremarkable, quickly forgotten. Over the next two decades, that code was examined by security experts, tested thousands of times by automated security tools, and scrutinised by millions of eyes. Nobody found anything wrong with it. Then, last month, an artificial intelligence model read the same code and immediately spotted a flaw, a vulnerability waiting to be exploited.
It gets worse — the AI that found it was not specifically looking for a flaw in this particular video module. It had been let loose in the wild by Anthropic as a testing exercise for its new model, called Mythos Preview, which had not yet been released (and still hasn’t, as we will see). The AI simply crawled all over the place — old machines, new machines, little machines, big machines, many operating systems, firewalls, browsers and the like. Some of these systems are nearly 30 years old.
The AI went looking for trouble, and it found it.
How much trouble did it find? More than 10,000 bugs over a diverse sample of computer systems. And that was just in a few weeks, in just a subset of the world’s computer and other software-dependent systems. And not just trivial bugs. For instance, this included a 27-year-old vulnerability in OpenBSD — widely considered one of the most security-hardened security layers in existence, which sits inside hundreds of thousands of firewalls and other security kits.
Anthropic is not releasing the model to the public, at least not yet. It is simply too dangerous. That decision tells you everything you need to know about how seriously it is taking what it has built.
Saanya Ohja, who writes a tech-oriented Substack page called The Change Constant, recently pinned down the importance of what happened:
“What’s perhaps most striking is that Claude Mythos wasn’t trained for cybersecurity. The capability seems to come along for the ride: if you can understand a large, messy codebase well enough to improve it, you can also understand it well enough to break it. The same skill set just points in a different direction. ‘Coding model’ is synonymous with ‘offensive capability’, whether anyone intends it or not.”
There lies the rub. Mythos Preview is the latest of the frontier AI models from Anthropic. Code-breaking aside, it is apparently a competition slayer on many of the other general benchmarks that have now been developed to measure AI capability. But other companies like OpenAI, Alphabet and a slew of others from the US, Europe and (especially) China are consistently releasing new models; there is one almost every week from somebody— smarter, deeper, broader and better than the last one.
While it is indeed true that Anthropic excels at coding and computer science-related AI, none of its leads is intrinsic. Other AI labs are on its tail, catching up, overtaking, falling a bit behind, catching up again. That is the nature of innovation at the edge of the industry. The fact that Anthropic’s latest version of Claude has found more than 10,000 bugs in global computation is serendipity — if not Anthropic, it soon would have been one of the others.
What happens now? If AI can break into millions of computers, then we have a situation akin to banks handing over the keys to their vaults to anyone who knows how to insert them into a lock.
Project Glasswing
The first thing that has happened is that Anthropic quickly set up Project Glasswing, an initiative into which they invited some of the top tech companies in the world (Glasswing refers to a butterfly with transparent wings). The idea is to have the companies share in the security findings of Mythos and to have these problems patched, either through their own efforts or by Mythos itself. Presumably, Anthropic will be pitching itself to these companies as its preferred bug detective forevermore.
This is where things get gnarly. There are numerous cybersecurity companies in the world, many of which are large publicly traded entities that have long-term engagements with major tech companies (including having their software running their customer’s systems on an ongoing basis).
These companies should be in a state of terror right now — their entire raison d’être has been shattered. A single AI from a single company can clearly do a better job than they can. It can do it orders-of-magnitude faster, cheaper and fix the problems in near-realtime. Last week’s crash in cybersecurity stock prices is a good indicator of their future.
Then there is this — while Anthropic is presumably making a good-faith effort to get ahead of the bad guys by not releasing its software and bringing selected partners into the inner circle to patch up its system urgently, I suggest that there is zero chance that this is going to prevent a hacking apocalypse — the exploit bucket will soon have far too many holes.
There are somewhere between 10 and 20 top AI labs, all of which are within striking range of this capability, which is now far beyond what any group of top cybersecurity software engineers can do. There are also open-source versions of some of these LLMs, already out there in the public domain, and whose future versions will certainly be able to hack at this level of aggression. Not all of them will choose to withhold release.
And, of course, a near-uncountable number of different pieces of hardware and machinery that would be threat targets — like cars, elevators, Scada systems, smart TVs and industrial control systems. Anything with a chip and memory and a management system with which to access its innards.
Which, these days is, well, just about everything with a power supply.
As this news starts to get digested, rumours of emergency situation rooms and fraught late-night meetings are rife. For instance, US Treasury Secretary Scott Bessent and Fed Chair Jerome Powell called an urgent meeting of Wall Street CEOs within 24 hours of the news to discuss it behind closed doors. I expect that the tenor of the conversation would have bordered on panic. And, presumably, these meetings are not just being held in the US.
Consider this — if the notorious Lazarus group in North Korea gets hold of this code (or any one of the other AI systems approaching this level of capability), then there is no bank that is safe. If a US enemy gets it, then no dam, no utility and no transport system is safe.
They will be hacked in weeks, if not days and perhaps even hours.
But there are, unsurprisingly, those who are sceptical, categorising this as just another “cry wolf” story from cybersecurity alarmists. These include well-respected commentators like Tom’s Hardware, hacker George Hotz and others who were a bit tetchy, accusing Anthropic of alarmism and overstating the threat.
I believe they are wrong, for the simple reason that autonomous AI hacking is improving quickly, just like other AI capabilities.
They should also remember how the “cry wolf” story ends. DM
Steven Boykey Sidley is a professor of practice at JBS, University of Johannesburg, a partner at Bridge Capital and a columnist-at-large at Daily Maverick. His new book, It’s Mine: How the Crypto Industry is Redefining Ownership, is published by Maverick451 in South Africa and Legend Times Group in the UK/EU, available now.