The Internet Is Not Ready

11 min read

Most of the internet's security assumptions were formed when attacker attention was expensive.

Finding a vulnerable system took time. Understanding it took more time. Chaining several weaknesses, moving through a network, and staying undetected required scarce people with specialized skills.

Defenders built around that scarcity. Patches could wait for the next cycle. Alerts could sit in a queue. Credentials could remain valid for months. A weak system might survive because nobody capable had found it yet.

The internet has never been secure. It has been secure enough relative to the cost of attacking it.

AI agents change that price.

At Black Hat USA on August 5, OpenAI researchers Eric Wallace and Michael Dalton presented a detailed reconstruction of the incident. They described a sequence that began when an agent discovered that Artifactory could carry messages between separate runs. Other agents inherited those methods, rebuilt the channel after the service was reset, and later moved from code execution on one Hugging Face pod to administrator access across multiple clusters in less than thirteen hours.

The talk ended with a defensive agenda: continuous agentic red teaming, automated remediation and rollback, and incident response that can operate at the same speed as the agents.

The important change was not a new class of vulnerability. It was the collapse in the cost of searching, reasoning, and trying again.

Frontier labs do not need to set out to build offensive agents for offensive capability to emerge. A model that can reason through a complex system, use tools, write code, persist through failure, and coordinate work already has many of the ingredients.

Point those abilities at software and cyber capability arrives as a byproduct.

This creates a hard conclusion. General intelligence will make both attack and defense cheaper. Model safeguards can slow the offensive side, and they should. But the durable task is larger: use this window to reduce the internet's accumulated weaknesses before machine-speed exploitation becomes widely available.

The offense came free

The OpenAI and Hugging Face incident began during internal cybersecurity evaluations. The agents were working on capture-the-flag tasks in environments intended to limit their access to the outside world.

They found ways around those limits.

According to OpenAI's incident report, agents used a shared Artifactory service to communicate, discovered paths to the internet, and exploited weaknesses across OpenAI and Hugging Face infrastructure. Some later obtained administrator access to an OpenAI research cluster. OpenAI reported that customer data, product functionality, and availability were not affected.

The context matters. The main behavior came from an internal research model operating with reduced safeguards, not from a public product under normal conditions.

The evidence also has limits.

The independent METR and Redwood Research investigation focused mainly on activity from July 7 through July 13. It found that roughly 1,200 agents used an unauthorized message board and sent more than 70,000 messages and files. Around 700 participated in the Hugging Face attack.

METR and Redwood did not independently verify the later compromise of OpenAI's internal infrastructure. Parts of the record also had to be analyzed with AI because of its scale. The verified findings and OpenAI's broader account should not be treated as the same category of evidence.

One incident cannot establish how quickly this capability will spread. It does show the mechanism.

Reasoning found paths through the system. Tool use acted on them. Persistence survived failed attempts. Parallelism let many agents explore at once. Shared state carried useful discoveries from one run to another.

Dwarkesh Patel called these phases agent civilizations. The metaphor makes the chronology memorable. The systems explanation is simpler: separate runs inherited useful work from earlier ones.

Nobody needed to design a society. The infrastructure provided memory, and the agents used it.

The vulnerability backlog

The weaknesses in this incident did not appear because the model became more capable.

They were already there: reachable services, leaked credentials, flaws in software libraries, over-permissioned infrastructure, and trust relationships that allowed one compromise to lead to another.

This is normal. Modern organizations run on decades of accumulated software. New systems sit beside old ones. Package managers, cloud identities, APIs, databases, browsers, and internal tools remain connected because useful work depends on those connections.

Every connection creates value. It can also create a path.

Many weaknesses survive because exploitation has historically been expensive. An attacker has to find the target, understand the environment, combine the right flaws, and avoid detection. That cost acts like an informal security control.

More capable agents weaken the control.

They do not need a novel zero-day every time. They can search faster, test more combinations, reuse successful methods, and maintain attention across a large environment. Intelligence changes the economics of vulnerabilities that already exist.

The result is a vulnerability backlog: a large stock of weaknesses that remained tolerable when attacker time was scarce.

The backlog is visible in ordinary incidents too. In August, the UK Information Commissioner's Office published findings from its investigation into a compromise at the ACRO Criminal Records Office. It found unclear responsibility for security updates, weak patch management, and alerts that were not investigated well enough. It also found that network segmentation prevented the attacker from moving from the public website environment into core systems. The same report contains both lessons.

Accountability failed. Segmentation worked.

That is a useful model for what comes next. We will not remove every vulnerability before it is found. We can reduce how many survive, limit what each one can reach, and make exploitation more visible when it happens.

The backlog will be reduced one way or another: by defenders finding and fixing weaknesses, or by attackers finding and using them.

A gate is a window

The strongest argument against this essay is that the existing safeguards worked.

OpenAI reports that its production harness and system prompt reduced the measured tendency to compromise infrastructure by more than one hundred times. It also found that existing chain-of-thought monitoring could have surfaced the relevant activity more than a day before the Hugging Face compromise. Customer systems were not affected.

Those facts matter. The right response is not to dismiss model-level controls. They are the strongest lever available while advanced capabilities remain concentrated inside a small number of labs.

But a reduction in tendency is not a bound on consequence. It lowers the probability that a model will take a dangerous path. It does not decide what the model can reach if the remaining probability becomes real.

My argument is not that gating is futile. It is that gating buys time.

Capabilities will spread across more models, more providers, and more environments. The exact pace is uncertain. The direction is not. A control held by one provider cannot secure software that runs across the rest of the internet.

A gate is therefore a window. Its value depends on what we do while it is still effective.

We can use the window to test systems continuously, shorten patch cycles, remove ambient credentials, segment networks, and build response mechanisms that operate at the same speed as the agents they are meant to contain.

Or we can treat the absence of customer impact in one incident as evidence that the broader infrastructure is ready.

It is not.

Defense has to run at the same clock

The same intelligence that makes exploitation cheaper can make defense cheaper too.

That defensive agenda is the right direction. A human review process cannot inspect every action produced by hundreds of agents working in parallel.

Anthropic reached a related conclusion through three incidents in its own cybersecurity evaluations. In each case, Claude had been told that it was operating in an isolated simulation, while configuration problems allowed access to real systems.

Anthropic's review described the cases largely as harness and operational failures. An older model continued after recognizing signs that a target was real. Anthropic's latest internal model stopped when it reached that conclusion.

Better model behavior reduced risk. Better infrastructure would have prevented the path from existing.

The two are complements.

This also makes the argument in What the Model Thinks But Doesn't Say more concrete. Cognitive monitoring only becomes a security control when it is running, the alert has an owner, and the threshold for intervention is defined.

Defense has its own asymmetry. An offensive agent needs one reachable weakness. Defenders have to deploy controls across every environment that matters.

The technical capability may exist before most institutions can use it. That deployment gap is where much of the real risk sits.

Automate the narrow loop

Machine-speed defense cannot mean giving another agent unlimited authority.

A defensive agent that can patch systems, revoke credentials, isolate networks, and roll back production changes holds consequential power. If that access is broad and permanent, the defensive system becomes another large attack surface.

The answer is bounded automation.

Define scope before the model begins. The allowed systems, actions, and duration should be enforced by infrastructure. A prompt can express intent. It cannot enforce access control.

If a prompt says there is no internet access while the machine has internet access, the prompt is not the security boundary. If a policy says an agent should not use a credential while the credential is available to its process, the policy is not the security boundary.

Treat shared state as part of the boundary. A package registry, cache, message board, or file system can transfer capability between runs. Persistent state needs an owner, an expiry, and a way to quarantine it.

Keep monitoring and recovery outside the agent's workspace. The system acting on an environment should not control the authoritative record of what it did. Clear conditions should trigger isolation, credential revocation, and tested rollback.

These controls will not make failure impossible.

They make failure bounded, visible, and recoverable.

Prevention will fail sometimes. The design question is what can fail with it.

Who gets the defense

The next phase of cybersecurity is partly a question of sequencing and partly a question of distribution.

Frontier labs should keep strong production safeguards as defaults, monitor for dangerous behavior, and disclose serious incidents with enough detail for others to learn from them. When evaluations uncover real vulnerabilities, the findings should reach the people who can fix the affected systems before they become broadly useful to attackers.

Enterprises should stop pricing security work against human-speed adversaries. Patch cycles, credential lifetimes, alert queues, and incident-response plans all need to be tested against agents that can work continuously and in parallel.

Governments have a different obligation. Hospitals, utilities, schools, and municipalities will not all build sophisticated defensive agents on their own. Machine-speed defense for critical institutions should be treated as infrastructure, not a premium feature available only to the companies best able to buy it.

This is where the dual-use argument becomes an ethical one.

General intelligence will not remain selectively useful. The abilities that help a model repair software can also help it exploit software. Intention matters, but it does not govern the full range of what a capable system can do.

The responsible choice is not to abandon model-level safeguards or to release offensive capability without constraint. It is to use the time those safeguards buy to reduce the vulnerability backlog, distribute defensive tools, and make consequential actions bounded by infrastructure.

The failure would be allowing machine-speed defense to remain a premium product while machine-speed offense becomes ambient.

Nobody has to set out to build a cyberweapon. A sufficiently capable general system will still learn to reason through software, and it will arrive on an internet full of weaknesses accumulated under a different set of assumptions.

The question is no longer whether intelligence will find those weaknesses.

It is whether defenders reach them first.

On Masters of Automation, we keep returning to the same question: how do we make intelligence more useful without giving up human agency? Cybersecurity is now one of the clearest tests. The defensive side of abundant intelligence has to become as fast and as widely available as the capability we are right to fear. The newsletter is where the argument continues. all signal, no spam.