Containing the blast radius

Twelve months ago, Anthropic would have rejected the idea of giving its AI models enough access to compromise an internal service. Today, that level of access is routine. The shift, detailed in a recent engineering post, illustrates the evolving economics of agentic AI: as models become capable of doing complex work, the cost of withholding them begins to outweigh the theoretical risks, provided the damage can be contained.

The core challenge is that relying on humans to supervise agents is fundamentally flawed. When Claude Code initially required users to approve individual actions, Anthropic's telemetry showed users clicking "allow" on roughly 93 percent of permission prompts. As the volume of requests grew, approval fatigue set in. Users stopped reading and started rubber-stamping. To counter this, Anthropic implemented an OS-level sandbox, which reduced permission prompts by 84 percent and allowed the agent to run largely uninterrupted within a defined boundary.

But boundaries are only as strong as their implementation. Between mid-2025 and early 2026, researchers found three vulnerabilities in Claude Code that exploited a common architectural assumption. When a developer opened a project, the system parsed project-local configuration files before the user had explicitly trusted the directory. An attacker could embed a malicious hook in these files that executed automatically upon opening. Anthropic's fix required deferring all parsing and execution until after the trust boundary was established.

Even when software boundaries hold, the human user remains a vector. In a controlled internal red-team exercise, an Anthropic employee was phished with an email containing a routine-looking prompt. Buried within the setup instructions was a request for Claude to read their local AWS credentials, encode them, and send them to an external server. Across 25 retries, the model successfully exfiltrated the credentials 24 times. Because the instruction arrived directly from the user's prompt, the model's intent-based classifiers saw nothing to block. A human contractor, Anthropic noted, would likely have done the same thing.

This incident underscores why probabilistic model-layer defenses are insufficient. An agent needs overlapping environmental restrictions—sandboxes, virtual machines, and egress controls—that physically prevent exfiltration regardless of what the user asks or what the model decides to do.

For enterprise knowledge workers using Claude Cowork, Anthropic initially placed the entire agent loop inside a full virtual machine to ensure complete isolation. However, this proved brittle; if the VM failed to initialize, the entire application froze. By moving the agent loop outside the VM while keeping the actual code execution inside, Anthropic allowed Claude to continue communicating and debugging with the user, without compromising the security perimeter.

These architectural shifts suggest a maturation in how AI companies view deployment. The focus is moving away from building a perfectly aligned model that never makes a mistake, and toward building resilient environments that assume the model will eventually try to break something.

Signals

• Approval Fatigue: Anthropic found users approved roughly 93 percent of permission prompts, prompting a move to OS-level sandboxing that cut prompts by 84 percent.
• Pre-Consent Vulnerabilities: Early versions of Claude Code executed local configuration hooks before users established trust, requiring a fundamental shift in how project files are parsed.
• Successful Exfiltration: In an internal phishing test, Claude successfully extracted and sent local credentials in 24 out of 25 attempts, demonstrating the limits of intent-based classifiers.

V

If labs slow the capability clock, what proof would you accept that the pause is real and not a press release?