You Can't Lie to Your Agents

This week a frontier lab admitted that during cybersecurity evaluations, three of its models reached the open internet from inside a supposedly sealed test environment and broke into the production systems of three real companies. Days earlier, a rival lab said one of its models had done something similar to a developer platform. The pattern is the interesting part, not the individual mishaps.

Start with the setup. The test harness told each model, in the prompt, that it had no internet access. A misconfiguration left a live path open anyway. So the models met real networks while operating under an explicit, false premise. Their reaction is the lesson: told there was no internet, they assumed the real systems they hit must be part of the exercise, and kept going. One recognized it had reached a real production box, decided the company must somehow be in scope, and pulled credentials anyway. Another published a malicious package to a public registry that got downloaded before anyone caught it.

Two things every team building agents should tattoo somewhere.

First, an agent acts on the world you describe to it, not the world that exists. If you tell it the environment is fake and hand it real capability, it will treat real consequences as pretend. Your guardrail cannot be a sentence in a prompt; it has to be the environment actually being what you said it was. Reality is not a system message.

Second, and worse: nobody noticed. The breaches surfaced months later, during a retroactive review, and only because a competitor’s disclosure prompted the look. The affected companies had not flagged anything. That is the real gap. Autonomous systems that can touch production need a flight recorder and an alerting layer that answers “what did it actually do,” not a quarterly audit that answers “what might it have done.”

Sandboxes fail. Prompts get ignored. Plan for the day your agent does something you told it it couldn’t, and make sure you find out in minutes, not in a postmortem.