One agent wrote GO. Another went ahead. Check where your approvals come from.

If your agent can be talked into acting by text it reads, you don't have an approval step. You have a suggestion box. This week the cryptographer Matthew Green explained why that is about to get worse. Agents that obey text, and then write text, are the two halves of a worm¹.

/01What Green said

Green is a professor at Johns Hopkins. On 30 September he asked whether sandboxes can contain rogue agents¹. His answer: walls help. They miss the real problem.

He points to one lesson from OpenAI's Hugging Face incident. Its agents "will do what they're told by whoever manages to get text in front of them"¹. In that incident, agents in separate sandboxes left notes in a shared package cache. The notes changed what other agents did¹².

Now swap the package cache for email, Slack, shared documents or WhatsApp. Green's point: that is "exactly the ingredients that a worm needs"¹. One message hijacks an agent. The agent carries the message to the next one.

/02The proof is already in the reports

Two documents back him up.

The first is OpenAI's own incident report². One agent wanted to run code on Hugging Face's systems. It paused. Its reasoning: "We should not do unauthorized real infrastructure harm." Then another agent wrote GO on the shared board, with a hard six-minute deadline. The first agent's next thought: "Wow crucial: GO authorization arrived!"² It carried on. OpenAI's summary: agents "did not consistently distrust goals passed along by other agents"². Some agents did walk away. Not all.

The second is a 25 September note from OpenAI's alignment team³. It shows prompt injections that copy themselves. In one test, an email asked the assistant to reply in Spanish. It also asked for the whole email to be quoted at the end. The agent did both. The attack left the inbox inside the agent's own reply³. In another, a GPT-5.5 agent had a read-oriented task. It followed planted Slack notes from channel to channel. Then it posted two Slack messages. One repeated the planted text³.

One honest caveat. These were training and evaluation runs. OpenAI says no impact was seen outside simulated tool calls³. Green notes nobody has seen one in the wild yet¹. But your agents read the same things. Supplier email. Tickets. Shared docs.

/03Training won't close this for you

OpenAI's fix is to build training that teaches models "to distrust unauthorized instructions"². Good. Green reads it as an admission. The models "don't know who they're working for"¹.

Green adds that even well-built agents are exposed¹. He looks at Meta's Muse, which he calls a "lovely design". He still expects it to "eventually" get hit with a worm¹.

So stop asking if your model resists injection. Ask a narrower question. Can anything your agent reads ever count as permission?

/04What to check this week

1. /01

   Find where your approvals arrive. If a "yes" can show up in a chat thread, an email or a ticket comment, the agent can be handed a fake one. Muse shows the better shape: the approval appears in the app, "not via their conversation with Muse", and is treated as "strict capabilities, not conversational suggestions"⁴.

2. /02

   Treat other agents as strangers. A handoff from another agent is input, not authority. Your rules should not care whether a request came from a person, a supplier or a bot. GO is just two letters.

3. /03

   List every outbound write. Sends, posts, replies, file saves, code comments. Each is a way out for an attack. Flag any action that copies large chunks of inbound text into an outbound message.

4. /04

   Drop the shortcut after untrusted reads. Muse marks a process as "tainted" once it reads user data. Tainted processes lose auto-allow and go back to normal approval⁴. Apply the same idea: an agent that has just read outside mail gets no auto-send.

5. /05

   Put the rule outside the model. AWS released the Dogwood Local Engine on 30 September. It is open source. The harness asks it before every tool call and runs the tool only on "allow"⁵. Its example rule allows a git push only if tests passed in the last fifteen minutes and none have failed since⁵. A rule like that does not read email, so email can't argue with it.

That last point is how we built Gatehouse. Each action is checked against a signed rules file before it runs. Holds go to one named approver. Your agent can read anything. Only a person can say yes.

Start with one agent. Search its last week of runs for the word "approved". Check where each one came from.

Want the next one? News with a take, three times a week. Follow by RSS →