Skip to content
florisknol.com
NLEN

A good model doesn't make an agent trustworthy

A friendly robot wearing reading glasses; beside him a large coral-coloured button under a glass dome and a pair of scissors.

Say you ask an agent to summarise a document. The model understands the task fine. But the tool it can use also gives it access to delete files or send messages.

Then the risk isn't only in what the model understands. It's also in the permissions of the tool and in the environment the agent works in.

So I look at the whole: the model, the instructions and boundaries, the tools and the environment. A mistake in one place can change the behaviour of the entire agent.

With my own agents you see this in small choices. An agent answering on WhatsApp has no tools during that turn. An agent reading mail from strangers may only read. An agent ordering a server does so within a ceiling and after a human says yes. The boundary sits in what the agent can do, not in what it promised.

You can walk through this today. For one agent, write down what it can technically touch: which files, which messages, which systems. Cross out what it doesn't need for its task. What remains is the real size of your risk.

Which part would you check first in an agent you build?

Source: Anthropic's research on trustworthy agents.

All pieces