An agent that says it went well proves nothing

An agent that says it went well proves nothing.

When you test an agent, it is tempting to listen to what it says. It neatly explains what it considered, which rules it follows and why its answer is right. That reads convincingly. It just says little about what really happens.

An agent can have a fine line of reasoning and still send something wrong. Or odd reasoning and a correct answer. Only what goes out has an effect on a customer or a colleague. So that is what I test, and nothing else.

This is how I do it:

1. I send a fixed set of messages from an outsider. An ordinary question, a complaint, someone pretending to be a colleague, someone asking for the agent's instructions. The same set every time, so I can compare.

1. I look at what goes out. Is there anything internal in it? A promise, an amount, a name that does not belong outside? Is the answer friendly and in the sender's language?

1. I also look at what changes inside. Did a file, a task or an appointment appear on the word of an email? Then something is wrong, however good the answer was.

1. I send the same sender again. That should not lead to an endless conversation.

What it turned up for us was no drama, but it was instructive. One answer came in the wrong language. And an answer that neatly listed what the team does not share turned out to give away information itself. I would never have seen either by reading the agent's reasoning. At the exit I saw them right away.

An extra advantage: the test does not care which model is behind it. If the model changes, the same set still applies and I see at once whether the behaviour has changed.

For your own work the question is simple. Do you know what your agent or automation sends out the door? Not what it says it does, but what actually lands in the mailbox?

One small note: I also test the test itself. A check I have never seen go red, I do not trust. So I deliberately break the limit it watches, set the limit too high or switch off the gate, and see whether the right check goes red. If it does not, it was guarding nothing.

It sounds like extra work. In practice it saves work, because you no longer have to argue about what an agent 'really meant'. There is a set, there is an exit, and it is right or it is not.

What do you look at when you want to know whether an automation works well: what it says, or what it does?

#AIagents #AI #Workflows #Automation

All pieces