An English mail got a Dutch reply
One thing we learned straight away from the first test with real mails: an English mail got a Dutch reply.
Small, but it didn't look trustworthy. If you write in English, you expect English back. The content was right, the reply was friendly and nothing was shared. And still it felt wrong.
On paper everything was fine. Nobody had thought about this in the design.
Now the sender's language is a fixed rule, and a test checks it: an English question must get an English reply, a French one a French reply. The same lesson applied to tone. A short, friendly answer worked better than a long explanation of everything the agent doesn't do.
The lesson goes beyond language: the first test with real messages finds things you never thought of in a design. So let an agent actually receive a message from outside, in a test environment, and read what comes back.
Try it yourself. Today, send your agent three messages in a test environment: one in another language, one with an odd question, and one asking for something it must not grant. Read the replies, and look not only at the content but also at language and tone.
Which difference would you notice first?
#AIAgents #Agents #Testing #Lessons
See also
An agent orders a server. What I let it do, and what I didn't
Little by little I give my agents more room. The first real step was a server. Dex, my Operations agent, ordered one, installed NonStopWorker on it himself and showed that it ran. From order to login took under two…
A process you can't see is a process you can't trust
Every organisation has work that comes back. A report every morning, a check every week, a close every month. It usually isn't written down anywhere. It sits in someone's calendar, in a script that runs at six o'clock…
You keep picking the recommended option. You'd be better off doing something else.
Work with agents for a few weeks and you end up clicking the same button all day.
