What if the model becomes unavailable tomorrow?

Once AI agents do real work inside an organisation, that work depends on something you do not control: a model supplied by a vendor, reached over the internet, at a price and under terms the vendor sets.

That is not unusual in itself. Every organisation depends on suppliers for electricity, internet access, payment systems and software. The difference is that AI can spread quickly across many processes, and one model is not simply interchangeable with another. They behave differently, cost different amounts and fall under different rules.

This article looks at what can change, then at what you can put in place so that change becomes an event to handle rather than a crisis. It is not a prediction about the world; it is a design question.

Some of these ideas come from my book AI-agenten die het werk doen. Others come from the environment where I run agents myself: several AI providers are available in a set order, budgets and caps apply, and I trust a control only after I have seen it fail.

The scenarios, without drama

Five kinds of change matter. Each calls for something different in your design. None is a prediction. They are kinds of change that have occurred before, in technology or in the world around it.

1. A model or provider is no longer available. The cause may be minor or major: an hour-long outage, a suspended account, an expired key, a provider withdrawing from a region, or trade measures and sanctions changing what is available to certain countries or customers. To your agent, the result is the same: a door has closed. The duration makes the difference. An hour-long outage may call for a retry. An outage lasting weeks needs another route and decisions from people.

2. New or stricter rules. AI regulation is changing. In Europe, the AI Act has taken effect in stages. It classifies uses by risk: some are prohibited, some face strict requirements, some mainly have transparency duties, and others have very few. Which deadlines apply to your use depends on its category, and those deadlines can change. Check the current position and ask someone qualified. There are also rules about personal data and transfers outside Europe, as well as sector-specific rules. Financial institutions, for example, already face requirements for managing ICT suppliers, including plans for leaving a supplier. A rule may restrict a process, require data to be stored elsewhere, or require you to show what an agent did.

3. A provider changes the model. Providers improve models, replace them with successors and retire older ones. Usually they give notice, sometimes with little time. Even an improvement can affect your process: an answer that used to be three lines becomes five, a date appears in a different format, or a new caution appears where there was none before. This is drift. It is troublesome because nothing visibly breaks and no one gets an alert. The work continues, just a little differently.

4. Prices rise or limits shrink. A process that paid off yesterday may not today, or a generous limit may suddenly feel tight. An agent caught in a loop can also inflate the bill without changing anything visible from the outside. You can prevent some of this yourself.

5. Pressure in the surrounding supply chain. An AI model runs on chips, in data centres, through cloud platforms and networks. Each link is concentrated: only a few manufacturers make the most widely used chips, a small number of large providers run much of the cloud, and access to computing power can be affected by export controls or trade measures. For your design, the main point is that dependence on one provider often also means dependence on one country, one cloud or one kind of chip.

Each scenario calls for a different response. The first calls for a second route. The second calls for evidence and flexibility about where things are stored. The third calls for a way to measure behaviour. The fourth calls for limits. The fifth calls for spreading your dependencies.

The principle: dependence is not the problem; hidden dependence is

Start with a list: which processes use AI, through which provider, with which key, and who notices if it stops? You can manage only the dependencies you know about.

The goal is not to become independent of every provider. No one can, and there is no need to. A supplier may be indispensable for what it supplies. None should be indispensable for what belongs to you.

What belongs to you: the instructions your agents receive, the examples that tell you whether they are doing well, the records of what they did, your customers' templates and data, and the rules about who may do what. The model itself is the one part that truly belongs to the provider. If everything that belongs to you stays in your hands, the model becomes a replaceable component.

The layer between your work and the model

The most important design principle is also the simplest: do not let a process talk directly to a model. Put your own layer in between.

In practice, this can be a small piece of software or configuration with one job: accept a request, send it to a model, return the answer and record what happened. Your processes talk to your layer; your layer talks to the provider.

That gives you several things:

• Switching takes one change. If you change model or provider, you update one place instead of every process.
• You see everything in one place. Usage, errors, timings and costs pass through the same point.
• You can enforce rules. Data that must never go to an external provider can be stopped here, regardless of what a process asks for.
• You can fall back. The layer knows the next option if the first one fails.

More than one provider, in a set order

A second provider is the most direct response to the first scenario, and one of the most underestimated. Three things often go wrong.

First, the backup is never tested. A second provider that you configured once and never used is an assumption. Only regular real work or a scheduled trial can show that its key still works, the limit is sufficient and the result is usable.

Second, the switch is silent. If work quietly moves to another provider, no one notices that quality or cost has changed. If you want to know, someone needs to be told: a switch happened, and these are its effects.

Third, an instruction that works well on one model may produce sloppy results on another. Write instructions as neutrally as possible and test them on both.

In the environment where I work, providers have a fixed order: if the first does not respond, the work goes to the next. It is simple: a list with a clear rule above it.

A fallback ladder, not a switch

Fallback is rarely all or nothing. A useful picture is a four-step ladder:

1. The best model, the one you normally use.
2. A cheaper or smaller model, from the same or another provider: shorter, less detailed, and sometimes more error-prone.
3. A small model on your own equipment, in Europe or on your own computer: simple work, with no internet connection or someone else's key.
4. A fixed rule: no model, just a template or table. It can do little, but it is the one option that always works.

For each step, write down what you give up and what you keep. A cheaper model might write a shorter but correct email. A small local model might sort incoming mail. A fixed rule might send an acknowledgement so a customer knows someone is handling the matter. Some work continues, and a person handles only the difficult part.

Not every task belongs on every step. Three questions help. What does an error cost? A wrong date in a confirmation is expensive; a stiff opening sentence is not. Does the work go out without a person seeing it? If so, it stays on the highest step or waits. Can it be postponed? A task that can wait a day need not fall back; it can wait for the best model. That task does not fall back. This is a choice too, and it should be written down somewhere.

For a critical process, a model you can run yourself, on your own equipment or with a European provider, gives you a floor that no outside party can switch off. Not because it is better, but because it is yours.

Spot drift before a customer does

When a provider changes a model, you want to know before your customer does. The method is plain and reliable: a fixed test set.

Make a list of a few dozen questions typical of your process, and write down what a good answer should and should not contain for each one. Run the set against the model regularly, and always before and after a change. Count the differences rather than relying on a feeling. The goal is not perfection; it is to notice what has changed and review it.

Three habits make this practical:

• Pin the version where you can. Some providers let you choose a specific model version. Do so, and record when the provider will retire it so you can choose when to switch.
• Keep the test set yourself, not with the provider. Then you can run the same set against a second model and compare what to expect before switching.

Costs: a cap with a consequence

For the fourth scenario, limits are the first line of defence against rising prices and shrinking allowances. Set budgets per task and per day, rather than one limit for everything. One spike in one process should not take the rest with it.

The important question is what happens when a cap is reached. A cap without a consequence is an alarm no one hears. A cap that stops work without sending a notice is an outage with a tidy explanation. Both happen. You want it to be clear who gets notified, what work is left waiting and what continues when a limit is reached.

The bill itself is an early warning too. Compare today's usage with yesterday's, by task. A jump is a reason to investigate, even when everything appears to work. And calculate the cost of an output for each process, along with the price at which it stops paying off. Then you know in advance which processes can absorb a price rise and which should move to step two or three.

Keep control of what matters

The second scenario, regulation, raises another question: can you show what an agent did, and can you move your data?

• Keep instructions and examples as files. Store your agents' instructions, with their history, and example sets with good answers under your control, not only in a provider's interface. A control such as “send amounts above this limit to a person” belongs in your own layer, not in a sentence the model is expected to remember.
• Keep your own records. What an agent saw, decided and did should be somewhere you control, not only on a provider's dashboard. Then you can reconstruct what happened after an outage or a change.
• Keep a copy of your data and templates outside the service. That is also your way out if you need to switch.
• Know where your data goes. Which data goes to which provider, in what part of the world, and what happens to it? Six questions are a good start for any provider: where may my data go, what happens to my input, how are changes announced, what happens when a model is retired, what can I export, and what happens if there is a payment problem? Write down when you read the answers, because terms change.

The process must still work by hand

Technology catches a short outage; a long one needs an organisational plan. If the agent stops working for a week, can people do the work?

That is why every AI-powered process needs a description of its manual version: the steps, who does them, what tools they use and how long they can keep it up. Run the process without AI while it is working well. That can feel like a step backwards. It is what you need on the day it matters.

Think in three modes:

• Full: agents do the work, while people monitor from the edges.
• Limited: agents do the parts that fit the lower steps; people handle the rest.
• Manual: people run the whole process, using the lists and templates the agents used too.

For each mode, describe who decides to enter it, how everyone is told and how to return. That last part is often forgotten: returning takes work. Anything unfinished during the outage must be completed, and an alert should not keep firing after everything is working again.

Who decides, and what goes on one page

During an incident, a lack of decisions is a bigger risk than a lack of technology. Who may say that work should switch to manual mode? Who talks to customers? Who contacts the provider?

A one-page playbook is enough to get started. Put on it:

• the time limits: when a notification goes out and when a process moves to the next mode
• who decides on each mode, with a backup
• what you tell customers
• where the lists, templates and copies are kept
• who contacts the provider, and through which channel

The time limits are yours to choose. Every organisation runs on a different clock: a departure date, a parcel depot's closing time, a filing deadline. Set the limit using the clock that matters to the process.

Practise: pull the plug

A control you have never seen fail has not proved that it works. That applies to your backup, your alert and your playbook. So practise deliberately.

In a safe environment, you can simulate an outage in four ways: remove or invalidate a key, set a very low cap, block the connection to a provider, or choose a model name that does not exist in the settings. Each simulates one of the scenarios.

For each exercise, answer three questions in writing, with a time attached:

1. Does anyone notice? After how many minutes does an alert arrive, for whom and with what message?
2. Does the backup work? Is the result usable?
3. Does the work recover? When you restore the connection, does the system return to the higher step without duplicate or unfinished work?

An exercise produces a short list. Give each item an owner and a date, or it remains just a report. Repeat the exercise after every major change.

A checklist for each stage

When introducing an agent

• Does the process go through your own layer instead of talking directly to a model?
• Are its instructions files under your control, with a history?
• Is there a set of example answers for this process?
• Is there a cap, and do you know who gets notified when it is reached?
• Is there a description of how the process runs by hand?
• Have you recorded which data goes to which provider?

When scaling up

• Is there a second provider or model, and have you seen it work?
• Have you filled in the fallback ladder for this process, including what you give up at each step?
• Does someone get notified at every step down?
• Are the records under your control and easy to read back?

When maintaining it

• Does the example set run regularly and before and after each change?
• Is the model version pinned where possible, and do you know when it will be retired?
• Have you reread provider terms after a change?
• Have you practised an outage this quarter, and fixed what went wrong?
• Are your copies of your own data recent enough?

Questions for management

These are questions you can ask without being technical, and the answers tell you a lot.

1. Which processes use AI, and which are critical to customers or revenue?
2. Which providers are behind them, and what is the fallback for each critical process?
3. How long can we work without AI, and what would we do?
4. Who decides during an outage, and do they know it?
5. What is the maximum daily usage per process, and what happens when it is reached?
6. When did we last practise, and what went wrong?
7. Which of our data and instructions exist only with a provider?

You do not need to answer every question tomorrow. Start with the first. The rest follows from what you discover.

What it costs, and what it gives you

This takes work, but you do not need to do it all at once. A list of dependencies and a cap with an alert can be arranged in a day. Build the rest process by process, starting with the one that would cost you most if it stopped.

The benefit is hard to measure, and that is honest. It works like insurance: you notice it on the day something happens. What you can notice now is a clearer picture. You know what belongs to you and what sits with someone else. A change becomes an item on a list instead of a surprise.

So the answer to the question in the title is not that it cannot happen. It is that you can design things so you are ready when it does.

#AI #AIagents #WorkProcesses #RiskManagement

All pieces