The prototype is not the product: where AI projects get harder

An AI prototype can be persuasive very quickly. Give a model a narrow task, a few good examples and a clean demo path, and it may look like most of the work is done.

It isn't. The difficult part is deciding what happens when the input is incomplete, the model is wrong, or the person using it expects an answer you cannot safely give.

Take a customer enquiry arriving through a website. A model might classify it, draft a reply and suggest the next step. In a demo, the enquiry is clear and the reply looks sensible. In production, the customer might mention two jobs, an address that doesn't match the service area, or a price agreed in an earlier conversation. The system has to know which facts it can rely on, what needs checking and when to hand the conversation to a person.

Those are product decisions, not prompt tweaks.

We ask three questions before treating an AI workflow as ready:

1. What is the model allowed to decide? Drafting a reply and sending it are different levels of authority. Make that boundary explicit.
2. What happens when it is unsure? A useful fallback might be a question, a human review queue or no action at all. Silence and confident guessing should not be the default.
3. How will we know it is working? A polished answer on a demo input is not a measure. Look at the actual cases coming through, the corrections people make and the outcomes the workflow was meant to improve.

In practice that means we start small. With the enquiry example, that might be letting the model sort enquiries and draft replies while a person still presses send, and keeping a note of every draft they had to fix. It is a less exciting demo. It is also the version you can actually learn from and safely improve.

The model matters, of course. But the value of an AI product is often in the decisions around it: what it can see, what it may do, how it recovers, and who remains accountable when it gets something wrong.