The gap

A demo proves it can happen once. Production is a promise about every time after that.

Most enterprise AI dies between the demo and the deployment. Not because the model was wrong, but because a notebook that works has no input contract, no retry behaviour, no record of what it did, nothing versioned to roll back to and nobody on call. This is what that distance is made of, what closing it costs, and how much of it your case actually needs.

Talk to us about a system How we run an engagement

LayerData & ingestion · Models & fine-tuning · Systems of recordthe whole stack

The demo that works

Seven things a working prototype does not have.

The prototype is not a lie and it is not wasted. It establishes one thing, precisely: that the model can do this task at least once, under supervision, on inputs a person chose. Everything a production system claims is a claim about the other times.

The usual shape is a notebook, or a small script with an API key in it, driven by the person who wrote it, over twenty or thirty inputs they picked themselves. It answers well. Someone shows it to the operations director and the room agrees this is obviously worth doing. That agreement is real evidence and worth having — most of the AI work that fails, fails earlier than this, on a task that was never worth automating. But the demo has proved the smallest possible version of the claim, and the meeting usually hears a much larger one.

Here is what it does not yet have. One: an input contract — a written and enforced statement of what an incoming record must contain, what may be null, and what happens to a record that does not match. Two: behaviour for the second time the same request arrives, which in any queued or retried system it will. Three: a record of what it did — which prompt version, which model, which retrieved passages, how long it took and what it cost. Four: versioning, so that there is a previous known-good state to return to, and this has to cover the prompt, the model identifier and the index, not just the application code. Five: a boundary on who may see what, because a demo run by its author under his own credentials has never once been asked whether this user is allowed to read this document. Six: a definition of a correct answer that survives someone changing the prompt — that is, an evaluation set, rather than the author's memory of how it behaved last Tuesday. Seven: an owner, meaning a named person with a runbook, an alert threshold and a rota, rather than the developer who built it and has since moved to another project.

None of those seven are model work. They are ordinary software engineering, of the sort that has been well understood for thirty years and is not made obsolete by anything in a language model. That is exactly why the gap is so often underestimated: the demo is the part that gets applauded, and the seven items above are the part that nobody schedules because none of them make a better slide. A project that budgets eight weeks for the model and two for everything else has the ratio backwards, and will discover this in month four.

The opposite failure is just as expensive and less often warned about. A team reads a list like the one above, decides it needs a platform, and spends two quarters building deployment machinery, a feature store, a bespoke evaluation harness and a monitoring stack for a tool that will serve forty requests a day to eleven people in the finance team. Hardening is proportional to blast radius. An internal summariser that a person reads before acting on needs an evaluation set, cost visibility and a rollback path, and needs very little else. A system that issues credit notes without a human in the loop needs all seven and an audit trail besides. The engineering question is not how much rigour exists in the world; it is how much of it this particular failure would justify.

The order matters as well, and it is not the order teams pick on their own. Contracts come first, because everything downstream is defined in terms of them. Evaluation comes second, because without it every later change is a guess. Retry behaviour, observability and rollback follow, in that sequence, because each one is cheaper to add before there is traffic than after. Ownership is last only in the sense that it is settled last; if it is not settled at all, everything above it degrades quietly over about six months until someone notices that a number in a report has been wrong since March.

A demo proves the model can do the task once. Production is the claim that it will do it on a Sunday, at volume, when the index is three weeks stale and the person who built it has left.

What closes the gap

Five pieces of engineering, and the incident each one prevents.

None of these are exotic. Each is the thing that, when absent, produces one specific and entirely predictable failure — so the honest way to read this list is as five incidents you can choose to have or not have.

01
Data contracts
Input and output schemas, validation at the boundary, explicit nullability, versioned interfaces between caller and service
A written and enforced statement of what goes in and what comes out. Every field named, every type fixed, every optional field marked optional on purpose rather than by accident, and a rule for what happens to a record that fails validation — rejected with a reason, or quarantined for a person, but never silently accepted. The output side matters as much as the input side, because whatever consumes the answer has to be able to parse it without guessing, and a model asked for structured output will occasionally produce something adjacent to the structure you asked for. The cost is real: contracts slow down the first two weeks, they make small changes require a version bump, and they will reject records that a human would have understood. That is the trade. You are buying a system that fails loudly at the door instead of quietly in the middle.
A supplier changes an upstream export and starts sending the date as a string rather than an ISO timestamp. Nothing errors. The model reads it, forms a plausible view, and for eleven days every record from that supplier is processed against the wrong month. The discovery is a person noticing an odd total, and the remediation is a reprocessing job over eleven days of records that nobody kept the originals of.
02
Idempotency and retries
Idempotency keys, at-least-once delivery, timeouts on every outbound call, exponential backoff with jitter, dead-letter queues
Idempotent means an operation can be performed twice and the second time changes nothing. This matters because in any real deployment messages arrive twice: a queue redelivers, a network call times out after the work was actually done, an operator replays a batch. The mechanism is a key derived from the request itself, checked before the side effect, so the second arrival finds the first result and returns it rather than doing the work again. Retries are the other half — every outbound call gets a timeout, failures are retried with an increasing and slightly randomised delay so that a recovering service is not immediately flattened by every client retrying in unison, and anything that fails repeatedly lands in a dead-letter queue where a person can look at it rather than disappearing. The cost is that idempotency requires storing keys and results, which is state you now have to size, expire and back up.
A queue consumer times out after the model call completes but before the acknowledgement lands. The message is redelivered. The system, having no memory of the first attempt, does the work again — and the work in this case was raising a credit note. The customer receives two. The reconciliation takes a fortnight, and the queue redelivered forty-one messages that night, not one.
03
Observability
Structured logs with a trace id across every hop, token and currency cost per call, latency percentiles, sampled prompt and response capture, redaction and retention policy
The ability to answer, after the fact, why the system produced that answer. In practice that means a trace identifier that follows a single request through every hop, structured logs rather than free text, and for each call: the prompt version, the model identifier, the retrieved passage identifiers, the token counts, the cost in currency, and the latency. Percentiles rather than averages, because the average is fine while the ninety-ninth percentile is what your users are complaining about. Then a sampled capture of full prompts and responses, because aggregate numbers tell you that quality moved and only the text tells you why. The costs here are specific and worth stating plainly: log volume from a chatty system is a real line on a bill and often exceeds the inference spend; prompts contain whatever the user typed, which means logging them is a data protection decision and not an engineering one, requiring redaction of personal data at the point of capture and a retention period someone has signed off; and every field logged is a field that must be maintained. Log what you would need to answer a complaint or reproduce a bad answer. Logging everything is not thoroughness, it is a liability you pay monthly to store.
A user reports that the assistant gave a wrong figure last Thursday. There is no trace, no record of which passages were retrieved and no record of which prompt version was live that day, because the prompt was edited twice since. Nobody can reproduce it, so nobody can say whether it is fixed. The system is now unfalsifiable, and the honest answer to the complaint is that you do not know.
04
Rollback and versioning
Version pinning for prompt, model identifier and index snapshot; one-action revert; a rehearsed rollback drill
Three things in an AI system change independently and any of the three can break the answers: the prompt, the model behind it, and the index the answers are drawn from. Application code is usually already versioned; these three usually are not. Prompts get edited in place, model identifiers get pinned to a floating alias that quietly moves to a new version underneath you, and indexes get rebuilt over the top of themselves. So all three get a version, every deployment records which combination was live, and reverting is one action rather than an archaeology exercise. The index is the one most often forgotten and the most awkward to undo, because a rebuild that dropped half the corpus cannot be reversed by editing a file — it needs a retained previous snapshot, which is storage you have to pay for and prune. Rehearse the rollback before you need it; a rollback path that has never been executed is a hypothesis.
An overnight re-index runs against a source export that was itself incomplete. The index rebuilds cleanly over the old one, so there is no error and no previous snapshot. By morning the assistant is answering confidently from a corpus missing a third of its documents, saying nothing about the gap, and the only route back is a full re-ingestion that takes nine hours and cannot begin until someone works out which export was wrong.
05
On-call and ownership
A named owner, a rota, a runbook, alert thresholds tied to user-visible symptoms, an escalation path, an agreed service level
A system that nothing and nobody owns decays on a schedule. Ownership means a named person, not a team inbox; a runbook that says what the common failures look like and what to do about each, written so that the person on call at two in the morning who did not build this can act; alert thresholds set on symptoms a user would notice — answers refused, latency past the agreed number, cost per day past the agreed number, retrieval returning nothing — rather than on the metrics that were easy to emit; and an escalation path with a second name on it. The cost is not the tooling, which is minor. The cost is the standing claim on someone's attention, and the fact that an on-call rota needs enough people in it to be humane. This is the single item most often left as an implementation detail at handover, and it is the one that determines whether the system is still trusted in a year.
Retrieval quality drifts as the corpus grows and nobody owns the number, so nothing is watched and nothing fires. The desk that used the assistant quietly goes back to searching the shared drive, because it was wrong twice and no one had anywhere to report that. Six months later a review finds a system nobody has logged into since spring, still running, still costing money, and no longer trusted by anyone who once used it.

The sequence

How a prototype is actually hardened, in the order it has to happen.

Each stage produces something you can point at. If a stage ends without its artefact, it has not ended — and the next one will be built on an assumption rather than a decision.

01
Fix the boundary
A versioned input and output schema, with a written rule for every rejection case and a decision on what happens to rejected records.
Before anything is rebuilt, write down what the system takes and what it returns, and make the code enforce it. This is where scope is decided in practice: the boundary defines what is inside the system and therefore what you are responsible for. Half the arguments that surface in month three are boundary arguments that were never had in week one — whose job it is to deduplicate, what happens to a record with a missing customer reference, whether a partial answer is acceptable. Have them now, in writing, when they cost an afternoon.
02
Build the evaluation set before you change anything
A held-out set of scored cases, a baseline score per category, and a pass threshold that has been agreed by the people who will use the system.
Collect real cases — including the ones the prototype got wrong and the awkward ones people remember — and have someone who knows the work write down the correct answer for each. Score the prototype against that set as it currently stands. This number is the baseline, and it is the only thing that will tell you whether the next four weeks of engineering improved the system or merely changed it. A hundred cases scored by someone who understands the domain is worth more than ten thousand scored automatically against a definition nobody agreed. Expect this stage to be the one the business finds hardest, because it requires deciding what correct means.
03
Harden the path, only as far as the failure justifies
A service with a timeout on every outbound call, an idempotency key on every write, a dead-letter queue with somebody assigned to read it, and a written decision on degraded-mode behaviour.
Now add the timeouts, the retries with backoff, the idempotency keys on anything with a side effect, and the dead-letter queue. Decide the behaviour when the model provider is unavailable — queue and wait, degrade to a simpler path, or stop and tell someone — because it will be unavailable, and the default behaviour of an unconsidered system is to fail in whichever way is most confusing. This is also the stage to resist building a platform. Ask, for each mechanism, what it would cost if this failed at your actual volume. If a wrong answer costs an apology, a retry policy and an alert are enough. If it costs a payment, do the work properly. Uniform rigour applied to a non-uniform blast radius is how six-month deliveries happen.
04
Instrument it, and decide what you are not logging
A dashboard showing volume, latency percentiles, refusal rate and cost per day; a written redaction and retention policy; and an owner named on the budget line.
Add the trace identifier, the structured logs, the cost and latency capture and the sampled prompt and response records. Then do the part that is usually skipped: decide what is deliberately not captured, redact personal data at the point of capture rather than in a later cleanup, and set a retention period that someone in the business has actually agreed to. Put cost per day and per unit of work in front of the person who owns the budget from the first week, not at the first invoice. Costs in these systems are usually driven by prompt length and by traffic patterns nobody predicted, and both are visible weeks before they are painful.
05
Release narrowly, then hand over properly
A runbook that a named on-call owner has used at least once, a rehearsed and timed rollback, and an agreed date at which the previous process is switched off.
Put it in front of one team, or one queue, or a percentage of traffic, with the previous process still running alongside for as long as it takes to build confidence. Compare the two on the evaluation categories rather than on impressions. Then rehearse the rollback while everyone still remembers how the system is put together — an unrehearsed rollback is a plan, not a capability. Handover is finished when a person who did not build the system has fixed something in it using the runbook, and not before. Everything up to that point is a demonstration of handover rather than a handover.

Tell them apart

Ready for production. Or a demo wearing a deployment badge.

Both are running in a cloud account with a URL and a login page. The difference is not visible from the outside, which is precisely why the second one gets signed off. These are the questions that separate them.

Ready for production

A record that fails validation is rejected with a reason and lands somewhere a person will see it, rather than being processed on a guess
The same request arriving twice produces one outcome, and someone can show you the key that makes that true
Any answer from the last ninety days can be traced back to the prompt version, model and passages that produced it
Prompt, model and index are each versioned, and someone has actually executed a rollback and knows how long it takes
There is a scored evaluation set with an agreed pass threshold, and it is run before a change ships rather than after a complaint
One named person owns it, alerts fire on symptoms a user would notice, and the runbook has been used by someone who did not write the system

A demo wearing a deployment badge

It works on the inputs it was built against, and nobody has written down what an input is allowed to look like
Retries exist, but nothing checks whether the work was already done, so the failure mode is doing it twice
The logs record that a request happened and nothing about what it decided, so no complaint can be reproduced
The prompt is edited in place in production, the model is pinned to an alias that moves, and the index is rebuilt over itself
Quality is assessed by whether anyone has complained this week, which measures reporting habits rather than the system
Ownership is a team channel, alerts fire on infrastructure metrics that nobody maps to a user-visible symptom, and the runbook is a wiki page written the day before handover

Asked often

The questions that come up when the demo has just been approved.

How long does hardening take?: For a contained internal system with one integration and a person reading the output before acting on it, four to eight weeks is a normal range, and most of that is evaluation and integration rather than model work. For something that writes to a system of record without a human in the loop, or touches money, or serves customers directly, plan for three to four months and expect the access and data-protection reviews to sit on the critical path rather than the code. The honest predictor is not the model. It is how many systems you have to integrate with, how clean their data is, and how quickly your organisation can decide what a correct answer looks like. A team that can answer that question in a week has a project half the length of a team that needs a committee.
Can we launch to a small group first?: Yes, and you should — with one caveat that is regularly missed. A narrow release is how you find the input shapes nobody thought of and the questions nobody predicted, and it is far cheaper to be wrong in front of eleven people than eleven hundred. The caveat is that a small group is usually a self-selected, sympathetic group who will work around problems rather than report them, so their satisfaction is not evidence of quality. Instrument the pilot properly, keep the previous process running beside it, and judge it on the evaluation categories and on what the logs show rather than on whether the pilot users are pleased. Also agree in advance what result would cause you to stop, because a pilot with no failure condition is a launch with extra steps.
Who runs it after handover?: Someone in your organisation, named, with the time for it in their week — and this should be agreed at the start of the project rather than discovered at the end of it. Our arrangement is usually a period of shared operation where our engineers are on the rota alongside your team and the incidents are worked together, then a defined handover point after which your owner leads and we are the escalation. What does not work, in our experience, is handing a runbook to a team that first sees the system on the day it becomes theirs. If nobody internal can own it, that is worth knowing in week one, because it changes what should be built — a system with no owner should be simpler, smaller and more conservative than one with a good one.
What breaks most often?: Not the model. In order: upstream data changing shape without notice, which is why contracts come first; retrieval quality drifting as the corpus grows, so answers that were good in month one are mediocre by month six and nobody notices because there is no scored set; cost rising quietly as prompts grow and traffic patterns shift; and integration credentials, permissions and rate limits at the boundary with existing systems. Model provider outages happen too, but they are brief, visible and easy to plan for, which makes them much less dangerous than the four above. The pattern is that the failures that hurt are the ones that produce a plausible answer rather than an error.
Do we need MLOps tooling for this?: Usually less than you have been told. The specific tooling category matters far less than whether the six behaviours exist: versioned prompts, pinned models, snapshotted indexes, a scored evaluation set, traceable requests and a rollback that someone has rehearsed. All six can be had with your existing source control, your existing deployment pipeline, your existing logging stack and a scoring script that runs in continuous integration. That is what we build first, because it is what we can hand over to a team that has to maintain it. Buy a dedicated platform when you are running several models across several teams and the coordination cost is genuinely being felt — which is a real point that some organisations reach. Buying it for a first system means learning a platform and a problem at the same time, and the platform will not tell you what a correct answer looks like.

In practice

The demo was the easy half. It was also the half worth doing first.

None of this is an argument against prototyping. Build the demo, show it, find out quickly whether the task is worth automating at all — that question is answered far more cheaply in a notebook than in a design document. The mistake is not the prototype. The mistake is treating the applause as the finish line, and budgeting the remaining work as a fortnight of tidying up.

The distance is mostly ordinary engineering: contracts, idempotency, observability, versioning and ownership. It is unglamorous, it is well understood, and it is the whole difference between a system your business can depend on and one it quietly stops using. How much of it you need is a question about consequences, not about ambition — and we would rather tell you that your case needs four weeks of hardening than sell you a platform to run one tool.

If you have a prototype that works and you are trying to work out what stands between it and Monday morning, that is a conversation we have most weeks and it is a short one.

Start a project How we run an engagement

How you know it works

How anyone can tell whether an AI system is actually working, rather than merely looking like it did in the demonstration.

Read

The last mile into your systems

What has to be true for a system to write into your ERP, HRMS or CRM — and why this is the layer that decides whether any of it was worth building.

Read

Where your data goes

When an AI system reads your documents, what leaves your control, what stays, and how would you prove either to a security reviewer?

Read

Loading…

FAQs

Frequently Asked Questions

Industrial AI, robotics conversion and adoption, Industry 4.0 automation and test automation, and the software those systems run inside: enterprise systems (ERP, HRMS, CRM, inventory and production management), web applications, AI agents and automation, and native mobile apps for iOS and Android.

A focused first release typically ships in 8–12 weeks. Multi-module enterprise platforms run 4–9 months, delivered module by module so you are using working software long before the final release rather than waiting on a single big-bang launch.

Both. A well-defined scope gets a fixed price against a signed specification. Longer or evolving builds run as a dedicated team on a monthly retainer. Either way you get a written estimate with every assumption listed before a contract exists, so you can see exactly what moves the number.

Discovery and scoping, then UI/UX design, then two-week build sprints that each end in a working demo. You get a staging environment from the first week, and QA, security review and deployment run inside the sprint instead of being bolted on at the end.

You do, in full, on final payment. Source code, design files, infrastructure and documentation are handed over in your own repositories and cloud accounts. We keep no licence over your product and there is no lock-in to our hosting.

Yes. We connect to MES and the equipment already on the line, accounting, HR and CRM platforms, payment gateways, WhatsApp Business, and legacy in-house databases. Where no API exists we build a sync layer and migrate historical records with reconciliation reports your finance team can check line by line.

Every project closes with a handover: documentation, team training, and a warranty period covering defects at no cost. After that you can run support in-house or keep us on a monthly plan covering monitoring, security patches, and an agreed block of enhancement hours.

We sign your NDA before the first technical conversation. Builds ship with role-based access, encryption in transit and at rest, and audit logging as standard. Production data stays inside your infrastructure and access is limited to the engineers assigned to your project.

Start a project

Didn’t find what you were looking for?

Tell us what you are building. You get scope, a timeline and a realistic cost — not a brochure.

Tools

The stack we build on

We design, build, and evaluate with a modern AI stack—LLMs, vector search, orchestration, and observability—so your features are fast, reliable, and secure.