Skip to content
Published on

Why PoCs Never Reach Production — Success Criteria, Security Review, Handover

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
Split View필사 모드
Share
Authors

The PoC Graveyard — How They Die

The most deflating scene in FDE work is a technically successful PoC disappearing in silence. The demo got applause, the point of contact was satisfied — and three months later, nothing has happened. Run the autopsy and the cause of death is usually one of five.

  • There were no success criteria. If nobody can rule it a success when it ends, the decision gets postponed indefinitely.
  • There was no champion. Plenty of people admired the demo, but without one person who will fight for it in the budget meeting, there is no next step.
  • The security review started last. When the security team first appears after technical validation ends, the review period kills the momentum outright.
  • The gap between PoC and production surfaced late. What was proven on dummy data, bypassed permissions, and a single server is not proof about production.
  • It missed the budget cycle. A PoC that ends out of sync with the decision-maker's budget calendar gets frozen until the next fiscal year.

Note that none of the five is a code problem. The stretch from PoC to production is decided by operational design, not engineering skill. And along this stretch the FDE role is two-layered: the engineer writing the code, and at the same time the de facto project manager tending success criteria, schedule, and stakeholders. The moment you decide the second layer is not your job, one of the five causes of death above gets reserved in your name. The four chapters below take that second layer in order.

Write the Success Criteria Before You Start

The success criteria document is written before the PoC starts, together with the customer, on one page. It needs all three of a number, a period, and a judge — missing any one, it is sentiment, not criteria.

[PoC success criteria — constructed example]
Goal      : cut agent document-search-to-first-response time by 40%
Measure   : search-to-first-response time over a 100-case sample, 2 weeks
Baseline  : current average 90s (agreed via 1 week of pre-measurement)
Judgment  : success at 54s or below; judge is the customer call-center ops lead
Failure   : if re-search rate exceeds 15%, ruled a failure regardless of speed
Next step : on success, scope and budget talks for pilot expansion within 4 weeks

The most valuable line here is the failure condition. A PoC with failure conditions written in advance leaves trust behind even when it fails, because everyone learned precisely what does not work. A PoC without them leaves suspicion behind even when it succeeds. And the last line — the promise of what happens on success — is what turns a PoC into a decision.

Security Review, Permissions, Data Boundary

The security review is not the final gate of a PoC; it is a week-one calendar item. Book 30 minutes with the customer's security team in the first week and agree on three things.

First, the data boundary. How far does customer data travel? Does it leave through an external API, and if so, which fields? What remains in logs? Is any of it used for model training? Draw the answers into a single diagram and agree on it, and most of the objections that would have arrived later get digested in advance. Second, permissions. PoC accounts are requested at least privilege, with an expiry date stamped on. The admin rights accepted for convenience become, in a later security audit, the exhibit that discounts the whole PoC. Third, agreement on the review timeline itself. Ask in week one which reviews a production transition requires and how many weeks they usually take, and the final-week surprise disappears.

The Debt Hiding Inside PoC Code

PoC code borrows deliberately in order to prove quickly. The problem is that the debt stays invisible while becoming the basis of a production decision. By the time transition is discussed, the debt list must be put on the table explicitly.

  • Credentials baked into config files — must migrate to a secrets-management system.
  • No error handling — code validated only on the happy path collapses at its first incident.
  • A single instance — restart, scaling, and deployment strategy are all undecided.
  • No monitoring — in production it can die with nobody noticing.
  • Bypassed auth — the SSO integration switched off during the PoC is often half of the transition work.

Raising this list in front of the customer first is FDE honesty, and simultaneously the basis of the transition estimate. It is also the answer to "the demo worked, so why does production take three months?" One caution: speak the debt list in the language of estimates, not threats. "Paying off these five items takes this many weeks" sounds to a customer like a plan rather than a cost, and the more concrete the list, the faster the transition budget gets approved.

The Handover Document — for the Day You Leave

The FDE success condition is peculiar: it only counts as success if it runs without you. If the system stops on the day the residency ends, what happened was a rental, not a deployment. That is why the handover document is built from mid-project, not in the final week.

[Handover document outline — constructed example]
1. System overview   — one architecture diagram and the data flow
2. Operations        — start, stop, deploy, backup, restore
3. Incident runbook  — the first 30 minutes for each frequent symptom
4. Access & contacts — account list, approvers, escalation path
5. Known limits      — what does not work, open issues, temporary fixes
6. Growth roadmap    — what was discussed as next steps, what was deferred

The document customers reopen most often is the incident runbook. It is for the same reason that the Handover role in the FDE Career RPG is designed to be playable only on missions that have a runbook: a system without one is a system that cannot be handed over. And on handover day, do not hand over only the document — completion is a rehearsal where a customer engineer works through one incident scenario by the runbook, hands on keys.

Practice by Doing

The feel of PoC operations is trained by decision-making under pressure.

  • FDE Career RPG — the Pre-sales PoC role must satisfy success criteria inside a time limit, and the Handover role only starts when a runbook exists. The two axes of this post, turned into game rules.
  • FDE Curriculum Roadmap — check the domains a transition depends on, security and data pipelines included, as a checklist.

FDE Complete Guide series

Discuss on Twitter • View on GitHub