Skip to content
Published on

What Stays Expensive — Four Things That Gain Value When Generation Gets Cheap

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
Split View필사 모드
Share
Authors

A List of Tools Goes Stale Fast

Ask what to prepare for and the most common answer is a list of things to learn. Lists are easy to read, and they go stale just as easily. Pull up a list from five years ago: half the entries no longer exist by name, and the surviving half got absorbed into something else.

Lists go stale not because the entries were badly chosen but because the axis was. A tool cannot be an axis. A tool is a variable. What you need is a test that survives the tools changing.

This series uses exactly one such test: the skills that hold value are the ones that stay expensive to verify. Writing code got cheap. Judging whether that code is right, and whether the thing was worth building in the first place, did not. Every capability covered in the next nine parts is derived from that one sentence.

Two Axes, Four Quadrants

Split any piece of work into two costs. The generation cost of producing the artifact, and the verification cost of confirming the artifact is right.

Verification is cheapVerification is expensive
Generation is cheapAlready automated, no value leftWhere most work has moved
Generation is expensiveThe next quadrant automation eatsWhere people still remain

Top left is format conversion, boilerplate, routine migration work. Anyone whose skill sat here has already watched its price fall.

Bottom left is the interesting one: hard to produce, easy to check. Algorithm problems with a known answer, transformations that pass or fail on compilation, optimizations with a benchmark score. When a scoring rule exists, a machine can run the trial-and-error loop for you, so this quadrant does not hold no matter how hard the generation is. The real precondition for automation is not that the problem is easy but that the output is gradeable.

Bottom right is hard to produce and hard to judge: choosing the migration path for a ten-year-old system, picking a design where regulation, contract, and performance all pull at once.

The Dangerous Quadrant Is Top Right

The quadrant to watch is the top right: cheap to produce, still expensive to judge. A large share of today's software work has moved into it.

It is dangerous because the output looks plausible. A wrong answer that looks obviously wrong costs little. The costly case is the nearly right answer. In the Stack Overflow developer survey published in 2025, 66 percent named "AI solutions that are almost right, but not quite" as their biggest frustration with AI tools, and 45 percent said debugging AI-generated code takes more time. In the same survey, 46 percent reported actively distrusting the accuracy of these tools against 33 percent who trust it.

A survey records perception, not performance, so these numbers must not be converted into a productivity conclusion. One thing does read clearly though: users are themselves reporting that the bottleneck moved from producing to judging.

What Makes Verification Expensive

So the next question is what drives verification cost up. Broadly, four things.

First, tacit context. When the information needed to judge lives neither in the code nor in the docs but only in someone's memory and the team's habits, only the person holding it can judge.

Second, delayed consequences. A design can be green today and collapse in production six months out. The later the result arrives, the more judging it today depends on prediction built from experience.

Third, the absence of an oracle. Ranking quality, whether an interface is good, whether a model output is appropriate — in domains with no answer key, the judgment itself has to be designed.

Fourth, accountability. Somebody has to put their name on the decision, and a signature does not delegate. Signing requires understanding, which makes accountability a device that forces comprehension.

The Twist — The Job Is Making Expensive Verification Cheap

Read only this far and the conclusion turns strange. If expensive verification raises my value, then keeping verification expensive would be in my interest. That is wrong, and the place where it goes wrong is where this frame connects to practice.

The engineering job is the exact opposite: build the machinery that turns expensive judgments into cheap ones. Pin down with types what people used to have to read for, pull forward to deploy-time signals what used to surface in production six months later, and stand up proxy metrics with sampled review where no answer key existed.

So the statement to keep is this: performing the judgment gets automated, designing the judgment stays. It is the same structure as a good test being expensive to write and free to run. Every capability in this series lives on the design side.

Do Not Use Your Own Perception as Evidence

One last thing to flag. In this territory your own sense of speed is not reliable evidence.

The randomized controlled trial METR published in July 2025 shows this well. Sixteen experienced developers working in open-source repositories they knew well completed 246 tasks, with each task randomly assigned to allow or disallow AI tools. The result: tasks where the tools were allowed took 19 percent longer. Participants had expected a 24 percent speedup going in, and after the study still believed they had been sped up by 20 percent.

Do not over-generalize this. The researchers themselves note the sample of 16, that participants were highly familiar with their own codebases, that the repositories held rigorous quality standards, and that outcomes might differ with more tool experience. What the trial establishes is not that the tools are bad but that a gap can exist between feeling faster and being faster, running in opposite directions.

That is why every recommendation in this series carries the same caveat: confirm it with a record. A feeling is a hypothesis, and hypotheses have to be measured.

Try It This Week

If you pick one thing, pick this. Write down ten tasks you did in the last two weeks and drop each into one of the four quadrants above. If a task resists placement, that is the most important signal you will get, because it means you have not yet settled how that work gets judged.

  • Engineering Skill Path — Linux, networking, databases, Kubernetes, and AI/LLM tracks in unit-sized lessons, useful for locating where you actually stand.
  • Problem-Solving Trainer — nine named moves for a problem you have never seen before, the closest available practice for building a judgment standard.

Where this frame does not apply, stated plainly: on an early product with few users and low failure cost, standing up judgment machinery before shipping is the larger loss. Verification design earns its keep once something is at stake.

Further Reading

What Stays Expensive series

Sources

Discuss on Twitter • View on GitHub