Split View: 100배 싸다는 주장은 과제를 좁혔을 때만 참입니다 — 검증과 손익분기

The Claim of 100x Cheaper Is True Only When the Task Was Narrowed — Verification and Break-Even

• When a post with 100x in the title lands in your channel
• What the original actually did
• Why retrieval suits this approach
• The numbers that can be confirmed and the numbers that cannot
• The point where post-training beats routing
• How to measure the boundary of a narrowed task
• The real cost is on the maintenance side
• The checklist to pass before adopting
• References

When a post with 100x in the title lands in your channel

A link goes up in the team channel. It says an open model was post-trained, beat a frontier model on a retrieval task, and costs 100 times less. And a question follows immediately: could we not do this too?

Instead of answering that question with "let us try it" or "that is marketing," it is far more useful to separate what can be confirmed in the original from what cannot. Because the conclusion is usually not one of those two but conditionally true.

What the original actually did

The post in question is Beating GPT-5.6 Sol on retrieval with 100x cheaper open models, published on 5 August 2026. It is a case report co-written by people from Castform and Neon, and summarized it goes like this.

A 4-billion-parameter-class open model was post-trained with reinforcement learning for a retrieval task. The training data was synthesized from their own corpus, and retrieval is a hybrid approach using keyword scores together with vector similarity. The reward function scores retrieval quality, citation accuracy, and final answer accuracy separately. During training, thousands of parallel rollouts each trigger dozens of calls, producing a very spiky load.

The key sentence of the original is this: on certain specific tasks, such as retrieval, a post-trained open source model can match or beat frontier models while costing orders of magnitude less per request. The qualifier specific tasks takes up half of that sentence.

Why retrieval suits this approach

It is no accident that this result came from retrieval. Retrieval has three conditions that make it good material for a narrow task.

First, success can be judged by code. Whether a retrieved document contains the supporting evidence, and whether a citation points to a real document, can be graded without a human. That means a reward function can be built, and that is the precondition for post-training.

Second, the output space is narrow. Forming a query, choosing results, and attaching evidence require almost no free-form writing.

Third, the domain corpus is fixed. The broad world knowledge that is the strength of frontier models is not much of an advantage here. The knowledge needed is inside the documents being searched.

If even one of these three conditions is missing, the probability of the same strategy succeeding drops sharply. When you plug your own task in, checking the first condition is the right order.

A counter-example makes it concrete. A support assistant answering customer inquiries satisfies none of the three cleanly. What counts as a good answer cannot be judged by code, the output is free-form, and common sense outside company policy is continually required. Move the same method here as-is and you are already blocked at the stage of building the reward function.

The numbers that can be confirmed and the numbers that cannot

Here is a part that has to be separated honestly. The concrete figures I was able to confirm in the original are on the comparison side: the statement that a single multi-turn retrieval request takes over 10 seconds with a frontier model and costs roughly $0.03 end to end.

Conversely, on the accuracy of the post-trained model — what percentage it scores, on which evaluation set, measured in what way — I could not confirm any independently verifiable figure in the original. There is a statement that performance is comparable, but there is no table in a form a third party could reproduce.

And this post was co-written by two companies that sell a database product and a training platform. That does not at all mean the results are false, but you should read it while allowing for the possibility that task selection and comparison conditions were set favourably to their side. So what to take from this post is not the numbers but the method and the conditions.

The point where post-training beats routing

If you have taken the method, the next step is the arithmetic. Post-training is a choice with large fixed cost and small variable cost; routing and prompt optimization are choices with small fixed cost and large variable cost. Which is better is determined by volume.

"""Post-training break-even: judge by payback point, not by percentage saved."""
from dataclasses import dataclass


@dataclass
class Option:
    name: str
    fixed_cost: float      # training, data construction, evaluation set creation (one time)
    monthly_ops: float     # serving infrastructure, retraining, on-call
    per_request: float     # cost per request

    def total(self, monthly_requests: int, months: int) -> float:
        return (
            self.fixed_cost
            + self.monthly_ops * months
            + self.per_request * monthly_requests * months
        )


def break_even(a: Option, b: Option, monthly_requests: int, horizon: int = 36):
    """Find the first month in which a becomes cheaper than b. None if there is none."""
    for m in range(1, horizon + 1):
        if a.total(monthly_requests, m) < b.total(monthly_requests, m):
            return m
    return None


frontier = Option("frontier API", fixed_cost=0, monthly_ops=0, per_request=0.03)
tuned = Option("post-trained 4B", fixed_cost=60_000, monthly_ops=4_000, per_request=0.0003)

for volume in (50_000, 500_000, 5_000_000):
    m = break_even(tuned, frontier, volume)
    print(f"{volume:>9,} requests/month -> payback: {m if m else 'none within 36 months'}")

What matters in this code is monthly_ops. Compare per-request unit prices alone and post-training always looks overwhelming, but in reality serving infrastructure, retraining, and on-call go out every month. A proposal that computes with this item set to zero is the error you see most often in practice.

How to measure the boundary of a narrowed task

There is another cost that does not enter the calculation. A post-trained model degrades quietly outside its boundary. A frontier model copes tolerably with unfamiliar requests, but a narrowly trained model is confidently wrong once it leaves the training distribution.

So what you need when adopting one is boundary detection. The minimum usable setup in practice has three parts: the distance between the input embedding and the centre of the training distribution, the model self-confidence on the result it produced, and whether the keyword score and the vector score diverge sharply in hybrid search. If even one of the three crosses a threshold, that request is escalated to the frontier model.

It is safer to set this fallback rate generously at around 20 percent at first and reduce it while watching real data. And that rate has to go straight into the break-even calculation.

The real cost is on the maintenance side

There is a reason the original talks at length about the load during training. Post-training is not a one-and-done job. It has to be run again when the corpus changes, again when the user query distribution shifts, and again when a hole in the reward function surfaces.

That is, this choice is not a decision to change a model but a decision to add one more operational object to the team. You need a training pipeline, a synthetic data generator, an evaluation set, a serving stack, and someone who knows all of them. On a small team this cost easily overwhelms the difference in per-request unit price.

The reward function in particular is not written once and done. Define retrieval quality as document hits and the model learns to produce broad queries with high hit rates; weight citation accuracy heavily and it can tilt toward safely copying long stretches of the source. Such biases only surface in production logs after training is finished, so it is realistic to schedule reward function revision and retraining as recurring work.

The checklist to pass before adopting

To sum up: only when you can answer yes to all five below do you go to the next step.

And one last thing. Even if you passed all five, the first thing to do is not post-training but trying to shrink the prompt and the context. A good share of per-request cost comes from input size rather than from the model unit price, and that can be attempted in a few days. Getting pulled in by the number 100x and opening a months-long project first is the most expensive mistake.

References

• Beating GPT-5.6 Sol on retrieval with 100x cheaper open models — neon.com, 2026-08-05 — the reinforcement learning post-training of a 4-billion-parameter-class model, the synthetic data, the hybrid search, the composition of the reward function, and the per-request time and cost level of frontier models are the contents of that post. Please read it allowing for the fact that it is a case report co-written by two companies that sell products.
• Accuracy figures and evaluation methodology on the post-trained model side were not confirmable in the original in an independently verifiable form. This post does not quote that part either.
• Related post on this blog: Coding agent spend is controlled by friction, not by caps
• The numbers in the break-even code are examples to show the structure of the calculation, not an actual quote.