Skip to content
←

필사 모드: Five Habits for Working Well with AI Coding Tools

English
0%
정확도 0%
💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — The Data Is In; What Is Left Is Behavior

In an earlier post, Does AI Actually Make Developers Faster, I walked through two well-designed randomized controlled trials that reached opposite conclusions. In one, developers were 55.8% faster. In the other, they were 19% slower.

That post was about what got measured. This one is about what you do differently tomorrow morning.

The five items below are rules, not vibes. Each is derived from an actual study, and each ends in one line you could falsify within two weeks.

Habit 1 — Pick Tasks by Familiarity and Cost of Verification

METR narrowed down five factors that likely contribute to the 19% slowdown. Two of them are about the relationship between the developer and the code: high developer familiarity with the repository, and implicit repository context — the undocumented knowledge the model has no way to reach.

The sharpest finding is this one: developers were slowed down more on tasks they knew better. Familiarity was not a shield; it was a tax.

The Copilot experiment sits at the opposite pole. Greenfield, clearly specified, and verifiable in seconds. And its gains were largest for the least experienced developers.

Compress both results into one sentence and you get: AI wins when it fills in context you lack, and loses when it has to catch up to context you already have. Which gives you two axes.

Verification is cheap (seconds/minutes)Verification is expensive (days/months)
UnfamiliarThe sweet spot. Use it freely.The danger zone. You cannot review what you do not understand.
FamiliarFine. Small gains, small losses.METR's quadrant. Writing it yourself is faster.

The top-right cell is the real trap. AI output from there turns quietly into debt, and the invoice arrives months later.

Rule: Before delegating, answer two questions in one line each. (1) If this is wrong, when do I find out? (2) Would I recognize a wrong answer? If (2) is "no" and (1) is "in a few months," what you need is not a better tool — it is to learn the area first.

Habit 2 — Never Merge What You Cannot Review

METR's participants accepted fewer than 44% of the code the AI generated. And 56% reported needing major changes to clean up AI code. Generation got cheap; the reading, judging, and discarding did not — and that time leaves no artifact behind.

The 2025 Stack Overflow survey (tens of thousands of respondents in the AI section) tells the same story from another angle.

  • The top frustration, cited by 66%, is AI answers that are almost right, but not quite.
  • Second, at 45.2%: debugging AI-generated code takes longer.
  • And 75.3% say that when they do not trust the AI's answer, they end up asking another person.

Here is the point. Obviously wrong code is cheap to catch. The expensive kind is code plausible enough to survive review and wrong enough to cost you. Which means review is not the chore that comes after the real work. Review is the work.

One piece of self-deception worth naming: asking the model to explain its own diff is not verification. The explanation is also generated. Real verification means running it, testing it, and checking it against invariants you actually know.

Rule: Apply the same size cap to AI-written PRs that you apply to human-written ones. A diff too large to review is a diff too large to merge. And put review time in the calendar as an actual budget line.

If verification is the bottleneck, readable code stops being a matter of taste and becomes a throughput problem — exactly the argument in Write Code for Humans and LLM Burnout.

Habit 3 — Build Machine Guardrails, Not Prose Rules

Anthropic's Building Effective Agents recommends poka-yoke in tool design: change the arguments themselves so it is "harder to make mistakes." The example in the piece captures the spirit exactly — when the model keeps getting relative paths wrong, you do not add a warning to the prompt; you change the tool signature to require absolute paths. The same document argues you should invest as much in the agent-computer interface (ACI) as in human-facing interfaces, and that you should actually run your tools to see what mistakes the model makes.

The principle that falls out is simple. A sentence in a rules file is a request. A failing type check is a fact. Prose is obeyed probabilistically; a compiler is obeyed deterministically.

// Prose rule: "never use a raw number for currency" -> the model may or may not comply.
// Machine guardrail: make it impossible to get through in the first place.
type Cents = number & { readonly __brand: 'Cents' }

export function charge(amount: Cents): Promise<Receipt> {
  /* ... */
}

charge(1000) // Type error. Human or model, you get stopped right here.
charge(toCents(1000)) // Passes.

None of this is new to the AI era. What changed is that as code volume goes up, the return on guardrails goes up with it.

Rule: The second time you correct the model on the same mistake, stop writing rules and write a check. Move it into a type, a lint rule, a test, a CI gate, or a narrower function signature. If it cannot be moved, it was probably a rule humans were not following either.

Habit 4 — Give It the Right Slice, Not Everything

One of METR's factors was implicit repository context: the knowledge that lives in a five-year veteran's head and is written down nowhere.

This is where a common mistake begins. "Fine, then just give the model everything." No. Shoving 200 repository files into the context window does not tell it your invariants. An invariant is usually one sentence: this table is append-only; this function is only ever called inside a transaction; this cache is not cleared on deploy. Two hundred files will not say that. One sentence will.

Anthropic's guidance points the same way: start with the simplest solution possible and add complexity only when needed, since a single call with retrieval and good examples is often enough. Context engineering is not the art of adding more. It is the art of deciding what to leave out.

Rule: Before you start, write down the three to five constraints a newcomer to this code would violate. That list is your context. If you cannot write it, you are standing in Habit 1's danger zone.

There is a useful side effect. That list is almost always usable as team documentation. The tacit knowledge you were forced to write down for the model turns out to be knowledge your teammates did not have either.

Habit 5 — Measure Your Own Throughput, Not the Feeling

The most uncomfortable number in METR is not 19%. Before starting, developers forecast that AI would make them 24% faster. After actually living through a 19% slowdown, they still estimated they had been 20% faster. Even hindsight had the sign backwards. Economics and ML experts got the direction wrong too.

The reason is mundane. Generation is visible and verification is not. The moment the model emits 40 lines in three seconds is memorable. The 20 minutes you spend reading them and throwing half away just feels like "working."

So surveys and retrospectives are the least trustworthy instruments on this particular topic. For exactly that reason, self-reported datasets like DORA and Stack Overflow should be read on the assumption that they lean optimistic.

Rule: Two weeks, two columns. Before a task, write your estimate. After, write the actual time and whether you used AI. But measure time-to-merged-and-stable, not time-to-first-diff. The tax on code that is "almost right, but not quite" is always billed after the draft.

Where AI Genuinely Shines

This post should not read as "do not use AI," so let me be specific about where the gains are real.

  • Greenfield work with a clear spec. The 95 developers in the Copilot experiment wrote an HTTP server in JavaScript. The Copilot group averaged 1 hour 11 minutes; the control group 2 hours 41 minutes. That number is real.
  • Unfamiliar languages, APIs, frameworks. The model has read more of those docs than you have. This is Habit 1's sweet spot.
  • Anything verifiable in seconds. Throwaway scripts, scaffolding, test fixtures, regexes, data munging. If it is wrong, you know immediately.
  • Work in the reading direction. Understanding code someone else wrote. Its claims can be checked against the code right there, so verification is cheap.
  • Mechanical bulk changes in well-tested areas. The guardrails are already standing.

And the Copilot finding that gains were largest for less experienced developers is, as the authors put it, a promising signal for people transitioning into software careers.

Honest Limits on These Rules

METR's limits. Sixteen participants, 246 tasks, early-2025 models, large and mature open-source repositories. The authors state explicitly that their work does not show that AI fails to speed up most developers, nor that there is no better way to use existing AI tools.

More importantly, METR is revising its own design. In a February 2026 follow-up, they acknowledged serious selection effects: some developers declined to participate at all because they did not want to work without AI, even at 50 USD an hour; 30 to 50% of developers never submitted tasks they thought AI would handle well; and time tracking broke down once people ran several agents concurrently. Their newer data actually hints at a speedup — but METR itself calls that "an unreliable signal of the current productivity effect."

The Copilot study's limits. One task, greenfield, and what was measured is completion time, not quality or maintainability. It is also worth recording that some authors were affiliated with GitHub.

The survey's limits. Stack Overflow is self-reported. And for precisely the reason METR demonstrated, self-report is the least reliable instrument available on this topic.

So the five habits above are not laws. They are heuristics drawn from the best evidence currently available. Try to falsify them with your own log. That is rather the point of Habit 5.

Closing

Compress all five into one sentence: AI wins when it fills in context you lack, and loses when it has to catch up to context you already have. The other four are operational details of that sentence — choose tasks to match it (1), budget verification as first-class work (2), hand verification to machines (3), compress and deliver context (4), and measure whether any of it is working (5).

One last thing. No rule in this post beats two weeks of your own log. It is not an accident that the fifth habit comes last.

References

현재 단락 (1/63)

In an earlier post, [Does AI Actually Make Developers Faster](/blog/2026-07-12-ai-developer-producti...

작성 글자: 0자원문 글자: 10,488자작성 단락: 0/63