The Economics of Refactoring: When Does It Pay Off — Calculating With Change Frequency

Introduction — An 83 Percent Cut, and 39.7 Cents
What the New Experiment Actually Measured — and the Caveats the Author Attached Himself
The Existing Evidence Is Thinner Than You'd Think
Calculating Payback Period — Change Frequency Is the Multiplying Term
Where the Interest Metaphor Works, and Where It Breaks Down
Persuading Your Manager — With History, Not Aesthetics
Closing — Code Only Charges Interest When It's Read
References

Introduction — An 83 Percent Cut, and 39.7 Cents

On July 30, 2026, Thoughtworks' Giles Edwards-Alexander published The Economic Benefit of Refactoring on martinfowler.com. The experimental design is clean. He picked one problematic 17,155-line Rust data-access module out of a 150,000-line application built entirely by an AI agent, fixed a single prompt requesting a "representative change," then refactored the module across 15 steps — re-running the same prompt at every step — and measured token consumption. Between steps, he discarded the changes so the agent could not learn from them.

The result is eminently quotable. Input tokens dropped from 159,564 to 27,360 — a reduction of 132,204 tokens, or 83 percent. And the rebuttal that immediately surfaced in the Hacker News discussion is just as quotable — at then-current prices, that savings amounts to 39.7 cents. Set against the eight hours a senior developer at 100 dollars an hour spent directing and supervising the refactor, it would take thousands of changes to earn that back.

Both numbers are correct. And that contrast shows exactly why the refactoring debate has failed to reach a conclusion for twenty years — the cost is visible and the benefit is scattered across the future, so the answer flips depending on how you count the benefit. This post is about that counting method: what the new experiment actually measured, how thin the existing evidence really is, the formula for calculating payback period, where the interest metaphor breaks down, and the commands for pulling numbers to use in front of your manager instead of aesthetics.

What the New Experiment Actually Measured — and the Caveats the Author Attached Himself

Let's first be precise about what this experiment measured. Not human comprehension time, but the agent's input tokens — a proxy for "how much code has to be read to make this change." That definition is what makes the experiment interesting. A human gets faster the second time they read the same file, but an agent reads from scratch every single time, so the cost of structure gets honestly billed on every single change. A signal that's hard to isolate in human studies, because of learning effects, shows up cleanly here.

There are a few more things worth reading out of the numbers.

The biggest single reduction came from the step that split files along domain lines. One step alone, the last of the fifteen, dropped input tokens from 107,205 to 27,360. The author attaches an important caveat here — splitting files arbitrarily into smaller pieces doesn't help much, because the agent then has to hunt across more files. The earlier steps (local cleanups like extracting duplication) were preparatory work that made that final split possible. In other words, the payoff is concentrated in the last step, but that step can't be done on its own.

Output tokens barely moved (roughly 1,700 to 2,460). That means the refactoring cut the "cost of reading" but had no effect on the "cost of writing," and the author himself draws no conclusion on this point.

And total code volume didn't shrink. The largest file went from 17,155 lines down to 9,269, but the data-access layer as a whole stayed at roughly the same 16,500 lines spread across 19 files. One Hacker News comment pointed this out specifically, noting that it cuts against the common assumption that refactoring usually reduces code volume.

The caveats the author attaches himself are fairly long, and worth carrying over in full.

The token-counting method is an approximation. He divided character count by four, and notes there was no reliable way to count tokens precisely in real time. Hacker News pushed back with "why not just use a proper tokenizer library."
It is a single experiment on one greenfield app, one developer, one module, one type of change.
Tokens spent planning and executing the refactoring itself weren't counted. He estimates an upper bound of 5 million tokens, but it isn't a verified figure.
And the most interesting caveat — the agent wasn't good at refactoring. It couldn't judge on its own which refactoring was appropriate, so a human had to give explicit instructions at every step, and the actual transformation work was done by a Python script using grep and sed, which frequently got confused about indentation. The single most valuable refactoring was missed the first time around and had to be applied again later.

In short, this piece is not a proof that "refactoring pays off" — it's a first attempt at proposing one way to measure the cost of structure in agent-based development. The author himself writes that it's "just one experiment, but an interesting first step." That's the right way to read it.

The Existing Evidence Is Thinner Than You'd Think

The claim that refactoring pays off comes wrapped in forty years of intuition and a surprisingly thin body of quantitative evidence. When you actually list what's worth citing, the list is short.

The most frequently cited work is Adam Tornhill and Markus Borg's Code Red: The Business Impact of Code Quality (2022). Combining source analysis, version-control history, and Jira issues across 39 commercial production codebases and 30,737 files, it reports three findings — low-quality code carries 15 times more defects, issue resolution takes 124 percent longer on average, and the worst-case cycle time stretches out to 9 times longer.

The study is serious, but its limits are clear too. Code quality was measured using metrics from a commercial tool the authors themselves built, and the relationship is correlational, not causal. It isn't ruled out that an inherently difficult problem domain produced both the messy code and the long resolution times at once. And critically, this study measured the cost of low-quality code, not the return on refactoring. The two are different — refactoring carries its own cost and its own regression risk.

Martin Fowler's own position is worth citing too. In Is High Quality Software Worth the Cost? (2019, updated in 2024 to cite the Tornhill and Borg study), he presents the design stamina hypothesis while at the same time acknowledging that there's no way to measure the functionality a software team has delivered, which makes it impossible to attach a solid number to the outcome. The graph he offers is illustrative, not data. For the author of the most influential claim in this field to be this explicit about the nature of his own evidence is, if anything, a trustworthy stance. One practical observation he adds is useful — skilled developers feel bad code visibly slowing them down within a few weeks, so the window in which you're actually trading quality against cost isn't very long to begin with.

Evidence pointing the other way exists too. A differentiated replication study (ESEC/FSE 2020) on the relationship between refactoring and bugs reports that certain types of refactoring — especially ones that touch inheritance hierarchies — are frequently associated with introducing new defects. Is refactoring always a good egg? (MSR 2022) finds that refactoring mostly either removes code smells or has no effect at all, which runs counter to prior work. More recent studies point to a methodological problem — refactoring, bug fixes, and bug-introducing changes are tangled together within the same commits, which makes isolating the effect difficult in the first place.

The honest summary is this: evidence exists that low quality is expensive; general evidence that refactoring recovers that cost is thin; and certain kinds of refactoring increase risk. So the discussion shouldn't start from the premise that "refactoring is good" — it needs a case-by-case calculation.

Calculating Payback Period — Change Frequency Is the Multiplying Term

When the evidence is thin, the approach available to you is a calculation for a specific module, not a general proposition. You only need five terms.

Annual savings come in two parts. One is savings on change cost, the other is savings on defect cost.

Annual savings = (C × T × S × H)  +  (D × F × K)
  C : number of changes touching this module per year   (pulled from git history)
  T : average time spent per change                      (pulled from the issue tracker)
  S : share of that time wasted due to structure          (estimate; 0.2-0.5 is conservative)
  H : hourly labor cost
  D : annual defects originating in this module
  F : total cost per defect (response + rework + customer impact)
  K : share of defects expected to be prevented by refactoring (estimate; 0.3 is optimistic)

Refactor cost = (R × H)  +  (P × F_reg)
  R : refactoring work hours (including added test coverage)
  P : probability of a regression × expected count
  F_reg : cost of a single regression

Payback period (months) = Refactor cost / (Annual savings / 12)

What matters structurally here is the fact that C is a multiplying term. If C is zero, savings are zero no matter how large T, S, and H are. This is the mathematical expression of "code that never changes isn't worth cleaning up." A messy module that was written five years ago, that nobody has touched since, and that still runs fine, will never pay back. Being aesthetically annoying and being economically costly are different problems, and this formula is what separates them.

Running the calculation across three modules makes the difference stark. This example sets hourly labor cost at ₩100,000, cost per defect at ₩2,000,000, S at 0.3, and K at 0.3. The refactor cost here only includes the labor term and leaves out the regression-risk term, so the real payback period would run longer than the figures below.

Module	Annual changes C	Time per change T	Annual defects D	Annual savings	Refactor cost	Payback period
Order processing (hotspot)	84	6 hrs	11	~₩21,720,000	160 hrs = ₩16,000,000	~9 months
Report generator	12	8 hrs	2	~₩4,080,000	120 hrs = ₩12,000,000	~35 months
Legacy settlement batch	1	20 hrs	0	~₩600,000	200 hrs = ₩20,000,000	~33 years

The third row is the point of this table. The legacy settlement batch is probably the single most painful file to read in the whole codebase, and probably the debt the team complains about most often. And it's code you should not touch. Its payback period outlasts the service itself.

Three cautions when producing these numbers. Keep S honestly low. Converting Code Red's 124 percent directly into an S value gives you 0.55, but that's an upper bound drawn from correlational data, not the value for your module. Don't set P to zero. This is exactly the term the opposing-direction research from earlier is talking about — attach a risk premium especially for refactoring that touches inheritance hierarchies. And if the calculation comes back "don't do it," accept that conclusion. The value of this calculation isn't in winning approval — it's in deciding where not to spend your effort.

Where the Interest Metaphor Works, and Where It Breaks Down

The term technical debt was coined by Ward Cunningham in 1992, and it has since become the default vocabulary for this discussion. The metaphor genuinely does two things well — it translates the structure of "pay more later for going fast now" into financial language, and that's what makes the argument land with people outside engineering. That alone makes it useful enough.

But it breaks down at three points.

First, interest accrues on contact, not on time. Financial debt piles up interest just sitting there, but technical debt is only billed when you touch that code. Cunningham's own phrasing was that "all the time that you spend on not-quite-right code is interest on that debt" — the condition is spending time, not time passing. This is exactly why C is a multiplying term in the previous section, and this single difference is what makes the financial intuition that "debt is better paid off quickly" wrong when applied to code.

Second, you can't calculate the principal. There's no interest rate, no repayment schedule, no maturity date. Tools exist that convert total debt into a single monetary figure on a dashboard, but few people can explain what the denominator of that number actually is. The moment you mistake the metaphor for a model, that figure becomes something to manage — and a figure that gets managed starts getting optimized.

Third, it isn't linear. Debt sitting at a highly coupled point costs far more than debt that's purely local. The same 100 lines of messy code cost something different depending on whether 20 modules depend on it. The financial metaphor has no concept of position.

And there's one misunderstanding Cunningham himself has corrected repeatedly — the debt he was talking about is not deliberately sloppy code written with the intent of doing it properly later. He meant the cycle of shipping fast to gain understanding of the domain, then refactoring to reflect that understanding back into the code. In the original metaphor, debt wasn't "bad code" — it was learning not yet reflected. This definition is more useful in practice because it tells you exactly what needs cleaning up: the places where what we currently know about the domain and what the code assumes have drifted apart.

Persuading Your Manager — With History, Not Aesthetics

"This code is messy" isn't persuasive. The person listening has no way to verify it, and it gets filed away as a matter of taste. Instead, pull the values for the formula from the previous section out of real data. Every one of them is an auditable number.

# 1) Change frequency C: top 20 files by commit count over the last 12 months
git log --since="12 months ago" --name-only --pretty=format: \
  | sed '/^$/d' | sort | uniq -c | sort -rn | head -20

# 2) Defect-related changes D: same aggregation, filtered to bug-fix commits only
#    (use --grep if you have a commit convention, or an issue-key pattern if not)
git log --since="12 months ago" --name-only --pretty=format: \
  --grep='^fix' --grep='hotfix' --grep='BUG-' \
  | sed '/^$/d' | sort | uniq -c | sort -rn | head -20

# 3) Hotspots: keep only the large files among those with high change frequency
git log --since="12 months ago" --name-only --pretty=format: \
  | sed '/^$/d' | sort | uniq -c | sort -rn \
  | awk '{ n=$1; f=$2; cmd="wc -l < " f; cmd | getline loc; close(cmd);
           if (loc > 500 && n > 20) printf "%5d changes  %6d loc  %s\n", n, loc, f }'

Walk in with this output as-is and the conversation changes. "The order-processing module was modified 84 times over the last 12 months, 23 of those were bug fixes, and 31 percent of P2 incidents pass through this file" is a verifiable statement. Attach a single payback period to that, and it stops being a request for approval and becomes an investment proposal.

What to attach next is a scoped experiment. Don't ask for a full cleanup — propose allocating a fixed amount of time to one hotspot and measuring lead time before and after. If it fails, it fails within that scope; if it succeeds, you have grounds for the next round. This is also where the new experiment discussed earlier offers something practical — running the same change request once before and once after refactoring is the cheapest way to measure it. If you're using an agent, token count becomes the proxy metric; if you're relying on humans alone, time to first commit does.

Finally, don't try to win the argument. Someone who calculates three modules, concludes "don't" on two of them, and proposes only one, gets approval far faster than someone who insists everything needs cleaning up.

Closing — Code Only Charges Interest When It's Read

To sum up:

The new experiment cut input tokens for the same change by 83 percent across 15 refactoring steps, but in monetary terms that's 39.7 cents per change, and the author himself is careful to call it a single greenfield experiment. Read it as a proposed measurement method, not a conclusion.
The existing evidence is thin. Correlational evidence that low quality is expensive does exist (15 times the defects, 124 percent longer resolution), but there's no general evidence that refactoring recovers that cost, and some types of refactoring are associated with introducing defects.
So calculate per module instead of arguing in general terms. Because change frequency is a multiplying term, code that never changes won't pay back no matter how messy it is.
The interest metaphor lands with managers, but it isn't a model. Interest accrues on contact rather than time, the principal can't be calculated, and it's nonlinear with respect to coupling.
Persuade with git history, not aesthetics. A proposal that rules out two of three candidates itself is the one that gets approved fastest.

Reduced to one line: the value of refactoring depends not on the state of the code, but on how many more times that code is going to be read.