Split View: Debian이 재현 불가능한 패키지의 testing 이행을 막기 시작했다 — 게이트가 실제로 막는 것과 막지 않는 것

Debian Has Started Blocking Unreproducible Packages From Migrating to Testing — What the Gate Actually Blocks, and What It Doesn't

• Introduction — A One-Sentence Policy Change, and What That Sentence Actually Means
• What Exactly Is a Reproducible Build
• Stress Testing vs. Actual Reproduction — the Distinction That Matters Most in This Post
• How reproduce.debian.net Actually Works
• The Gate: What britney Actually Blocks
• The Numbers — as of May 9, 2026
• What Breaks Reproducibility
• Honest Limits
• So What Can Your CI Take From This
• Closing
• References

Introduction — A One-Sentence Policy Change, and What That Sentence Actually Means

At 4:47 AM UTC on May 10, 2026, Paul Gevers posted a message to debian-devel-announce on behalf of the release team. The subject was the plain "bits from the release team," and in the opening he described it this way — we're about halfway through the forky release cycle, and he wanted to announce "a small step in code, but a giant leap in commitment."

The body of that leap was two sentences.

Reproducibility
===============

Aided by the efforts of the Reproducible Builds project [1], we've decided it's
time to say that Debian must ship reproducible packages. Since yesterday, we
have enabled our migration software to block migration of new packages that
can't be reproduced [2] or existing packages (in testing) that regress in
reproducibility.

(Since it says "since yesterday," the day this actually went live was May 9, 2026.)

Tech outlets lifted only the first sentence. Phoronix ran with "Debian Must Now Ship Reproducible Packages," and The Register with "Debian 14 cracks down on unreproducible packages." Read the headlines alone and it sounds like Debian no longer accepts unreproducible packages, period.

But read the second sentence as written, and the scope is far narrower. What's actually blocked is (1) the migration of new packages that don't reproduce, and (2) the migration of packages already in testing whose reproducibility has regressed. That's it — just those two. The hundreds of unreproducible packages already sitting in testing stay exactly where they are.

This post is about that gap — what Debian actually turned on, how the check works technically, where the numbers stand today, and what your own build pipeline could actually steal from this case.

What Exactly Is a Reproducible Build

The Reproducible Builds project's definition is this:

A build is reproducible if given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of all specified artifacts.

The key qualifier here is that all three conditions hold "the same": same source, same build environment, same build instructions. Under those conditions, whoever builds it should get a bit-for-bit identical artifact. Reproducibility doesn't mean "the same result in any environment" — it means "the same result when you reconstruct the specified environment." That distinction keeps tripping people up later in this story.

Why bother? Here's the problem: free software's source is public, but most people install precompiled binaries. And nobody actually knows whether the two correspond — not even whoever built the binary. If the build machine was compromised or the compiler planted a backdoor, no amount of source auditing catches it. Reproducible builds close that hole — if multiple independent parties produce the same bits from the same source, you can verify that the distributed binary actually came from that source.

It's only fair to carry over the caveat the project attaches to itself. Here's how Holger Levsen put it in the May presentation:

Reproducible Builds are an important building block in making supply chains more secure. Nothing more, nothing less. (Un)secure software build reproducibly still remains (un)secure software.

Vulnerable software built reproducibly is still vulnerable software. Reproducibility only guarantees "this binary came from this source" — it guarantees nothing about whether that source is safe.

Stress Testing vs. Actual Reproduction — the Distinction That Matters Most in This Post

Debian runs two systems that look at reproducibility. Confuse the two and you'll read the numbers completely wrong.

tests.reproducible-builds.org is the older of the two, running since 2015. In Paul Gevers's words, this is a stress test — it deliberately builds the same package twice under different conditions. The slides call this approach variance testing, and its purpose is to hunt for "factors that could interfere with a package's reproducibility." It deliberately shakes variables to find weak links.

reproduce.debian.net is the newer one, started in 2024, and it's the one that just got wired into the gate. Its purpose is the opposite. In the slide's own words:

The goal of reproduce.debian.net is to replicate the same build process that is used by Debian during package publication -- not to seek out additional sources of variance.

It does not go looking for additional sources of variance. It tries to replicate the exact build process Debian used when it published the package.

Paul Gevers restated this distinction in a follow-up mail on May 30, and since it's the crux of this post, here it is verbatim:

there is a very clear difference in the way reproducibility is tested on https://tests.reproducible-builds.org/ vs https://reproduce.debian.net/. While the former is a stress test, the latter tries, with the info in the .buildinfo file, to reproduce what we ship. It's the latter case what some people call "easy", but it's also what matters to our users: can the binaries they use be reproduced.

He concedes up front that some people call the latter "easy." Then he pushes right back — easy or not, that's the one that matters to users: can the binaries they actually use be reproduced.

That's an engineering judgment worth sitting with. If they'd wired the gate to the stress test, the pass rate would have been far lower and the policy would have been pushed off for years. Instead they wired it to the "easy" side, which let them turn it on now. It means they chose the bar they could actually enforce today instead of waiting for a perfect one — and that's exactly the point raised in the LWN comments: the scope of this requirement is limited to "building within an instance of Debian's build environment," and reproducibility is a spectrum, not a yes/no binary.

How reproduce.debian.net Actually Works

The architecture itself is honest and simple. It's a rebuilderd instance.

rebuilderd is a tool kpcyrd wrote in Rust; development started at the 2019 Marrakech summit, and it supports rebuilding for Arch, Fedora, Debian, and Tails. There are roughly 5 instances for Arch, 1 for Fedora, and 5–6 for Debian. In other words, this isn't something built fresh for Debian — it's something that was already running for other distros, adopted as-is.

Here's what reproduce.debian.net does:

Attempts to bit-for-bit identically rebuild each Debian binary package found in the distribution archive, using the .buildinfo file produced when the buildd originally built the package. For each distributed package, rebuilderd calls debrebuild that calls debootsnap, mmdebstrap and finally sbuild to build that package within a user namespace.

The pivot here is the .buildinfo file. When Debian's build daemon (buildd) builds a package, it leaves behind a control file recording that build environment — which packages, at which versions, were installed at the time. The rebuild reads this file, resurrects that environment, builds inside it again, and compares the result to the distributed binary bit-for-bit.

Binary package distributed by ftp.debian.org
        │
        │   .buildinfo left behind by buildd's original build
        │   (= a record of that build environment at the time)
        ▼
  rebuilderd
        └─► debrebuild ─► debootsnap ─► mmdebstrap ─► sbuild
                                                        └─ rebuild inside a user namespace
        │
        ▼
   bit-for-bit comparison against the distributed binary
        │
        ├─► identical  ─► good
        └─► different  ─► bad  (diffoscope checks what differs)

A side note, but a fun one: in the slides, Holger points out that .buildinfo is effectively an SBOM — what everyone talks about today as a Software Bill of Materials is roughly the same thing as the .buildinfo file Debian designed back in 2014.

Coverage, as of the May 9, 2026 presentation, is this: from trixie onward it covers all of main and non-free-firmware, across the suites experimental, unstable, forky, trixie, trixie-security, trixie-updates, trixie-proposed-updates, and trixie-backports. Architectures are everything except s390x and loong64 (trixie also includes armel). At presentation time they were running 8 instances, one per architecture.

loong64 being missing isn't an accident. According to a different section of the same May 10 mail, loong64 had only been added to the archive two weeks earlier, so all sorts of packages had to be rebuilt for it across every architecture, and there was a note asking for patience because the CI queue was backed up. It means that when a new architecture arrives, the reproducibility infrastructure has to catch up to it too.

The Gate: What britney Actually Blocks

In Debian, once a package is uploaded to unstable, it "migrates" to testing once it satisfies certain conditions. The software that decides this migration is britney, and the next stable release is made by freezing testing. So blocking migration to testing means the package doesn't make it into the next release.

Here's how the presentation slide puts it:

britney now finally has been configured to not let (most) unreproducible packages migrate to testing!

The word to watch here is most, in parentheses. Not all. Here's the actual rule the slide itself lays out.

Migration is blocked when:

• A new (NEW) package that doesn't reproduce
• A regression — a package that used to reproduce and stopped

And immediately attached caveats:

• It can be unblocked on request
• An allowlist is possible (current example: udeb)
• Results from a specific architecture can be ignored

The release team's position, as summarized on the slide:

• All packages should be reproducible
• But for now, only new unreproducible packages and regressions are blocked
• Unreproducible packages are not yet treated as RC bugs, and are not yet auto-removed

That last line tells you the actual strength of this policy. In Debian, an RC (release-critical) bug is a severity that blocks the release, and if left too long the package automatically drops out of testing. Being unreproducible isn't at that severity yet. So the unreproducible packages already in testing effectively stay grandfathered in.

And the presenter himself nails down forky specifically:

for forky we will be rather liberal with accepting unreproducible packages.

For forky, they'll be fairly liberal about accepting unreproducible packages. For the next release, duke, the slide leaves it open as a question — should they declare a goal of zero unreproducible packages in main? There's a question mark attached, and nothing is decided.

Paul Gevers's follow-up mail on May 30 points the same direction. He added this, saying he'd forgotten to mention it in the original announcement:

I forgot to mention that while we of course want all packages in testing to reproduce, we recognize there will be exceptions (which numbers hopefully will reduce over time). If you think your package needs an exception, please file a bug against the release.debian.org pseudo package (option 4 "other" in reportbug for now) and explain why you're not able to fix the issue.

There will be exceptions, and if you need one, file a bug against the release.debian.org pseudo-package explaining why you can't fix the issue. There's an officially open channel for requesting an exception.

In short, this gate is a ratchet — a device that raises the bar one notch and stops it from sliding back down. "From now on, anything new has to reproduce, and anything that already did must not break." It's not a device for cleaning up existing debt; it's a device for stopping debt from growing further. If that sounds less impressive, think about it the other way — for a project running tens of thousands of packages, raising the reproduction rate with no mechanism to block regressions is a bucket with a hole in the bottom.

Holger's slide states this political nature quite candidly.

100% reproducible is a political decision and nothing technical. We need to change debian-policy!

100% reproducible is a political decision, not a technical one. debian-policy has, since 2017, only said packages "should" build reproducibly, using the word "should," and changing that to "must" is a policy amendment process. So the release team chose to enforce it through the migration gate before the policy itself changes. In the slide's words: "In practice the release team will enforce this before it becomes official Debian policy."

The Numbers — as of May 9, 2026

Everything below is a reproduce.debian.net figure taken from Holger Levsen's MiniDebConf Hamburg 2026 presentation slides (May 9, 2026). It's a snapshot at presentation time, and the numbers may have moved since.

forky (the suite that will become the next stable release):

Other suites, for comparison:

A few things worth reading into this.

The gap between the flagship architecture and the rest. amd64 sits at 97.2% while ppc64el is at 91.1% — a 6.1 percentage-point gap. Reproducibility is a property you have to achieve separately per architecture, and the architectures that get the most attention and manpower climb first. This table shows exactly why the gate carries the caveat that results from a specific architecture can be ignored.

backports is low. trixie-backports sits at a noticeably low 79.5% / 83.5%. That reads as a sign that, being the channel for grafting newer versions onto a stable release, its build environment is more prone to drift.

Stress testing and actual reproduction have converged to nearly the same value. Under the heading "The diff between theory and practice today?", the slide places two numbers side by side — 96.6% on CI (the stress test) versus 96.54% on amd64.reproduce.debian.net (actual reproduction). Earlier I called the latter the "easy" side, but in practice the two values differ by only 0.06 percentage points. Granted, the two systems target different suites and aggregate differently, so this isn't a rigorous apples-to-apples comparison — but it's interesting enough as a counterpoint to the suspicion that "the numbers are inflated because it's the easy test." The slide attaches its own caveat here: "we've only been doing this for 10 months & the freeze..."

You can check it directly on your own system. Installing the debian-repro-status package tells you how many of the packages currently on your system fail to reproduce. The presenter's own system output is on the slide.

$ sudo apt install debian-repro-status
$ debian-repro-status
INFO debian-repro-status > 60/2268 packages are not reproducible.
INFO debian-repro-status > Your system is 97.35% reproducible.

This tool struck me as the most impressive part. It means reproducibility isn't just a dashboard number on a website anymore — it's a property you can check with a single line on your own laptop.

What Breaks Reproducibility

The way the slide lists the reasons is itself a joke.

Common reasons for unreproducibilities:

  timestamps, timestamps, timestamps
  timestamps, timestamps, timestamps
  build paths, build paths
  all the rest

  (and somewhere in there there might be backdoors...)

Timestamps six times, build paths twice, everything else once. That's a summary of what this project has been fighting for the past 12 years. And the last line tells you why they bother with this tedious fight — somewhere in there, there might be a backdoor.

The standard fix for timestamps is SOURCE_DATE_EPOCH. Its specification reads:

SOURCE_DATE_EPOCH specifies the last modification of something, usually the source code, measured in the number seconds since the Unix epoch, ie. January 1st 1970, 00:00:00 UTC.

When a distribution sets this environment variable centrally, build tools stamp this value instead of the current time. But you have to read this alongside the caveat the Reproducible Builds documentation itself attaches — "Setting the system clock is not enough for reliable reproducible builds." Pinning the clock isn't enough. SOURCE_DATE_EPOCH only handles the timestamp problem; it does nothing about variance coming from a nondeterministic build process or unstable dependencies.

For a sense of scale: per CI statistics as of May 8, 2026, 4,347 reproducibility-related bugs have been fixed (most of them pushed upstream), with 262 patches still pending. Today's roughly 97% is the result of steadily pushing a handful of these a day for 12 years. There was no magic.

And the list of what's still not working is humbling. The slide reports that of the 1,337 most popular packages, 22 source packages don't reproduce, and names them — ffmpeg, vlc, bluez, nss, grub2, graphviz, pipewire, qtwebengine, bind9, and others. What stings more is the list of 96 out of 5,500 packages in the build-essential dependency chain that don't reproduce. That list includes rustc, python3.12, python3.13, ghc, gcc-13-cross, qemu, valgrind, and chromium.

The compilers and interpreters themselves don't yet reproduce. That's the most compact explanation you'll get for why this is hard.

That's not to say there's no progress. Per the June 2026 report, a debian-installer upload reproduced successfully for the first time, and the debian:sid container became 100% reproducible. diffoscope shipped five versions — 319, 320, 321, 322, and 323 — in that single month.

Honest Limits

Setting aside the temptation to read this charitably, here's where things stand.

The headline is stronger than the policy. "Debian must ship reproducible packages" is a strong sentence, but the mechanism that's turned on blocks only new packages and regressions. Existing unreproducible packages stay put, aren't RC bugs, and aren't auto-removed. The presenter explicitly said they'll go liberal on forky.

There are several doors out. Unblock on request, an allowlist (currently udeb), ignoring per-architecture results, filing an exception with release.debian.org. Each is reasonable on its own, but taken together, the gate's actual strength rests on the release team's discretion.

The check's scope is narrow. reproduce.debian.net replicates Debian's own build environment exactly and compares against it. Whether it reproduces under a different build path, a different kernel, or a different locale is not a question this gate asks. As the LWN comment noted, "reproducible" isn't a binary but a spectrum, and this gate picks one point on that spectrum. Paul Gevers's counter — that this point is the one that matters most to users — is also valid, and both statements are true at once.

The tooling is still rough. Paul Gevers himself admitted this in his follow-up mail — for packages that fail to reproduce because of an NT_GNU_BUILD_ID difference, the current diffoscope log isn't ideal. The actual difference gets clipped out and only shows up in the dbgsym package, which isn't currently part of the automatic comparison. So there are cases where a maintainer tries to see "why" and the log simply doesn't answer.

Trust is concentrated in one place. This is a problem the presenter raised about himself. The slide reads — "Because do you really want to put all your trust in me???" Right now the structure trusts the judgment of one specific instance, reproduce.debian.net. The original point of reproducible builds was for multiple independent parties to each verify — with a single verifier, that point is only half realized. So the slide asks for more rebuilderd instances to be stood up, and lists distributing trust and comparison via transparency logs as future work.

So What Can Your CI Take From This

There are a few things worth generalizing from this case. But Debian is a distribution that has piled up tens of thousands of packages and 12 years of manpower — your service is not. Don't copy this verbatim.

Takeaway 1 — the ratchet beats the goal. What Debian actually turned on wasn't "make everything reproducible," it was "block new debt and block regressions." This isn't specific to reproducibility. Whether it's test coverage, type coverage, or lint warnings, attempts to clean up existing debt in one shot usually fail, while gates that block regressions usually succeed. Put existing violations on an allowlist, freeze them, and block only new violations and regressions — the same way Debian put udeb on the allowlist.

Takeaway 2 — a file that records the build environment. The idea behind .buildinfo is simple: leave a record, next to the artifact, of what existed at what version when it was built. With this you can resurrect that environment later; without it, reproduction can't even start. It's roughly the same thing people call an SBOM today, and Debian made this in 2014. Committing your lockfile, pinning base images by digest instead of tag, and recording toolchain versions are practices in the same family.

Takeaway 3 — SOURCE_DATE_EPOCH. 12 years of data conclude that timestamps are the most common culprit. If your tooling supports this environment variable, turning it on costs almost nothing.

Takeaway 4 — run reproduction checks as a separate job. What matters is that rebuilderd sits outside the build pipeline, rebuilding the published artifact from the outside, rather than living inside it. Put a reproduction check inline in your main CI and you double your build time, and eventually it gets turned off. A separate pipeline that rebuilds and compares asynchronously after deployment is the realistic move.

What not to do. Frankly, bit-for-bit reproducibility is overkill for most teams.

• If nobody outside needs to verify your deployed artifact — you run an internal service as a container and pin the image digest — reproducibility doesn't add much extra assurance. You already know for certain that "that digest is that digest."
• Reproducibility earns its keep when what you're shipping goes to many untrusted third parties who need to independently verify that the source and binary correspond — distributions, browsers, wallets, messengers, firmware. That's exactly why Debian spent 12 years on it.
• If you're somewhere in between, it's better to target just one core artifact for reproducibility rather than everything. Debian didn't do the whole tree at once either.
• And selling reproducibility as a security measure is dishonest. To quote the earlier line again: vulnerable software built reproducibly is still vulnerable software.

Closing

What happened in Debian on May 9, 2026 boils down to this: britney started reading reproduce.debian.net's verdicts, and started blocking testing migration for new packages that don't reproduce and packages whose reproducibility has regressed. As of the presentation, forky's amd64 reproduction rate is 97.2%, with 512 unreproducible source packages remaining.

It's a narrower move than the headline suggests. The exception channel is open, there's an allowlist, per-architecture results can be ignored, existing unreproducible packages stay in place, and they've explicitly said they'll go liberal on forky. The check's scope is also limited to replicating Debian's own build environment.

Still, this matters because something that was "should" for 12 years has, for the first time, actually started blocking something. Amending the policy document requires a project-wide consensus process, but turning on the gate just took changing britney's configuration. In the original announcement's words, it's a small step in code. The release team took that small step first, choosing, as the slide puts it, to "enforce this before it becomes official Debian policy."

What's worth learning here might be the methodology more than reproducibility itself. Declare the goal as 100%, but turn on just one notch of ratchet. Instead of waiting for a perfect bar, pick the bar you can enforce now, take the criticism that it's the "easy" bar head-on, and answer that it's still the one that matters to users. Whether duke will target zero remains an open question.

References

• bits from the release team — Paul Gevers, debian-devel-announce, May 10, 2026 (original announcement)
• Re: bits from the release team — Paul Gevers, debian-devel, May 30, 2026 (exception process and the difference between the two test systems)
• reproduce.debian.net: Reproducing Debian in the real world — Holger Levsen, MiniDebConf Hamburg 2026, May 9, 2026 (source of all figures)
• Reproducible Builds in May 2026 — monthly report
• Reproducible Builds in June 2026 — monthly report (debian-installer, debian:sid container, diffoscope 319–323)
• Debian to require reproducible builds — LWN, May 11, 2026
• reproduce.debian.net — rebuild verdict dashboard
• rebuilderd — kpcyrd, a rebuild daemon written in Rust
• SOURCE_DATE_EPOCH specification
• Definition of a reproducible build
• Debian Release Team: Debian Must Now Ship Reproducible Packages — Phoronix
• Debian 14 cracks down on unreproducible packages — The Register