- Introduction — We Made the Cache Bigger and CI Time Didn't Move
- Cache Hits Are a Key Problem, Not a Size Problem
- Three Levers for Narrowing Hash Inputs
- Affected-Target Detection — The Fastest Run Is the One You Skip
- The Trust Boundary That Appears When You Share a Remote Cache
- Why Cache Poisoning and Hermeticity Are the Same Problem
- Measuring Hit Rate Honestly
- Conclusion — A Cache Is a Contract, Not a Storage Device
- References
Introduction — We Made the Cache Bigger and CI Time Didn't Move
When a ticket comes in that monorepo CI is slow, the first proposal is almost always the same: "the cache keeps getting evicted, so let's make it bigger." So teams raise the storage quota, grow the runner disk, and extend artifact retention. And the following week, CI still takes 12 minutes.
The cause is usually not size. The cache is being written just fine but keeps missing on lookup. Maybe a slightly different environment variable slips into the hash on every build, a timestamp gets baked into the artifact, the toolchain version shifts whenever the runner image changes, or the cache key includes the commit SHA and so can never be reused in the first place. Growing storage in this state just lets the garbage stick around longer.
Using the Turborepo, Nx, Bazel, and GitHub Actions caches as material, this post organizes monorepo CI caching around one axis: an accurate cache key paired with hermetic inputs. General reasons builds are slow are covered in Why Is My Build So Slow?, and the principles behind content-addressable storage are in Content-Addressable Storage. Here we look only at what's specific to monorepo CI.
Cache Hits Are a Key Problem, Not a Size Problem
Every build cache rests on the same contract: the same inputs produce the same outputs. The cache key is a hash of those "inputs," and the hit rate is proportional to how accurately the key captures them. If the key is wider than reality — it includes things it doesn't need to — you get no hits. If the key is narrower than reality — it omits an input that actually matters — you reuse the wrong result.
Each tool honors this contract differently.
| Tool | Cache unit | What goes into the hash | Hermeticity guarantee | Remote cache |
|---|---|---|---|---|
| Turborepo | Per-package task | Package source, declared env vars, hashes of internal dependency packages, task definition | Convention-based. Undeclared environment variables are blocked in strict mode | Vercel-hosted or a self-hosted OpenAPI-compatible server |
| Nx | Per-project task | File set defined via namedInputs, env vars, hashes of dependency projects | Convention-based. The inputs definition is the contract | Nx Cloud or a self-hosted cache |
| Bazel | Action (a single command) | Declared input files, command line, environment, entire toolchain | Enforced by sandboxing. Undeclared files are simply invisible | gRPC remote cache protocol |
| GitHub Actions cache | An arbitrary directory tarball | A key string the user writes by hand | None. Entirely the author's responsibility | Repository-scoped, branch-scoped |
What you should take from this table isn't a performance ranking — it's where the responsibility sits. Bazel physically blocks undeclared inputs via its sandbox, so the tool itself enforces hermeticity. Turborepo and Nx set up a convention — "what you declare is everything" — and expect the user to honor it. GitHub Actions cache has a human hand-write the key string — the most flexible option, and the one that goes wrong most often.
So the first diagnostic step in practice is always the same: ask the tool why it missed. Until you distinguish whether the cache was evicted, the key changed, or the task was never cacheable to begin with, any fix you apply is just a guess.
Three Levers for Narrowing Hash Inputs
In practice, a wrong key almost always traces back to one of three causes.
First, the file inputs are too broad. The default is usually "every file in the package." Edit the README and tests rerun anyway, invalidating the entire downstream subgraph that depends on that package. You need to declare only the actual inputs, per task.
// turbo.json — narrow the inputs per task
{
"tasks": {
"build": {
"dependsOn": ["^build"],
"inputs": ["src/**", "tsconfig.json", "package.json"],
"outputs": ["dist/**"]
},
"test": {
"inputs": ["src/**", "tests/**", "vitest.config.ts"],
"outputs": []
}
}
}
Nx does the same job with namedInputs. You define them once and reuse them across projects, which makes this easy to manage in a large repository.
// nx.json
{
"namedInputs": {
"default": ["{projectRoot}/**/*", "sharedGlobals"],
"production": [
"default",
"!{projectRoot}/**/*.spec.ts",
"!{projectRoot}/**/*.md"
],
"sharedGlobals": ["{workspaceRoot}/tsconfig.base.json"]
},
"targetDefaults": {
"build": { "inputs": ["production", "^production"], "cache": true }
}
}
Second, environment variables leak in. This goes wrong in both directions. If a value the CI vendor injects, like GITHUB_RUN_ID, ends up in the hash, you never get a hit again. Conversely, if something that actually changes the build output, like NODE_ENV or an API endpoint, is missing from the hash, a development build gets reused as the production cache.
Turborepo's strict environment mode is the standard fix for this. It's strict by default, and any variable not declared via env or globalEnv is simply invisible at task runtime. If a build suddenly fails, that's evidence the variable was being used secretly all along.
{
"globalEnv": ["NODE_ENV"],
"globalPassThroughEnv": ["CI", "GITHUB_ACTIONS"],
"tasks": {
"build": {
"env": ["NEXT_PUBLIC_API_URL", "SENTRY_*"],
"outputs": [".next/**", "!.next/cache/**"]
}
}
}
passThroughEnv passes the value into the task but keeps it out of the hash. It's meant only for variables that don't change the result, like a logging flag. Put a variable that does change the result in there, and the cache starts lying the moment you do.
Third, nondeterministic values get baked into the artifact. Build timestamps, absolute paths, build numbers, random chunk IDs. This isn't a key problem but an output problem — yet it ends up changing the input to whatever task consumes that output, which breaks the cache chain further up. The standard fix from the reproducible-builds world is to pin SOURCE_DATE_EPOCH to the commit timestamp and make paths relative.
# pin the commit timestamp as the build timestamp
export SOURCE_DATE_EPOCH="$(git log -1 --pretty=%ct)"
# build twice from the same inputs and check the artifacts are byte-identical
pnpm build && cp -r dist /tmp/build-a
rm -rf dist && pnpm build && cp -r dist /tmp/build-b
diff -r /tmp/build-a /tmp/build-b && echo "reproducible"
These three checks are day-one work for any cache project. If the artifact isn't reproducible, every cache discussion built on top of it is moot.
Affected-Target Detection — The Fastest Run Is the One You Skip
Even a cache hit isn't free. A remote cache lookup is a network round trip, and downloading and unpacking the artifact both take time. In a repository with thousands of tasks, "not even looking, because it was never in scope" beats "look everything up and hit on everything" by a wide margin.
# Turborepo — only packages changed since origin/main and their dependents
turbo run build test --filter="...[origin/main]"
# Nx — determine affected via the project graph and git history
nx affected -t build test --base=origin/main --head=HEAD
# see what was selected and why, visually
nx show projects --affected --base=origin/main
nx graph --affected --base=origin/main
# Bazel — query reverse dependencies from the changed files
bazel query "rdeps(//..., set($(git diff --name-only origin/main)))" --output=label
A common mistake here is picking the wrong base commit. Using --base=origin/main in a PR workflow pulls in unrelated changes as affected whenever main has moved ahead. The correct reference point is the merge base.
BASE="$(git merge-base origin/main HEAD)"
nx affected -t build --base="$BASE" --head=HEAD
And with a shallow clone, this computation isn't even possible. GitHub Actions' checkout action defaults to depth 1, so any workflow using affected needs to fetch enough history.
- uses: actions/checkout@v4
with:
fetch-depth: 0 # or at least enough to include the merge base
It's also worth knowing where affected detection loses your trust. Implicit dependencies the project graph doesn't capture — module paths assembled as strings at runtime, generated code, shared config files, Docker base images — get dropped from affected even when they've changed. It's safer to declare these explicitly in sharedGlobals or globalDependencies so they're treated as a global invalidation target. A slightly lower hit rate beats letting a wrong result slip through.
The Trust Boundary That Appears When You Share a Remote Cache
The payoff from a remote cache scales with how widely it's shared. Share it only among CI runners, and you benefit only on reruns of the same commit. Extend it to developer machines, and whoever pulls main in the morning can start working immediately, with no build.
But the moment you widen the sharing scope, one problem appears: whoever can write to the cache determines everyone else's build output. Upload one poisoned artifact from a laptop, and every build that downloads it uses that result. So the practical rules are simple.
- Writes come only from trusted CI. Developer machines and fork PRs get read-only access.
- Writers run only in a reproducible environment. A pinned container image, a pinned toolchain version.
- Sign the artifacts. Treat anything that fails verification on download as a miss.
Turborepo supports signing. Turn it on in turbo.json, supply the key as an environment variable, and it attaches an HMAC-SHA256 signature; any artifact that fails verification is discarded and treated as a cache miss.
{ "remoteCache": { "signature": true } }
# CI: a trusted job with write access
export TURBO_API="https://cache.example.internal"
export TURBO_TEAM="platform"
export TURBO_TOKEN="***"
export TURBO_REMOTE_CACHE_SIGNATURE_KEY="***"
turbo run build
# developer machines / fork PRs: distribute read-only tokens only
export TURBO_TOKEN="read-only-***"
Self-hosting isn't hard either. Turborepo publishes an OpenAPI spec for the remote cache API, so you just put a thin server in front of S3-compatible storage. Bazel uses the gRPC remote cache protocol and has flags that separate write access per job.
# read-only consumers (developers, fork PRs)
bazel build //... \
--remote_cache=grpcs://cache.example.internal \
--noremote_upload_local_results
# writers (trusted CI)
bazel build //... \
--remote_cache=grpcs://cache.example.internal \
--remote_upload_local_results
Why Cache Poisoning and Hermeticity Are the Same Problem
2026 was a year that confirmed, repeatedly, that CI caches are a supply-chain attack path. In the TanStack incident on May 11, a poisoned cache was reportedly written into main-branch scope, and a chain of malicious package versions shipped from it. In its June 26 changelog, GitHub changed the default so that untrusted triggers now get issued a read-only cache token.
Here is exactly what changed. For events that can be triggered without repository write access — pull_request_target, issue_comment, a workflow_run derived from a fork PR — where the execution context and cache scope come from the default branch SHA, the cache token becomes read-only. Trusted triggers like push, schedule, and workflow_dispatch, along with pull_request and release using a non-default-branch scope, keep read-write access. So if a workflow that used to write to the cache falls under the conditions above, you need to split it: move the write to a push-triggered workflow and leave the rest as restore-only.
What matters here is that this is an access-control measure, not a root fix. The root problem is that a cache entry doesn't prove what it was built from. In a hermetic build, the cache key is a hash of the entire input, so it's hard to slip a poisoned artifact under a legitimate key. But if the key is a string a person wrote — as with GitHub Actions cache — matching the key is all it takes to put anything behind it.
Here are the rules worth following in practice.
- Eliminate cache sharing across trust boundaries. Scope keys so fork PRs and the default branch never share the same key space.
- Never cache files built from untrusted input. GitHub provides CodeQL queries that catch this pattern — Cache poisoning via code injection, Caching of untrusted files.
- Keep cache-entry TTLs short. GitHub's default is a 7-day sliding window from last access, which means poisoning can survive a full week.
- Separate the dependency cache from the build-artifact cache. The former is deterministic, keyed by a lockfile hash; the latter is an execution result, so it needs a different level of trust.
Trust boundaries across the CI pipeline as a whole are covered more broadly in CI Agents and the Prompt-Injection Supply Chain.
Measuring Hit Rate Honestly
There are three exaggerations that show up most often in cache-work status reports.
Comparing the time of two runs on the same machine. The second run hits the local cache. You learn nothing about whether the remote cache actually works. Measurement has to happen on a fresh runner, from a fresh clone.
Lumping affected-skips together with cache hits. They're different optimizations with different failure modes. When affected under-selects, a wrong build slips through; when the cache under-hits, things just get slower. Combine the two counts and you can't tell which one degraded.
Looking only at hit rate by task count. If 900 trivial lint tasks hit and 10 heavy build tasks miss, your hit rate reads 98 percent while CI time doesn't budge. Hit rate needs to be tracked by time saved as well.
The tools themselves provide the measurement.
# Turborepo — dump a run summary as JSON and aggregate cache status
turbo run build --summarize
jq '[.tasks[] | {task: .taskId, status: .cache.status, ms: .execution.duration}]' \
.turbo/runs/*.json
# compare two run summaries to find which input differed
turbo run build --dry=json > /tmp/run-a.json
# (rerun in a different environment)
diff <(jq -S . /tmp/run-a.json) <(jq -S . /tmp/run-b.json)
# Bazel — diff two runs' action logs to find non-hermetic elements
bazel build //... --execution_log_compact_file=/tmp/exec-ci.log
bazel build //... --execution_log_compact_file=/tmp/exec-local.log
# same action key but no hit -> a config problem; different key -> a different input
The diagnostic principle from Bazel's docs applies regardless of tool — a different action key means a different input; the same action key with no hit means a cache-configuration problem. Splitting into these two branches first cuts your investigation surface in half.
Let me also be honest about target numbers. In a repository where the remote cache works properly, a small PR against main typically lands in the 90-percent-plus hit-rate range. If repeated runs fall below 80 percent, it's reasonable to suspect the key or the storage layer. That said, this figure varies a lot with repo structure and task distribution, so you should judge it by a two-week trend in your own repository, not someone else's benchmark.
Conclusion — A Cache Is a Contract, Not a Storage Device
Approach cache tuning as a storage-capacity problem and you will fail almost every time. A cache is the contract that "the same inputs produce the same outputs," and hit rate is a measure of how precisely you've described that contract.
- Check reproducibility first. If two builds from the same input produce different bytes, the cache conversation comes after that, not before.
- Narrow
inputsandenvper task, and use strict environment mode to expose leaking variables. If a build breaks, you've just found an input that was secretly in use. - Affected detection is an optimization that comes before caching. Declare implicit dependencies explicitly as global invalidation targets instead of hoping the graph catches them.
- Restrict remote-cache writes to trusted CI, turn on signing, and keep untrusted triggers read-only. Every 2026 incident traced back to a missing version of this boundary.
- Measure from a fresh runner and a fresh clone, keep affected and cache accounting separate, and track by time saved.
No CI gets faster because its cache is bigger. Only a CI that knows its inputs precisely gets faster.
References
- Turborepo — Caching concepts
- Turborepo — Environment variables and strict mode
- Turborepo — Remote Caching, self-hosting, and artifact signing
- Nx — Run Only Tasks Affected by a PR
- Nx — Distribute Task Execution (Nx Agents)
- Bazel — Remote Caching
- Bazel — Debugging Remote Cache Hits (comparing execution logs)
- GitHub Docs — Dependency caching reference
- GitHub Changelog — Read-only Actions cache for untrusted triggers (2026-06-26)
- GitHub Changelog — Actions cache size can now exceed 10GB per repository (2025-11-20)
- CodeQL — Cache Poisoning via low-privileged code injection
- CodeQL — Cache Poisoning via caching of untrusted files
- Why Is My Build So Slow? (related post)
- Content-Addressable Storage (related post)
- CI Agents and the Prompt-Injection Supply Chain (related post)
현재 단락 (1/143)
When a ticket comes in that monorepo CI is slow, the first proposal is almost always the same: "the ...