Skip to content
←

필사 모드: Why Is My Build So Slow? — Compilation, Linking, Caching, and Monorepos

English
0%
정확도 0%
💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction — Time Spent Staring at a Progress Bar

The time a developer spends waiting on builds is larger than it seems. You change a line of code and wait 30 seconds, a minute, sometimes several minutes to see the result. Stack that short wait dozens of times a day and focus breaks and flow shatters. "The build is slow" is a common complaint, but surprisingly few developers know exactly where that slowness comes from.

There is always a structural reason a build is slow. What gets compiled, what gets linked, what can be cached, and what can run in parallel all decide the speed. This post decomposes each stage of the build to pinpoint where time leaks, then covers incremental builds and caches, monorepo strategies, CI caching, and how to actually measure a build. The goal is to turn a vague feeling of "it's slow" into the diagnosis "this is the bottleneck."

A Build Is Not a Single Action

First we must recognize that the single word "build" is actually a multi-stage pipeline. Take a compiled language like C/C++; it goes through roughly these stages.

source code (.c, .cpp)
    |
    | 1. preprocessing (include headers, expand macros)
    v
preprocessed source
    |
    | 2. compilation (source -> object file)
    v
object files (.o)
    |
    | 3. linking (objects + libraries -> executable)
    v
executable

Each stage has a different character and a different reason for being slow. So when diagnosing "the build is slow," the first question is always "which stage is slow." A slow compile and a slow link are entirely different problems with different fixes.

Compilation vs Linking — Different Bottlenecks

Compilation is the stage that translates each source file independently into a machine-code object file. Its key property is that it is independent per file. Compiling a.cpp has nothing to do with compiling b.cpp, so multiple files can be compiled in parallel across multiple cores. The typical causes of slow compilation are:

  • Huge headers: in C++, headers are re-preprocessed for every source file. When hundreds of files include a heavy header, the same content is parsed hundreds of times.
  • Templates and metaprogramming: C++ templates generate code on every instantiation, which is a heavy load on the compiler.
  • Optimization level: high optimization like -O2 or -O3 demands far more analysis and is slower.

Linking is the stage that stitches all compiled object files and libraries into the final executable. What makes it decisively different from compilation is that linking is inherently a global operation. It cannot begin until every object file is ready, and the process of resolving and laying out symbols is largely single-threaded. So linking is hard to parallelize and often becomes the final bottleneck in a large project.

This difference matters in practice because of this: adding cores speeds up compilation but does little for linking. So large projects adopt a faster linker (like the parallel linkers lld or mold) or strategies to reduce linking itself (incremental linking, dynamic linking). If "no matter how many cores I add, the build doesn't get proportionally faster," linking is often the culprit.

Cold Build vs Warm Build

Something you must always distinguish when talking about build speed is cold versus warm.

  • Cold build: building everything from scratch, from nothing. The first build right after you clone a repository or after wiping all build artifacts.
  • Warm build: with the previous build's artifacts still present, rebuilding only the parts that changed.

A cold build is slow by nature, because it does everything from scratch. What really matters is the warm build. During development you change code a little and rebuild dozens of times a day, and whether that repeated build is fast determines productivity. The key technologies that make warm builds fast are exactly the incremental builds and caches we look at next.

One common mistake is evaluating a build system by its cold build time alone. What a developer actually experiences all day is the warm build, so "how long does it take to rebuild after changing just one file" is a far more important metric.

Incremental Builds — Don't Redo What Didn't Change

The principle of an incremental build is simple. Don't rebuild what didn't change. The build system knows which inputs each artifact depends on (the dependency graph), and if the inputs didn't change, it reuses the previous artifact.

Traditional make decides this by file timestamps. If a source file's modification time is later than its object file's, it considers it "changed" and recompiles.

a.cpp (modified 10:05) --> a.o (created 10:03)
   => a.cpp is newer, so recompile a.o

b.cpp (modified 09:50) --> b.o (created 10:03)
   => b.o is newer, so skip b.o

For an incremental build to work correctly, the dependency graph must be accurate. A common trap here is header dependencies. If a.cpp includes config.h and config.h changes, a.cpp must be recompiled too. If the build system doesn't know this relationship, you get the bug where "I edited the header but nothing changed." So the compiler also emits dependency information recording which headers each object file depends on, and the build system reads it to complete the graph.

The timestamp approach has its own limits. Even opening a file and just saving it (with identical content) changes the timestamp and triggers an unnecessary rebuild. So modern build systems move toward deciding changes by content hash instead of timestamps. If the content is actually the same, it doesn't rebuild. This connects to the caches we look at next.

Compiler Caches — ccache and sccache

If an incremental build "skips what didn't change within this project," a compiler cache goes one step further. It stores the result of a compilation and, when the same input comes again, pulls the stored result instead of compiling.

The representative tool is ccache. ccache wraps the compiler, and when a compile request arrives, it computes a hash of the inputs (preprocessed source, compile options, compiler version, and so on). If it has seen the same hash before, it immediately returns the stored object file.

compile request
    |
    v
compute input hash (source content + options + compiler)
    |
    +--> in cache (hit)  --> return stored .o immediately (no compile)
    |
    +--> not in cache (miss) --> actually compile, then store the result

The strength of this approach is that it helps even on a cold build. Even if you wipe all build artifacts, the ccache cache remains, so on rebuild most of it is pulled from the cache. It helps especially when you hop between branches and repeatedly compile the same files.

sccache extends ccache's idea. It can put the cache not only on the local disk but also in remote storage (like cloud storage), so a whole team or CI shares the cache. One person's compilation result is served to another as a cache hit. It is especially widely used in the Rust ecosystem.

For a cache to be correct, the cache key must be accurate. The compiler version, options, source content, and even the included headers must all be reflected in the key, so that you never pull the wrong cache when conditions differ. Cache correctness rests on "what you put in the key."

Task Caches — Turbo, Nx, Bazel

If a compiler cache caches "one compilation," higher-level build tools cache "an entire build task." Here we look at three tools widely used in JavaScript monorepos and large multi-language projects.

Turborepo and Nx are tools mainly for JavaScript/TypeScript monorepos. Their core is hashing each task's inputs (build, test, lint, and so on) and caching the result. If you built package A and none of its inputs (source, dependencies, config) changed, it restores the previous output instead of running again. It even caches the console logs and replays them "as if it just ran."

turbo run build
    |
    v
compute each package's input hash
    |
    +--> same hash (cache hit)  --> restore stored output, skip execution
    |
    +--> changed hash (cache miss) --> actually build, then cache output

Bazel is a large-scale build system built by Google that pushes this idea to the extreme. Bazel's philosophy is hermeticity. It fully declares the inputs of every build action (nothing outside the declared inputs may be touched) and deterministically caches outputs by the hash of those inputs. Because it guarantees that identical inputs produce identical outputs, local machines, CI, and even an entire team can safely share a remote cache. Bazel can even distribute the build actions themselves across remote machines.

The common principle of all three tools is one thing. Hash the inputs, and reuse the stored output for identical inputs. It is the same idea as a compiler cache, lifted from the file level to the task or target level. The larger the scale, the greater the value of this "don't redo it."

The Monorepo Problem — What to Rebuild

A monorepo puts many projects in a single repository. It helps with code sharing and consistency, but it creates a unique problem from the build's point of view. What does one small change force you to rebuild?

In a monorepo where hundreds of packages depend on one another, editing a single shared library affects every package that depends on it. Naively rebuilding everything spends enormous time on a small change. Conversely, rebuilding nothing means the change isn't reflected. The key is to rebuild "only what's affected, exactly."

       shared-utils (changed)
        /      |      \
   app-web  app-api  lib-auth
                        |
                    app-admin

  editing shared-utils:
  affected -> app-web, app-api, lib-auth, app-admin (rebuild)
  unaffected -> the rest of the packages (skip)

This is called "building/testing only the affected set." Nx's affected command and Turborepo's filters automate it. They analyze the dependency graph and pick only the packages reachable from the changed files to build and test. Thanks to this, even in a giant monorepo you only need to verify "what you touched and its blast radius," which dramatically cuts CI time.

The real difficulty of a monorepo is keeping this dependency graph accurate. If the graph is broader than reality you build unnecessarily much; if it is narrower you miss something affected and a broken build passes. So the quality of a monorepo build tool comes down to "how accurately it knows the dependency graph."

The Limits of Parallelism

"Give it more cores and the build gets faster" is largely true but not unlimited. Parallel builds have fundamental limits.

First, the dependency chain. If B depends on A's artifact, you cannot start B before A finishes. When such a chain is long, no matter how many cores you have, you must traverse it in order. This longest dependency chain is the critical path, and it sets the lower bound on build time. No amount of parallelization can go faster than the critical path.

Second, Amdahl's law. If part of a build can't be parallelized (like the linking we saw earlier), that part drags on the overall speed. However fast you make the parallelizable part, the sequential part remains, putting a ceiling on total improvement.

  1 core:  ████████████████████  (100s)
  4 cores: █████ + sequential part  (the parallel part shrinks to 1/4,
                                     but the sequential part stays)
  => the larger the sequential part (linking, etc.), the smaller the gain

Third, resource contention. Even as you add cores, if disk I/O or memory bandwidth is the limit, that becomes the new bottleneck. Builds that write and read many object files in particular tend to make the disk the bottleneck. So blindly raising the parallelism to the core count is not always best; you must measure and find the sweet spot.

CI Caching — Don't Start From Scratch Every Time

Locally, previous build artifacts remain and enable warm builds, but CI is different. CI usually starts in a clean environment, so with no measures it is a cold build every time. So the key to making CI fast is preserving the cache between sessions.

CI caching has roughly two layers.

  • Dependency cache: node_modules, package manager caches, compiled third-party libraries — things that rarely change. Downloading or rebuilding these every time is a huge waste, so cache them keyed by the hash of the lockfile. If dependencies didn't change, restore them wholesale.
  • Build cache: share the ccache/sccache caches and the Turbo/Nx/Bazel task caches across CI sessions. With a remote cache, multiple CI jobs and developers share the same cache.

The most important thing in CI caching is cache key design. If the key is too narrow (like the commit hash), you get a cache miss every time and it's useless; if too broad, you miss changes and use stale results. A common practice is to use the lockfile hash as the primary key, with the branch or OS as a secondary key to reuse appropriately.

One more caveat is that restoring and storing the cache itself takes time. If the cache is too large, the time to download and decompress it can eat the time the cache saved. So you need the balance of judging "what is worth caching" and caching mainly what is expensive to regenerate.

Profiling a Build — Measure, Don't Guess

We've seen many causes of slow builds, but in a real project you must not guess which one is the culprit. You must measure. The key question of build profiling is this. Where is the time going?

Some practical approaches:

  • Per-stage timing: split out how long preprocessing, compilation, and linking each take. If linking dominates, that's a signal that adding cores won't help.
  • Per-file compile time: find which source files take unusually long. Usually a handful of files that use heavy headers or templates dominate the total time.
  • Build graph visualization: many modern build tools show each task's start-end time as a timeline. Here the critical path and the sections that can't be parallelized come into view.
  • Cache hit rate: look at the hit rate of ccache or the task cache. A low hit rate means the cache key is wrong or the cache isn't being shared properly.

The compiler itself offers profiling options too. For example, some compilers can produce a report of the time spent in each compile phase (parsing, template instantiation, optimization). Reading it enables a concrete diagnosis like "this file spends half its time on template instantiation."

The principle of profiling is one thing. Before you make the build fast, find exactly where it's slow. In most builds, time is concentrated in a handful of bottlenecks. Finding those bottlenecks with data and targeting them is far more effective than vaguely optimizing this and that.

Practical Checklist

The items to check, in order, when facing a slow build:

  • Which stage is slow? Split compilation from linking first. If it's linking, consider a faster linker.
  • Is the warm build slow, or the cold build? If warm is slow, the incremental build's dependency graph may be inaccurate.
  • Are you using a compiler cache? Adopting ccache/sccache alone often speeds up repeated builds greatly.
  • In a monorepo, are you building only what's affected? Check that you aren't rebuilding everything.
  • Is the parallelism level appropriate? Measure whether cores, disk, or memory is the bottleneck.
  • Does CI run cold every time? Check that dependency and build caches are preserved across sessions.
  • Is the cache key correct? Verify it isn't too broad or too narrow, using the hit rate.
  • Did you measure? Before guessing, confirm the bottleneck with data through profiling.

Conclusion

A slow build is not fate but a diagnosable problem. A "build" consists of stages with different characters — compilation and linking — and each stage can be largely skipped with incremental builds and caches. Keeping the warm build fast usually matters more than shrinking the cold build time, and in a monorepo the precision of rebuilding "only what's affected" is the key. Parallelism is powerful but bounded by the critical path and sequential sections, and in CI, preserving the cache between sessions is decisive.

Above all, the most important principle is measurement. Build time is usually concentrated in a few bottlenecks, so finding them precisely through profiling and targeting them yields large improvement for little effort. Next time you find yourself staring at a progress bar, turn "why is it slow" from a vague complaint into a concrete question. The answer is usually in the data.

References

현재 단락 (1/113)

The time a developer spends waiting on builds is larger than it seems. You change a line of code and...

작성 글자: 0자원문 글자: 14,107자작성 단락: 0/113