필사 모드: Why ripgrep Is So Fast

• Introduction — "Fast" Doesn't Quite Cover It
• The Regex Engine — Throwing Out Backtracking
• SIMD memchr — Many Bytes at Once
• Parallel Directory Traversal — Don't Let Cores Idle
• Respecting .gitignore — Not Reading Is the Fastest Read
• Smart Case — Inferring Human Intent
• Skipping Binary Files — Avoiding Useless Data
• mmap — Mapping a File Straight Into Memory
• Putting the Pieces Together — The Whole Pipeline
• Lessons for Writing Fast Tools
• Conclusion
• References

Introduction — "Fast" Doesn't Quite Cover It

The reaction to using ripgrep (the binary is rg) for the first time is usually the same: "why is this so fast?" You search a repository with hundreds of thousands of files for a single string, and the results come back almost instantly. If you grew up on grep -r, ack, or git grep, the difference feels like it belongs to a different category.

A common misconception is that "it's fast because it's written in Rust." The language only helps at the margins — you can absolutely write slow programs in Rust. ripgrep's speed is not a single trick; it is the cumulative result of dozens of decisions, from the choice of regex engine to the filesystem traversal strategy to byte-level optimizations. This post pulls those decisions apart one by one to see what a fast command-line tool actually does differently.

The Regex Engine — Throwing Out Backtracking

The heart of ripgrep is the regex crate written in Rust. The single most important design choice in that engine is that it does not backtrack.

Most traditional regex engines in the Perl/Python/PCRE lineage are backtracking engines. That approach is powerful — it supports expressive features like backreferences and lookaround. But it comes at a cost: some patterns blow up to exponential time in the length of the input. This is the infamous "catastrophic backtracking." Feed a pattern like (a+)+$ the right input and the CPU will grind for seconds or minutes. This is also the mechanism behind ReDoS (regular-expression denial of service) attacks.

ripgrep's engine takes a different road. It compiles the regex into a finite automaton. In theory, any regular expression without backreferences can be expressed as a finite automaton, and a finite automaton scans the input exactly once, in constant time per character, to decide whether it matches. In other words, it is linear in the length of the input. It never blows up, even in the worst case.

The contrast looks like this:

The key here is the trade-off. ripgrep deliberately gave up some features like backreferences. In exchange it gets a linear-performance guarantee that never explodes on any input. For a search tool, that is the right call — one accidental pattern from a user should never be able to freeze the tool.

Internally the engine does not use just one automaton. It mixes several strategies depending on the situation: short literals go through dedicated literal search, complex patterns use a lazy DFA, and literals extracted from the prefix of a pattern quickly narrow down candidate positions. The real behavior is less "run one automaton against every regex" and more "filter as cheaply as possible first."

SIMD memchr — Many Bytes at Once

Before regex matching, there is a more fundamental bottleneck: finding where in a large block of text the candidates might be. ripgrep solves this with memchr.

memchr is exactly what the name says — "find a specific byte in memory." Implemented naively, it is a loop comparing one byte at a time. But modern CPUs have SIMD (Single Instruction, Multiple Data) instructions — SSE2/AVX2 on x86, NEON on ARM. These instructions compare 16, 32, or even 64 bytes simultaneously.

The core idea is this: whether the pattern is a regex or a literal, there is usually a byte that must appear for a match. If you are searching for function, there is no match anywhere without an f. So ripgrep first uses SIMD memchr to scan at high speed for occurrences of f, then only attempts actual regex matching near those positions. Most of the text is not even a candidate, so it skips the expensive matching logic entirely.

  large text buffer
  ├─ (1) SIMD memchr scans for the required byte at high speed
  │       → most regions are eliminated immediately here
  ├─ (2) only near candidate positions
  │       → run the real regex / literal match
  └─ results

This "filter cheaply first, run the expensive check only where needed" strategy is a philosophy that runs through all of ripgrep. memchr is the first gate. Incidentally, the memchr crate is so well built that it is widely used on its own throughout the Rust ecosystem.

Parallel Directory Traversal — Don't Let Cores Idle

Searching a single file quickly and searching an entire repository quickly are different problems. When you sweep hundreds of thousands of files, how much you parallelize is decisive.

Traditional grep -r walks directories on a single thread by default. ripgrep, by contrast, spins up multiple threads that traverse the directory tree in parallel. Worker threads share a work queue: they push discovered subdirectories onto the queue and search files as they surface. With eight cores, it can search eight files at once.

A subtle but important design point falls out here: output ordering. When you search in parallel, results can come back out of order. By default ripgrep does not guarantee ordering, for performance, but options like --sort path will produce a deterministic order. In other words, it hands the user the choice between parallelism and reproducibility.

Parallelism is not free. There is coordination cost between threads and there are bottlenecks in the filesystem itself (especially on network filesystems or slow disks). So ripgrep auto-tunes the number of workers to the logical core count and lets you control it directly with --threads. The lesson of parallelism is not "more threads is always faster" but "split the work evenly and minimize coordination cost."

Respecting .gitignore — Not Reading Is the Fastest Read

Half of ripgrep's speed comes not from reading fast but from not reading at all. This is ripgrep's most practical insight.

grep -r is naive. Ask it to, and it will churn through node_modules, .git, build artifacts, and log directories. The source code the user actually wants is a tiny fraction of the whole, yet grep spends its time sweeping the giant pile of junk around it.

ripgrep respects .gitignore by default. Files and directories caught by ignore rules are never read; they are skipped outright. In real projects, the things listed in .gitignore — dependencies, build caches, artifacts — usually account for most of a repository's size. Skipping them wholesale dramatically shrinks the amount of data that has to be searched at all.

The ignore rules ripgrep consults come in several layers:

• .gitignore — applied hierarchically per directory
• .ignore — a general-purpose ignore file shared by ripgrep and other tools
• .rgignore — a ripgrep-specific ignore file
• The global gitignore and .git/info/exclude

The .gitignore syntax itself is no small thing. Glob patterns, negation (!), directory scoping, per-level overrides — all of it has to be implemented correctly. ripgrep handles this rule matching in a separate, well-optimized library. For example, given a .gitignore like this:

# Exclude dependencies and build artifacts from search
node_modules/
dist/
*.log
build/

# But do track this one log
!important.log

ripgrep interprets the rules exactly: it skips node_modules/, dist/, build/, and every *.log, but still searches important.log. And if you want to turn off ignore rules and truly search everything, rg -uuu peels back the ignore behavior in stages.

This design embodies a shift in mindset. grep treats "search everything by default, exclusion is the exception." ripgrep treats "search only meaningful source by default, searching everything is the exception." It sets the default to what a developer actually wants.

Smart Case — Inferring Human Intent

Smart case is a usability optimization more than a speed one, but it captures ripgrep's "make the default what the human wants" philosophy well.

The rule is simple:

• If the query is all lowercase → search case-insensitively.
• If the query contains any uppercase letter → search case-sensitively.

For example, rg error finds error, Error, and ERROR. Since you typed all lowercase, it reads your intent as "show me anything." Meanwhile rg Error finds exactly Error. Since you deliberately typed an uppercase letter, it reads your intent as "exactly this form."

This behavior matches real developer habits remarkably well. When searching loosely you type loosely in lowercase; when hunting for a specific symbol (MyClass, HTTPError) you type it exactly. The tool reads that habit. If you want to force it explicitly, -s (case-sensitive) or -i (case-insensitive) override the behavior.

Skipping Binary Files — Avoiding Useless Data

Another "not reading" optimization. By default, ripgrep detects and skips binary files.

Searching for text with a regex inside images, executables, or compressed archives is usually meaningless. And such binaries are often large. Sweeping through them wastes not only time but can also flood your terminal with control characters and mangle the screen.

ripgrep's detection is pragmatic. It reads a bit of the start of a file and checks for a NUL byte (\0). Text files usually have no NUL bytes; binaries commonly do. If a NUL is found, the file is treated as binary and processing stops. It is not a perfect classifier, but it is a cheap heuristic that works extremely well in practice.

Of course you can force the issue. rg -a (or --text) treats binaries as text and searches them, and --binary reports matches even in binaries. The default is "skip" because that is overwhelmingly what developers want.

mmap — Mapping a File Straight Into Memory

There is optimization in how files get read, too. Depending on the situation, ripgrep uses mmap (memory mapping).

Ordinary file reads use the read() system call to copy data from a kernel buffer into a user-space buffer. mmap, by contrast, maps the file directly into the process's virtual address space. You can then access the file's contents as if it were one giant byte array, and the actual disk reads happen lazily through page faults, only when needed. One data-copy step is eliminated.

mmap tends to win when searching a single large file, because the mapping overhead is amortized across the whole large file. Conversely, when searching many small files, the cost of creating and tearing down a mapping per file can outweigh the benefit, and ordinary buffered reads are faster.

So ripgrep does not commit to one approach. It looks at the number and size of files and heuristically chooses between mmap and normal reads. If needed, --mmap or --no-mmap force the choice. The lesson here is not "mmap is always faster" but "pick the I/O strategy that fits the situation." There is no universal answer.

Putting the Pieces Together — The Whole Pipeline

Drawn as a single flow, the optimizations so far look like this:

  run rg PATTERN
        │
        ▼
  [1] start traversing directories with parallel workers
        │
        ▼
  [2] exclude entire paths that don't need reading,
      via .gitignore / .ignore rules
        │
        ▼
  [3] per remaining file: check if binary (NUL byte)
        │           if binary, skip
        ▼
  [4] text file: load via mmap or buffered read
        │
        ▼
  [5] SIMD memchr scans for required-byte candidates at high speed
        │
        ▼
  [6] run finite-automaton regex matching only near candidates
        │
        ▼
      print matches

Notice how each stage shrinks the amount of data handed to the next. .gitignore reduces the file count, binary detection reduces it further, and memchr reduces the positions where matching is attempted. By the time the expensive operation (regex matching) is reached, the data has already been filtered down to a minimum.

Lessons for Writing Fast Tools

Take ripgrep apart and general performance principles emerge that go well beyond this one domain.

• The fastest work is the work you don't do. The gains from respecting .gitignore and skipping binaries often exceed the gains from a clever matching algorithm. Design what not to process first.
• Filter cheaply first, run the expensive check only where needed. Narrow candidates with SIMD memchr before running the regex. The point is to keep most data from ever reaching the expensive path.
• Guarantee the worst case. Choosing finite automata for a linear-time guarantee prioritizes "never explodes on any input" over average performance. A tool loses trust in its worst moment.
• Good defaults are a feature. Smart case, respecting .gitignore, and skipping binaries all default to "what the user usually wants." Doing the right thing with no configuration is what real usability looks like.
• Don't commit to one strategy. Like mmap versus buffered reads, adaptively picking the favorable option beats a single fixed answer.
• The language only sets the floor. Rust provided zero-cost abstractions and safe parallelism, but the essence of the speed is in the algorithmic and structural decisions above.

I want to emphasize that last point. "It's fast because it's Rust" is only half true. Rust gave a good foundation of lightweight abstractions and fearless concurrency, but the real reason ripgrep is fast is a design that doesn't read what it won't read, does the expensive work as little as possible, and guarantees the worst case. Those principles apply everywhere — command-line tools, web servers, data pipelines.

Conclusion

ripgrep's speed is not a single piece of magic but the sum of disciplined engineering. In the regex engine it gave up some expressiveness for linear time; in I/O it made "not reading" the default; on the CPU it processed bytes in bulk with SIMD; on the filesystem it traversed in parallel. Each is a small decision, but together they add up to a tool that feels categorically different.

The next time you sweep a giant repository with rg in an instant, picture how these pieces interlock behind it. And ask the same question of the tools you build: "This work — could I just not do it in the first place?"

References

• ripgrep repository: https://github.com/BurntSushi/ripgrep
• ripgrep is faster than grep, ag, and git grep (by author Andrew Gallant): https://blog.burntsushi.net/ripgrep/
• Rust regex crate: https://docs.rs/regex/
• memchr crate: https://docs.rs/memchr/
• RE2 (finite-automaton regex engine): https://github.com/google/re2
• Regular Expression Matching Can Be Simple And Fast (Russ Cox): https://swtch.com/~rsc/regexp/regexp1.html