Reading Git commands through its internals — reset and rebase seen as objects, refs and the index

Introduction — memorizing commands versus knowing the model

Search for how to use Git and you mostly get lists of commands. This command for this situation, that command for that one. The problem is that a situation not on the list will definitely arrive. At that moment the person who memorized commands searches again, and the person who knows the data model deduces.

Happily the model you have to memorize is tiny. Four object types, refs, the index. Those three are all of it. This post opens up those three, then walks back through reset, rebase, cherry-pick and detached HEAD to see why each behaves the way it does. If you are curious about how the data itself is stored, How Git Actually Stores Your Data goes into more detail. This post focuses on re-reading the commands with that model.

Four objects — and the same content gets the same hash

Git stores only four kinds of object.

A blob holds file content only. There is no file name in it, no permissions, no location. A tree represents a single directory and holds a list of mode, type, hash and name. A commit holds one tree plus its parent commits, the author and committer, and the message. A tag object is created when you make an annotated tag and holds the target object, the tag name and the signature.

And the name of every one of these objects is a hash of its content. So if the content is the same, the name is the same. That is easy to verify.

$ printf 'hello\n' > hello.txt
$ cp hello.txt copy.txt
$ git add hello.txt copy.txt

$ git ls-files --stage
100644 ce013625030ba8dba906f756967f9e9ca394464a 0	copy.txt
100644 ce013625030ba8dba906f756967f9e9ca394464a 0	hello.txt

The file names differ and the hash is identical. Only one object is stored. If the content is the same, no matter where in the repository or how many times you commit it again, the object count does not grow.

There is a consequence of this that confuses people regularly in practice. Because a blob does not know its own name, Git does not record file renames. The tree holds the name, and a change that only renames something is just a new commit with a different tree. The rename markers you see in logs and diffs are not a stored fact, they are Git guessing on the spot from content similarity. That is why the option for following history across renames sometimes behaves oddly.

Opening them up — hash-object and cat-file

How the hash is computed takes two commands to verify.

$ printf 'hello\n' | git hash-object --stdin
ce013625030ba8dba906f756967f9e9ca394464a

$ printf 'blob 6\0hello\n' | shasum
ce013625030ba8dba906f756967f9e9ca394464a  -

The second line is what Git actually does. It prepends a header made of the type name, the byte count and a null byte, then hashes the whole thing. Because of that header, the same byte sequence gets a different hash as a blob than as a tree.

A commit is nothing but text either.

$ git commit -q -m "첫 커밋"
$ git cat-file -p HEAD
tree aaa96ced2d9a1c8e72c56b253a0e2fe78393feb7
author Demo <demo@example.com> 1785081963 +0900
committer Demo <demo@example.com> 1785081963 +0900

첫 커밋

There is no parent line. This is the initial commit. Now follow the tree one step deeper.

$ git cat-file -t aaa96ce
tree

$ git cat-file -p aaa96ce
100644 blob ce013625030ba8dba906f756967f9e9ca394464a	hello.txt

The commit points at a tree, and the tree holds names paired with blobs. That structure simply repeats recursively across the whole repository. There is not one byte of file content inside a commit object, and not even a file name.

If you want to see how many objects there are and what type they all are at once, there is a batch query.

$ git cat-file --batch-check --batch-all-objects | head -4
ad50a5790e96b8f448d121aa76cee2cc3f05afce commit 178
aaa96ced2d9a1c8e72c56b253a0e2fe78393feb7 tree 37
ce013625030ba8dba906f756967f9e9ca394464a blob 6

Refs — why a branch is a 41-byte file

A branch is not a data structure. It is a text file containing one commit hash.

$ cat .git/HEAD
ref: refs/heads/main

$ cat .git/refs/heads/main
951f4a64d6435393fdedb7eb6aadd53939826b4c

$ wc -c .git/refs/heads/main
      41 .git/refs/heads/main

Forty hex characters and one newline. So creating a branch is the act of writing one file, and it costs the same no matter how large the repository is. Carrying over the instinct from version control systems where branching was a heavy operation makes people ration branches, and there is no reason at all to ration them.

HEAD has one more layer. Instead of pointing at a commit directly, it is a symbolic ref that points at a branch name. That is why making a commit updates the file of the branch HEAD points at.

$ git symbolic-ref HEAD
refs/heads/main

$ git update-ref refs/heads/experiment 951f4a6
$ git rev-parse experiment
951f4a64d6435393fdedb7eb6aadd53939826b4c

There is one trap. Refs do not always exist as files. Once housekeeping has run, refs get bundled into a single file and the individual files disappear.

$ git pack-refs --all
$ ls .git/refs/heads/
$ cat .git/packed-refs
# pack-refs with: peeled fully-peeled sorted
951f4a64d6435393fdedb7eb6aadd53939826b4c refs/heads/main

An empty directory does not mean the branches are gone. Instead of reading the files directly, always ask Git.

$ git rev-parse main
951f4a64d6435393fdedb7eb6aadd53939826b4c
$ git show-ref --heads

A new storage format is also being prepared for large repositories where refs grow into the tens of thousands. It is still experimental, but it is worth evaluating in environments where ref lookup itself is the bottleneck.

$ git init --ref-format=reftable myrepo

The index — what the staging area really is

The staging area is not a concept, it is a file. One binary file inside the repository, holding a list of paths, blob hashes and file state information.

$ git ls-files --stage
100644 ce013625030ba8dba906f756967f9e9ca394464a 0	hello.txt
100644 3b18e512dba79e4c8300dd08aeb37f8e728b8dad 0	note.txt

Two things should be read out of this.

First, the moment you stage a file its content is already stored as an object. Staging is not a reservation, it is a recording. That is why you get the result below.

$ echo v1 > note.txt
$ git add note.txt
$ echo v2 > note.txt
$ git commit -q -m "메모 추가"

$ git show HEAD:note.txt
v1
$ cat note.txt
v2

If you edit the file further after staging it, what gets committed is the content as of staging time. This is not a bug, it is the behavior that follows straight from the definition of what the index is. For the same reason, content that was only staged and never committed can be recovered after an accident. The object is already stored.

Second, the 0 in the listing is a stage number. Normally there is only stage 0, but during a conflict the same path expands into three lines.

$ git ls-files --unmerged
100644 a1b2c3d4e5f60718293a4b5c6d7e8f9012345678 1	src/checkout/retry.ts
100644 b2c3d4e5f60718293a4b5c6d7e8f90123456789a 2	src/checkout/retry.ts
100644 c3d4e5f60718293a4b5c6d7e8f90123456789ab1 3	src/checkout/retry.ts

Stage 1 is the common ancestor, 2 is the current branch, 3 is the incoming side. Being in a conflicted state means the index holds three candidates for one path. Resolve the conflict and stage it and the three lines collapse into a single stage 0 line, and only then can you commit. You can also pull out each version directly while resolving.

$ git show :1:src/checkout/retry.ts > base.ts
$ git show :2:src/checkout/retry.ts > ours.ts
$ git show :3:src/checkout/retry.ts > theirs.ts

The index also carries state information such as file size and modification time. That is why a status check can decide whether something changed without opening the files, and it is also why that very check becomes the bottleneck once you have hundreds of thousands of files. The prescription for that point is written up in When a repository gets slow.

The commands the model explains

Now read the commands again. There are only three criteria: does it create new objects, does it move a ref, does it overwrite the working tree.

Once you read the table, the frequently asked questions untangle themselves.

The reason reset is called a dangerous command is not that it moves a ref. Moving a ref means rewriting a 41-byte file, and the old value stays in the reflog. What is dangerous is only the option that overwrites the working tree. The last column of the table is the risk level.

Why rebase creates new commits instead of moving them is equally clear. A commit object holds the parent hash inside it and the name of the commit is a hash over that entire content, so if the parent changes the name has no choice but to change. Rewrite is not a metaphor, it is a literal description. The team rules that fact produces are covered in merge and rebase.

Why cherry-pick conflicts comes from here too. Cherry-pick does not cut and paste a stored diff. Git does not store diffs in the first place, only snapshots. So cherry-pick performs a three-way merge using the parent tree of the target commit as the common ancestor. The further the parent state of the commit you are pulling in is from the state of your current branch, the more conflicts you get. If you are repeatedly moving the same change across several branches, you are better off enabling reuse of recorded resolutions.

A detached HEAD is not a special state either. The HEAD file simply holds a commit hash directly instead of a branch name.

$ git switch --detach 951f4a6
HEAD is now at 951f4a6 메모 추가

$ cat .git/HEAD
951f4a64d6435393fdedb7eb6aadd53939826b4c

Make a commit in this state and HEAD itself is updated to point at the new commit, but there is no branch ref to update. So the moment you move somewhere else, nothing is left pointing at that commit. That is why Git is kind enough to warn you.

$ git switch -
Warning: you are leaving 1 commit behind, not connected to
any of your branches:

  d66381f 임시 실험

If you want to keep it by creating a new branch, this may be a good time
to do so with:

 git branch <new-branch-name> d66381f

Switched to branch 'main'

A detached HEAD is not a malfunction, it is a state with no name tag on it. The commit object is stored perfectly well, and attaching a name brings it right back.

From SHA-1 to SHA-256 — collision attacks and how Git responded

Naming objects by hash is an elegant design, but it is tied to the lifespan of the hash function. What Git picked in 2005 was SHA-1, and attacks have advanced steadily since.

In 2017 researchers at CWI Amsterdam and Google published two different PDF files with the same SHA-1 hash. In 2020 Leurent and Peyrin realized a chosen-prefix collision and brought the cost down to the range of tens of thousands of dollars. A chosen-prefix collision means the attacker can dictate the leading part, which is far closer to forging a real document.

The Git hash is primarily for integrity checking and identification, not a signature. But signed commits and tags ultimately sign that hash, so if collisions are possible the meaning of the signature is shaken.

The response went down two tracks. The first is a defense that could be applied immediately. Since Git 2.13 a SHA-1 implementation with collision detection ships by default. When it is asked to hash data bearing the traces of a known attack technique, it refuses instead of finishing the computation.

$ git hash-object --stdin < shattered-1.pdf
fatal: SHA-1 collision detected on 38762cf7f55934b34d179ae6a4c80cadccbb7f0a

The second is the fundamental fix. Since Git 2.29 the SHA-256 object format can be used experimentally.

$ git init --object-format=sha256 sha256demo
$ cd sha256demo

$ git rev-parse --show-object-format
sha256

$ printf 'hello\n' | git hash-object --stdin
2cf8d83d9ee29543b34a87727421fdecb7e3f3a183d337639025de576db9ebb4

Same content, completely different name, and 64 characters long. Inside the repository every command works as usual. It is not something to migrate to in production right now, though. Interoperability between SHA-1 and SHA-256 repositories is not finished, and the major hosting services do not accept them.

There is still something worth doing today. Scripts and regular expressions that assume a hash length of 40 will all break eventually. If you are building a tool, asking the repository is safer than hardcoding the length.

$ git rev-parse --show-object-format
sha1

Closing — three questions instead of a command table

You do not need to memorize every Git command. When you meet an unfamiliar one, ask three things. Does this command create new objects, does it move a ref, does it overwrite the working tree.

If it creates new objects the hashes change, so be careful with shared history. If it only moves a ref, the reflog will bring it back. If it overwrites the working tree, uncommitted changes vanish, and that is the one thing no tool can revive. The genuinely dangerous commands in Git are not the ones that change history, they are the ones that delete what has not become an object yet.

The undo commands organized with the same model are in Undoing by situation.