Split View: AI 에이전트 하네스 해부 — 규칙이 스스로 지켜지게 만든 35개 파일

Anatomy of an AI agent harness — 35 files that make the rules enforce themselves

The surroundings decide the result, not the model

With the same model, in one repository an agent wipes out other people's edits with git stash, and in another that never happens. The difference is not the model but what surrounds it: the harness. Where the rules are written, which commands are blocked, which gates a commit passes through, what survives after the session ends.

This post is a file-by-file dissection of the harness in the LabHub repository. Since its first commit on 20 August 2026 an AI agent has worked there alongside a person for three weeks, and the result is on record as numbers.

The last row is the subject of this post. Counting in the logs of this working session, out of those gate runs there were 127 successful pushes, 41 blocked by the gate, and 23 where origin had moved ahead and the commit was re-applied. The harness is not a document; it is a machine that runs dozens of times a day.

Principle one: rules in one place, mechanisms separate

The entire harness arrived in a single commit on 7 September 2026 (83821fa0, 35 files, 1,746 lines). The commit message says why.

The rules lived in one place, AGENTS.md, but the mechanisms that enforced
them were being rebuilt in a scratchpad every session (push.sh, gate.sh,
bumpdigest.py). They vanished when the session ended, and the next session
rebuilt the same tools only after suffering the same accident once more.

So the first structural principle is one source of truth for rules. The rules live in AGENTS.md (707 lines), and the CLAUDE.md that Claude Code reads is a pointer to that file. In CLAUDE.md's own words:

If the same content is written in two places, one of them is guaranteed to be left stale.

.claude/README.md defines what goes under .claude/:

The source of the rules is AGENTS.md. What lives here is only what Claude Code reads —
not a restatement of the rules, but the things that step in automatically so the rules are kept.

Five layers, plus two

The README divides the harness into five layers. I counted the actual files.

The criteria for what goes into which layer are also in the README. The sentence about hooks is the clearest:

There is one criterion for putting something in a hook — it actually happened here,
and it was hard to notice by eye. General best practices are not added. Once warnings
become common, nobody reads them.

For skills: "procedures repeated three or more times", but "the parts where a wrong order causes an accident go into scripts under scripts/agent/, not into the prose". Of the 13 plugins, commit-commands was deliberately left out, because /commit runs git commit in the working tree, which is forbidden in this repository.

Path rules: guidance that appears only when you open that file

Each of the six files in rules/ has a paths: field and enters the context only when those paths are touched. Same principle: if the whole rulebook is read every time, nobody reads it.

• app-js.md — backend/static/app.js has its whole-file SHA-256 embedded in the security check script, so "a single changed character blocks the deploy, and the message at the blocking point is misleading (it mentions streamSSE)".
• pipeline.md — "A push is polled by Jenkins every minute and goes to production without approval (about 20 minutes). Deploy only through GitOps."
• migrations.md — "Never modify a migration that has already been applied."
• generated.md — a table of files that vanish on the next generation if edited by hand, with their generators.
• curriculum.md — "Test graders in both directions — checking only that the right answer passes is half the job."
• tests.md — "Write in the test's docstring why the test exists and what accident it came from."

Tests watch these rules. tests/test_claude_harness.py checks that each paths: matches real files and that each body points back to AGENTS.md or scripts/.

Hooks: four things blocked, two things asked

hooks/guard.py receives the tool call on standard input before it runs and returns deny or ask on standard output. It contains exactly four things, each from a real accident.

GIT_STATE = re.compile(
    r"\bgit\s+(?:stash|checkout|switch|restore|reset|rebase|merge|pull|clean)\b")
GIT_OK = re.compile(r"\bgit\s+(?:worktree|stash\s+list)\b")
KUBECTL_WRITE = re.compile(
    r"\bkubectl\b[^|;&]*\s(?:apply|delete|patch|scale|edit|replace|annotate|label|cordon|drain)\b")
LOCKED = "backend/static/app.js"

1. Git commands that change the working tree are denied. The comment gives the reason: "One git stash reverted the edits of four other people."
2. kubectl commands that change cluster state are denied. Only reads and temporary pods are allowed.
3. A git push that bypasses push.sh gets ask, with evidence attached: "Today alone that gate blocked four CI failures."
4. Editing app.js gets ask, because forgetting the digest update blocks the deploy silently.

hooks/after_edit.py looks only at the file just edited: ast.parse for Python, node --check for JavaScript, json.loads for JSON. The full test suite takes over two minutes and cannot run on every edit, so this catches only "does this file still stand on its own" and leaves the rest to the gate.

The danger of hooks is in one README sentence: "Hooks die silently — if you change one and do not run the tests, the accident it was meant to prevent simply happens." So the 13 tests in tests/test_claude_hooks.py feed JSON directly to the hooks and confirm deny, ask, and pass.

Subagents: roles that only read

None of the three roles in agents/ has a writing tool.

A sentence in grader-reviewer's definition says why the role exists: "There really were several 'graders that always pass', and that is worse than none." The .claude/ survey for this post was itself delegated to safe-researcher. It saves the parent's context and removes any way to accidentally modify something during research.

Tests enforce this too: at least three agents, no Edit, Write or NotebookEdit in tools, and a negative phrase such as "does not" in the description.

Procedures as scripts: push.sh and gate.sh

The centre of gravity is the eight scripts in scripts/agent/. Two are the core.

push.sh is the only road to commit and push. It never commits in the working tree. It creates a fresh worktree from origin/main, copies only the files I named into it, runs the gate there, then commits and pushes. If someone else is editing the same file, instead of copying the whole file you pass APPLY="script:path", a script that applies only your change. If CI has pushed a deploy commit during the four minutes the gate runs and origin has moved ahead, your single commit is re-applied on the new origin/main by cherry-pick, up to three times. That retry happened 23 times in this session's logs.

The comment explains why the worktree path differs on every run: "With a fixed path two pushes overlapped and deleted each other's directories, and the test logs became entirely false."

gate.sh runs locally the same thing as CI's Verify stage: the security check, the curriculum check, the grader audit, then every unit test. The header comment explains why it exists.

Built after builds 441–446 failed six times at the same spot. Tests were being
hand-picked for every push, and while editing the curriculum test_i18n_catalog
was not picked. The moment you pick, you miss. So run everything, and filter
local-only failures (uninstalled dependencies, no DB) against gate_baseline.txt.

The comparison with the baseline uses comm -13. It used to be diff | grep, but under set -o pipefail diff's exit code failed the whole pipeline, so a new failure was found and then ignored. That is how test_ko_source_is_current leaked three times, builds 451–453. The incident is nailed down as a test: gate.sh must use comm -13 and must not use any diff other than git diff.

The node tests have a 300-second watchdog attached. node --test has no default time limit, and "activation.test.js hung twice on the Mac and held the whole push".

The locked file and the evidence of deployment

bumpdigest.py updates the digest of app.js, but always does three things together. It compares the count of trust boundaries (11 patterns such as streamSSE, renderTrustedLessonMarkdown, innerHTML, eval() against the file before the change; anything that differs must be explicitly approved with ALLOW; and a NOTE explaining what changed and why is attached to the history block. Expected values are not hard-coded as numbers because "it once blocked falsely after someone else's legitimate change". Five tests feed temporary files to this logic: unapproved changes rejected, the same history written twice rejected, an empty NOTE rejected.

deploy_status.sh "does not trust ArgoCD's green Synced". Both apps have selfHeal, so the screen is always green, and an image three days old really was shown as Synced once. So it reads four things separately: the results and durations of the last Jenkins builds, gitops newTag versus the actual Deployment image tag, the state of both ArgoCD apps, and whether the /app.js served by the production site is byte-for-byte identical to the file in that commit. The static file is the final evidence of a deploy.

The harness tests itself

The 21 tests in tests/test_claude_harness.py check the structure of the harness, not the code. A few:

• A skill's description must exceed 80 characters, contain "use", and say when not to use it.
• A skill body must have all four sections: ## Why, ## Procedure, ## Output format, ## Do not.
• Every scripts/agent/* a skill mentions must exist; .sh files must be executable and pass bash -n, .py files must pass compile().
• Nowhere in .claude/ or scripts/agent/ may there be a machine-specific path such as /Users/ or /private/tmp/claude.
• push.sh must contain git worktree add and no git stash, and any git checkout must be together with origin/main.
• The README must mention every skill directory and every plugin in enabledPlugins, and commit-commands must not be there.

Thanks to these tests the structure has not changed since it was introduced. Only two commits touched .claude/ and scripts/agent/: the introducing commit and one that tidied two lines of the baseline.

What survives outside the session: memory

If the harness inside the repository is "rules", what remains outside it is "experience". The memory directory of this working environment holds 65 files: 53 project facts, 9 pieces of user feedback, 3 external references. Each file holds one fact plus "why" and "how to apply", and one index line per file is read at the start of every session.

A few written this week show what gets kept: "the gate reads the baseline in my local copy, not the worktree", "making something is not the same as showing it", "a hand-made Job easily loses its pod labels", "a CronJob failure holds ArgoCD until the next success". None of these has a place in the code, and all of them would go stale in a document.

What actually happened in this session

Accidents still happen with a harness. They just happen at the blocking point. Yesterday, pushing a one-line validator fix took four push runs.

1. push.sh had been changed to take a commit message file as its first argument. Given a string, it ended with "commit message file not found".
2. The gate caught test_production_cli_…_fails_unprovisioned as a new failure. It was an environmental failure unrelated to my change: cryptography 50.0.1 on this Mac differed from the locked 50.0.0, so a different error came first.
3. The same failure was caught again. The cause: gate.sh reads gate_baseline.txt from the local copy, not the worktree, and the local copy was stale and the file empty.
4. With the baseline fixed it passed, and since CI had pushed a deploy commit in the meantime it went through "origin is ahead — re-applying (1)" before it was pushed.

None of the four reached production. That is what all 41 gate blocks look like. And the lesson from step 3 became a memory file so the next session does not stop at the same spot.

There was one trap in an APPLY script too. The script first checks "has this already been applied?", and the key for that check was a string that also existed in the original (resolve(demo.map(lang))), so it said "already there" and did nothing. The key must be a string that only the new version contains.

Limits

This harness is not finished. The documentation itself names two limits.

There is no human approval stage. From docs/CI-CD.md: "Whatever is pushed to main goes straight to production in the same job once it passes Verify and the dev E2E. Those two gates are all that protects production, which is why what hangs on them matters." The approval gate is written down as the goal of the fail-closed design and does not run yet.

The baseline differs per machine. gate_baseline.txt filters failures of the local environment, yet it lives in the repository while in practice it must differ from machine to machine. Incident 3 above came from that contradiction.

And during a rolling deploy two pods serve different builds. Yesterday the browser kept a script received from the old pod in its cache and a new feature was briefly invisible. The harness protects commits and deploys; it does not yet protect the browser cache.

In one line

A harness is not about writing more rules; it is about stacking, one at a time, mechanisms that automatically block accidents that actually happened, at the spot where they happened. Rules in one place, mechanisms at the chokepoints of tool calls, commits and deploys, experience in memory outside the session. In this repository those three layers run dozens of times a day, and the result is recorded as 41 gate blocks.