https://www.youngju.dev/blog 천천히 올바르게. AI Researcher & DevOps Engineer Youngju's blog. GPU/CUDA, LLM, MLOps, Kubernetes AI workloads, and data engineering — plus mindset essays on confidence, routines, health, and sport psychology. ko fjvbn2003@gmail.com (Youngju Kim) fjvbn2003@gmail.com (Youngju Kim) Wed, 12 Aug 2026 00:00:00 GMT https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.en https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.en The title harness engineer is still rare in job postings, but the work already exists in every team shipping agents. The final part 8 of the harness engineering series covers why this job emerged, how existing software skills get rearranged into it, and a step-by-step practice path through the six tiers of the harness RPG, from observation and fingerprints to the self-improvement loop. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트careerai-engineer https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.ja ハーネスエンジニアという肩書きは求人票にはまだ珍しいものの、その仕事はエージェントをデプロイするすべてのチームにすでにあります。ハーネスエンジニアリング連載の最終回である第8回では、この職務がなぜ生まれたのか、既存のソフトウェア技能がどう再配置されるのか、そして観測と指紋から自己改善ループまで、ハーネスRPGの6ティアで段階的に練習する経路を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트careerai-engineer https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide 하네스 엔지니어라는 직함은 채용 공고에 드물지만, 그 일은 에이전트를 배포하는 모든 팀에 이미 있습니다. 하네스 엔지니어링 시리즈 마지막 8편에서 이 직무가 왜 생겼는지, 기존 소프트웨어 역량이 어떻게 재배치되는지, 그리고 관측과 지문부터 자기 개선 루프까지 여섯 개의 근육을 하네스 RPG의 6개 티어로 단계별 연습하는 경로를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트careerai-engineer https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-career-guide.zh 框架工程师（harness engineer）这个头衔在招聘启事里还很少见，但这份工作已经存在于每一个部署智能体的团队里。框架工程系列收官的第 8 篇，整理了这个岗位为何出现、既有软件技能如何重新排布，以及借助框架 RPG 的六个层级，从观测与指纹到自我改进循环的分级练习路径。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트careerai-engineer https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.en https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.en The context window still has room, yet agent accuracy is dropping. Context is a finite attention budget, and tool schemas spend it too. Part 2 of the harness engineering series covers turning prompt accumulation into a playbook, choosing drop policies and compaction criteria, and the real cost of delegating to subagents. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트context-engineering컨텍스트엔지니어링 https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.ja コンテキストウィンドウにはまだ余裕があるのに、エージェントの正確さは落ちていきます。コンテキストは有限の注意予算であり、ツールのスキーマもその予算を食います。ハーネスエンジニアリング連載第2回では、プロンプト累積をプレイブックに変える方法、ドロップポリシーとコンパクションの基準、サブエージェント委任の本当のコストまで、コンテキスト予算の設計を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트context-engineering컨텍스트엔지니어링 https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering 컨텍스트 창은 아직 남았는데 에이전트 정확도는 떨어집니다. 컨텍스트는 유한한 주의 예산이고, 도구 스키마도 그 예산을 먹습니다. 하네스 엔지니어링 시리즈 2편에서 프롬프트 누적을 플레이북으로 바꾸는 법, 드롭 정책과 컴팩션의 기준, 서브에이전트 위임의 비용까지 컨텍스트 예산 설계를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트context-engineering컨텍스트엔지니어링 https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-context-engineering.zh 上下文窗口明明还有富余，智能体的准确率却在往下掉。上下文是有限的注意力预算，工具的 schema 也在花这笔预算。框架工程系列第 2 篇，整理了把提示词堆积改成 playbook 的做法、丢弃策略与压缩（compaction）的标准，以及委派子智能体的真实成本。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트context-engineering컨텍스트엔지니어링 https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.en https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.en Two teams use the same model, so why do their agents perform so differently? For most teams the model is a fixed input, and what actually ships is the harness around it: the tool surface, the failure return format, the loop and its stopping conditions, the context policy, the permissions, and the evaluator. Part 1 of the harness engineering series covers the definition of a harness, its six knobs, and why the name prompt engineering undersells this work. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트prompt-engineering https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.ja 同じモデルを使っているのに、なぜチームごとにエージェントの成果が違うのでしょうか。ほとんどのチームにとってモデルは固定入力であり、実際にデプロイしているのはツール表面、失敗の返し方、ループと停止条件、コンテキストポリシー、権限、評価者まで、モデルを取り囲むハーネス全部です。ハーネスエンジニアリング連載の第1回として、ハーネスの定義と6つのつまみ、そしてプロンプトエンジニアリングという名前がこの仕事を過小評価する理由を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트prompt-engineering https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is 같은 모델을 쓰는데 왜 팀마다 에이전트 성과가 다를까요. 대부분의 팀에게 모델은 고정 입력이고, 실제로 배포하는 것은 도구 표면, 실패 반환 형식, 루프와 정지 조건, 컨텍스트 정책, 권한, 평가자까지 모델을 둘러싼 하네스 전부입니다. 하네스 엔지니어링 시리즈 1편으로, 하네스의 정의와 여섯 개의 손잡이, 그리고 프롬프트 엔지니어링이라는 이름이 이 일을 과소평가하는 이유를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트prompt-engineering https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-engineering-what-it-is.zh 用的是同一个模型，为什么各团队的智能体表现差这么多？对大多数团队来说，模型是固定输入，真正交付的是围绕模型的整个框架（harness）：工具表面、失败返回格式、循环与停止条件、上下文策略、权限，以及评估者。这是框架工程系列第 1 篇，整理了框架的定义、六个旋钮，以及为什么提示词工程这个名字严重低估了这项工作。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트prompt-engineering https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.en https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.en If the score does not move no matter how much you fix the harness, the bottleneck may be the evaluator, not the harness. You cannot select for a quality you cannot measure, which is why a weak grader becomes the ceiling of the whole system. Part 5 of the harness engineering series lays out the evaluator ladder from smoke tests through unit tests, rubrics, sealed grading data, and counter-metric panels, and why evaluator calibration has to come first. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트evaluationllm-as-judge https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.ja ハーネスをどれだけ直してもスコアが動かないなら、ボトルネックはハーネスではなく評価者かもしれません。測定できない品質は選択できず、だから弱い採点者がシステム全体の上限になります。ハーネスエンジニアリング連載第5回では、スモークテストからユニットテスト、ルーブリック、採点データの隔離、牽制指標のパネルまで、評価者のはしごと、評価の較正を先にすべき理由を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트evaluationllm-as-judge https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck 하네스를 아무리 고쳐도 점수가 안 오른다면, 병목은 하네스가 아니라 평가자일 수 있습니다. 측정할 수 없는 품질은 선택할 수 없고, 그래서 약한 채점자가 시스템 전체의 상한이 됩니다. 하네스 엔지니어링 시리즈 5편에서 스모크 테스트부터 단위 테스트, 루브릭, 채점 자료 격리, 견제 지표 패널까지 평가자의 사다리와 평가 보정을 먼저 해야 하는 이유를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트evaluationllm-as-judge https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-evaluator-bottleneck.zh 不管怎么修框架，分数就是不动——瓶颈可能不在框架，而在评估者。测不出来的质量就选不出来，所以弱评分者会成为整个系统的上限。框架工程系列第 5 篇，梳理了从冒烟测试到单元测试、评分细则（rubric）、封存评分数据、牵制指标面板的评估者阶梯，以及为什么必须先校准评估。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트evaluationllm-as-judge https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.en https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.en The success rate moved with no prompt commit and no model change — so what do you roll back? Part 7 of the harness engineering series covers the harness fingerprint: a single normalized hash summarizing every decision that makes up the harness. What goes into the fingerprint and what stays out, why comparisons only hold between equal fingerprints, and how to bisect regressions and roll back over the fingerprint history. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트versioningregression-testing https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.ja プロンプトのコミットもモデル変更もないのに成功率が動いたなら、何をロールバックすべきでしょうか。ハーネスエンジニアリング連載第7回は、ハーネスを構成するすべての決定を正規化したハッシュひとつに要約するハーネス指紋を扱います。指紋に何を入れて何を外すか、なぜ指紋が同じでなければ比較が成立しないのか、そして指紋の履歴でリグレッションを二分探索してロールバックする方法まで整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트versioningregression-testing https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting 프롬프트 커밋도 모델 변경도 없는데 성공률이 움직였다면, 무엇을 되돌려야 할까요. 하네스 엔지니어링 시리즈 7편은 하네스를 구성하는 모든 결정을 정규화된 해시 하나로 요약하는 하네스 지문을 다룹니다. 지문에 무엇을 넣고 무엇을 빼는지, 왜 지문이 같아야 비교가 성립하는지, 그리고 지문 이력으로 회귀를 이등분해 롤백하는 방법까지 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트versioningregression-testing https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-fingerprinting.zh 没有提示词提交、没有换模型，成功率却动了——该回滚什么？框架工程系列第 7 篇讲框架指纹：把构成框架的所有决定归一化后压成一个哈希。指纹里放什么、不放什么，为什么指纹相同比较才成立，以及如何沿着指纹历史二分定位回归并回滚。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트versioningregression-testing https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.en https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.en Agent loops fail in two directions: the infinite loop that repeats the same call dozens of times, and the early stop that quits at the first obstacle. Part 4 of the harness engineering series covers retry caps, the three stopping conditions of fixed steps, goal checks, and confidence, the trap in confidence-based stopping, and escalation to humans or subagents. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트agent-loop https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.ja エージェントのループは2方向に失敗します。同じ呼び出しを何十回も繰り返す無限ループと、最初の障害で諦める早すぎる停止です。ハーネスエンジニアリング連載第4回では、再試行上限、固定ステップ・目標チェック・確信度という3つの停止条件、確信度ベース停止の落とし穴、そして人間やサブエージェントへのエスカレーションまで、ループ設計を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트agent-loop https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping 에이전트 루프의 실패는 두 방향입니다. 같은 호출을 수십 번 반복하는 무한 루프와, 한 번 막히자마자 포기하는 조기 정지. 하네스 엔지니어링 시리즈 4편에서 재시도 상한, 고정 스텝·목표 체크·확신도라는 세 가지 정지 조건, 확신도 기반 정지의 함정, 그리고 사람·서브에이전트로의 에스컬레이션까지 루프 설계를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트agent-loop https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-loops-and-stopping.zh 智能体循环的失败有两个方向：把同一个调用重复几十次的无限循环，和一碰到障碍就收工的过早停止。框架工程系列第 4 篇，整理了重试上限，固定步数、目标检查、置信度这三类停止条件，置信度停止的陷阱，以及向人和子智能体升级的路径。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트agent-loop https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.en https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.en If the cheapest way for an agent to pass the tests is to edit the tests, the agent will edit the tests. Reward hacking is not a bug; it is the exact optimization of the goal we wrote down. Part 6 of the harness engineering series covers the common forms such as loosening grading criteria and deleting assertions, the half that permission isolation erases, and the design of counter-metrics. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트reward-hackingevaluation https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.ja エージェントにとってテストを通す最も安い方法がテストを書き換えることなら、エージェントはそうします。リワードハッキングはバグではなく、私たちが定義した目標の正確な最適化の結果です。ハーネスエンジニアリング連載第6回では、採点基準の緩和やassertionの削除といったよくある形、権限の隔離が消してくれる半分、そして牽制指標の設計まで、リワードハッキングへの対応を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트reward-hackingevaluation https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking 에이전트가 테스트를 통과시키는 가장 싼 방법이 테스트를 고치는 것이라면, 에이전트는 그렇게 합니다. 리워드 해킹은 버그가 아니라 우리가 정의한 목표를 정확히 최적화한 결과입니다. 하네스 엔지니어링 시리즈 6편에서 채점 기준 완화와 assertion 삭제 같은 흔한 형태, 권한 격리가 지우는 절반, 그리고 견제 지표 설계까지 리워드 해킹 대응을 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트reward-hackingevaluation https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-reward-hacking.zh 如果对智能体来说，让测试通过最便宜的办法是改测试，它就会去改测试。奖励作弊不是 bug，而是对我们写下的目标的精确优化。框架工程系列第 6 篇，整理了放宽评分标准、删除断言这类常见形态，权限隔离能抹掉的那一半，以及牵制指标的设计。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트reward-hackingevaluation https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.en https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.en Adding more tools and watching the agent success rate drop is not rare. The tool surface is the agent interface, and the names, descriptions, parameters, failure returns, and response sizes are all design material. Part 3 of the harness engineering series covers the curse of tool count, namespacing and description copy, poka-yoke parameters, and the format in which failure comes back. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트tool-designmcp https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.ja https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.ja ツールを増やしたのにエージェントの成功率が下がる、という事態は珍しくありません。ツール表面はエージェントのインターフェースであり、名前・説明・パラメータ・失敗の返し方・応答サイズのすべてが設計対象です。ハーネスエンジニアリング連載第3回では、ツール数の呪い、ネームスペーシングと説明文、ポカヨケなパラメータ、失敗を返す形式まで、ツール表面の設計を整理しました。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트tool-designmcp https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface 도구를 더 붙였는데 에이전트 성공률이 떨어지는 일은 드물지 않습니다. 도구 표면은 에이전트의 인터페이스이고, 이름·설명·파라미터·실패 반환·응답 크기가 전부 설계 대상입니다. 하네스 엔지니어링 시리즈 3편에서 도구 수의 저주, 네임스페이싱과 설명 문구, 포카요케 파라미터, 실패를 돌려주는 형식까지 도구 표면 설계를 정리했습니다. Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트tool-designmcp https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.zh https://www.youngju.dev/blog/llm/2026-08-12-harness-tool-surface.zh 加了更多工具，智能体成功率反而下降，这并不少见。工具表面就是智能体的接口：名字、描述、参数、失败返回、响应大小，全都是设计对象。框架工程系列第 3 篇，整理了工具数量的诅咒、命名空间与描述文案、防错（poka-yoke）参数设计，以及失败以什么格式返回。 Wed, 12 Aug 2026 00:00:00 GMT fjvbn2003@gmail.com (Youngju Kim) llmagentharness-engineering하네스엔지니어링AI에이전트tool-designmcp