Skip to content

Split View: AI 엔지니어 회의 영어: 모른다고 말하면서 신뢰를 잃지 않는 법

✨ Learn with Quiz
|

AI 엔지니어 회의 영어: 모른다고 말하면서 신뢰를 잃지 않는 법

들어가며: 이 분야에서 제일 많이 하는 말

AI 팀 회의 녹취를 훑어보면 눈에 띄는 점이 있습니다. 시니어일수록 확신하는 문장을 적게 씁니다. 대신 아는 것과 모르는 것의 경계를 긋는 문장을 많이 씁니다. 이 두 가지는 완전히 다른 능력입니다.

영어 회의 자료는 대부분 반대로 가르칩니다. 자신 있게 말하라고, 애매한 표현을 줄이라고 합니다. 일반적인 사무 환경에서는 맞는 조언일 수 있습니다. 그런데 AI 개발에서는 이 조언을 그대로 따르면 오히려 신뢰를 잃습니다.

이유는 이 분야의 숫자가 전부 조건부이기 때문입니다. 어떤 평가 점수든 어떤 하네스로, 어떤 심사자로, 어떤 데이터에서 쟀는지에 따라 달라집니다. 리더보드와 벤치마크를 읽는 법에서 정리한 것처럼, 자체 보고 수치와 하네스 차이와 심사자 편향은 예외가 아니라 기본 조건입니다. 그런 분야에서 조건 없이 숫자만 말하는 사람은 자신감 있어 보이는 게 아니라 조건을 모르는 사람으로 보입니다.

그래서 이 글의 주제는 하나입니다. AI 회의에서 가장 자주 필요한 영어는 확신의 언어가 아니라 불확실성을 신뢰감 있게 표현하는 언어입니다. 모르겠다는 말을 무능하지 않게 하는 능력이 이 분야의 핵심 회의 기술입니다.

왜 초반의 모른다가 신용 자산인가

먼저 왜 이게 손해가 아닌지부터 정리하겠습니다. 이 논리를 스스로 납득하지 않으면 실제 회의에서 그 문장이 안 나옵니다.

불확실성을 정확히 말하는 사람은 확실하다고 말할 때 믿을 수 있습니다. 반대로 매번 확신하는 사람은 한 번 틀리는 순간부터 모든 숫자에 할인이 붙습니다. 이게 실질적인 손해입니다. 다음에 정말로 확실한 결과를 들고 갔을 때 그 확신이 전달되지 않습니다.

두 번째 이유는 이 분야의 구조에서 옵니다. 숨긴 회귀는 반드시 드러납니다. 다른 사람이 같은 평가를 돌리고, 다른 팀이 같은 모델을 붙여 보고, 결국 사용자가 봅니다. 늦게 드러날수록 비용이 큽니다. 이미 그 결과 위에 다른 결정이 쌓였기 때문입니다.

세 번째는 시간입니다. 원인을 다 밝힌 다음에 보고하겠다는 태도는 성실해 보이지만, 그 사이에 팀은 잘못된 전제로 일정을 짭니다. 정보가 없어서 못 하는 결정과, 잘못된 정보로 이미 내린 결정은 복구 비용이 다릅니다.

이 글에는 결과를 좋아 보이게 만드는 문장이 없습니다. 의도적으로 뺐습니다. 그런 문장은 한두 번 통하고 그 뒤로는 통하지 않으며, 통하지 않게 된 시점에는 이미 커리어 비용이 발생한 뒤입니다.

신뢰감 있는 불확실성의 세 조각

모른다는 말이 무능하게 들리는 건 모르기 때문이 아닙니다. 어디까지 모르는지를 말하지 않기 때문입니다. 같은 상황에서 나온 두 문장을 비교해 보겠습니다.

[신뢰를 잃는 형태]
I'm not sure. I'll look into it.

[신뢰를 얻는 형태]
I don't know yet whether it's the model or the harness.
What I do know is that it started after Tuesday's deploy,
and it only shows up on the long-context slice.
I'm bisecting the commits and I'll have an answer Thursday.

두 번째 문장에도 모른다는 말이 그대로 들어 있습니다. 정보량이 다를 뿐입니다. 여기에 들어간 세 조각이 이 글의 뼈대입니다.

조각하는 일영어에서의 형태
경계아는 것과 모르는 것의 선을 긋는다I don't know whether A or B. What I do know is C.
근거지금 아는 것이 어디서 왔는지 밝힌다This is from a single run on our internal set.
다음 단계그 경계를 좁힐 행동과 시점을 준다I'm running X. I'll have an answer by Thursday.

경계가 빠지면 무능하게 들립니다. 모른다는 상태만 보고하는 셈이라 듣는 사람이 할 수 있는 게 없습니다.

근거가 빠지면 가볍게 들립니다. 같은 82점도 한 번 돌린 값인지 다섯 번 평균인지에 따라 완전히 다른 문장인데, 그 차이가 사라집니다.

다음 단계가 빠지면 회피처럼 들립니다. 이 부분이 가장 자주 빠집니다. 모른다고 말한 뒤에 침묵하면 상대가 다음 행동을 만들어야 하고, 그러면 모른다는 말의 대가를 상대가 치릅니다.

아래 상황별 표현은 전부 이 세 조각의 변주입니다. 표현을 외우기보다 어느 조각이 빠졌는지 보는 눈을 기르는 편이 오래 갑니다.

상황 1: 평가 결과를 발표하면서 한계를 함께 말하기

가장 자주 오는 상황이고, 가장 자주 실수하는 상황입니다. 흔한 실수는 숫자를 먼저 말하고 한계를 나중에 붙이는 것입니다. 그러면 한계가 변명처럼 들립니다. 한계를 먼저 말하고 숫자를 주면 같은 내용이 신중함으로 읽힙니다.

표현
Two caveats before I give you the number.숫자 드리기 전에 두 가지만 먼저요.
This is on our internal set, so it's not comparable to the published numbers.자체 세트 기준이라 공개 수치와 직접 비교는 안 됩니다.
The headline number is 82. I'd read that as a range, not a point.대표 수치는 82인데, 점이 아니라 범위로 보시는 게 맞습니다.
This eval doesn't capture multi-turn behavior at all.이 평가는 멀티턴 동작을 전혀 못 잡습니다.
What this measures is retrieval quality. What it doesn't measure is whether the answer is useful.이게 재는 건 검색 품질이고, 답이 쓸모 있는지는 안 잽니다.
I'd treat this as directional rather than decisive.방향 참고용이지 결론으로 쓸 숫자는 아닙니다.
The interval overlaps with the baseline, so I wouldn't call it a win yet.구간이 베이스라인과 겹쳐서 아직 개선이라고 부르기는 이릅니다.
We're up two points, and two points is inside the noise on this set.2점 올랐는데 이 세트에서 2점은 잡음 범위입니다.
I ran it once. I'd want a second seed before we plan around it.한 번 돌렸습니다. 이걸 근거로 계획 세우기 전에 시드 하나 더 보고 싶습니다.

숫자를 말할 때 함께 붙이면 좋은 조건이 넷 있습니다. 어떤 데이터, 몇 번 실행, 어떤 심사자, 무엇과 비교. 넷을 다 말할 필요는 없지만, 하나도 안 말하면 그 숫자는 회의실에서 실제보다 단단하게 유통됩니다.

조건을 붙이는 표현
That's on the held-out set, not the one we tuned on.튜닝에 쓴 세트가 아니라 격리 세트 기준입니다.
Three runs, and that's the mean. The spread was about a point and a half.세 번 평균이고 편차는 1.5점 정도였습니다.
The grader is the same model family, which I'd flag as a possible bias.심사자가 같은 모델 계열이라 편향 가능성은 짚어 두겠습니다.
Compared to last month's checkpoint, not to the vendor's number.벤더 수치가 아니라 지난달 체크포인트와 비교한 값입니다.

상황 2: 회귀가 났는데 원인을 아직 모를 때

여기서 가장 큰 유혹은 원인을 알아낸 다음에 말하는 것입니다. 하루를 아끼려다 팀 전체의 사흘을 태우는 선택입니다.

표현
Heads up: we've got a regression on the summarization eval.알려 드립니다, 요약 평가에서 회귀가 났습니다.캐주얼, 팀 채널
I want to flag a regression before it gets buried in the sprint update.스프린트 보고에 묻히기 전에 회귀 건을 먼저 알립니다.격식, 문서나 상급자
I don't have a cause yet.원인은 아직 못 찾았습니다.중립적 사실
I'd rather flag this early than wait until I have the full picture.전모를 파악할 때까지 기다리는 것보다 일찍 알리는 편을 택했습니다.태도 설명
It started somewhere between Tuesday and Thursday. I'm bisecting now.화요일과 목요일 사이에 시작됐고 지금 이분 탐색 중입니다.경계와 다음 단계
I can't rule out that it's the harness rather than the model.모델이 아니라 하네스 문제일 가능성도 아직 못 배제했습니다.정직한 범위
No action needed from you yet. I'll have more by Thursday.아직 조치하실 건 없습니다, 목요일에 다시 보고하겠습니다.상대 부담 제거
I'd rather over-report this than sit on it.묻어 두는 것보다 과하게 보고하는 편이 낫다고 봅니다.원칙 표명

Heads up은 엔지니어링 대화에서 표준에 가까운 표현이지만 구어체입니다. 임원에게 가는 문서나 격식 있는 메일에서는 I want to flag를 쓰는 편이 어울립니다. 이 판단은 회사 문화에 따라 다릅니다. 문서에서도 캐주얼한 톤을 쓰는 팀에서는 굳이 바꿀 필요가 없습니다.

No action needed from you yet은 짧지만 효과가 큽니다. 나쁜 소식을 들은 사람의 첫 반응은 대개 내가 뭘 해야 하나이고, 그 질문에 먼저 답해 주면 보고가 훨씬 가볍게 받아들여집니다.

상황 3: 지표는 올랐는데 실제 품질은 아닐 때

이 분야에서 가장 말하기 어려운 상황일 수 있습니다. 좋은 소식을 스스로 취소하는 자리이기 때문입니다. 그런데 이걸 말하지 않으면 팀 전체가 잘못된 방향으로 한 분기를 씁니다.

표현
The metric moved, but I don't think the task did.지표는 움직였는데 과제가 움직인 것 같지는 않습니다.
The score went up and the samples got worse. One of those is lying.점수는 올랐는데 샘플은 나빠졌습니다. 둘 중 하나는 거짓입니다.
This looks like the grader rewarding length rather than quality.심사자가 품질이 아니라 길이에 점수를 주고 있는 것 같습니다.
I think we optimized the scorer, not the task.우리가 최적화한 건 과제가 아니라 채점기인 것 같습니다.
Before we call this a win, I'd like to read twenty outputs by hand.개선이라고 부르기 전에 출력 스무 개를 직접 읽어 보고 싶습니다.
I'd call this a metric win, not a product win.지표상 개선이지 제품 개선은 아니라고 봅니다.
The gain disappears when I change the prompt format, which makes me suspicious.프롬프트 형식만 바꿔도 개선분이 사라져서 의심스럽습니다.

이 상황의 이름이 이미 있습니다. 리워드 해킹입니다. 지표는 진짜로 올랐는데 과제는 실패한 상태를 가리키고, 시스템 고장이 아니라 우리가 정의한 목표를 정확히 최적화한 결과라는 점이 핵심입니다.

영어권 ML 팀에서는 That might be Goodhart처럼 굿하트의 법칙을 줄여 부르는 말이 통합니다. 다만 이건 ML 팀 안에서만 통하는 축약이라 다른 부서가 섞인 회의에서는 풀어 말하는 편이 안전합니다. We may be optimizing the measure instead of the thing we care about 정도면 누구에게나 전달됩니다.

상황 4: 데이터 품질과 라벨링 이견

라벨링 논쟁이 길어지는 이유는 대개 사람들이 서로 다른 것을 논쟁하고 있어서입니다. 모델이 틀렸는지를 다투는 줄 알았는데 실은 라벨 정의를 다투고 있는 경우가 많습니다. 먼저 무엇을 다투고 있는지 이름 붙이는 문장이 시간을 가장 많이 아낍니다.

표현
I think we're disagreeing about the label definition, not about the model.우리가 다투는 건 모델이 아니라 라벨 정의인 것 같습니다.
Can we look at ten examples together?예시 열 개만 같이 볼까요?
What's our inter-annotator agreement on this slice?이 구간의 어노테이터 간 일치도가 어떻게 되나요?
I'd want to re-check the gold labels before we treat this as model error.모델 오류로 처리하기 전에 정답 라벨을 다시 보고 싶습니다.
Some of these look mislabeled to me. Can I flag a batch for review?일부는 라벨이 잘못된 것 같습니다. 한 배치 검토 요청해도 될까요?
I'm not questioning the annotators. I'm questioning whether the instructions were answerable.작업자를 문제 삼는 게 아니라 지침이 답할 수 있는 형태였는지를 묻는 겁니다.
If two careful people label this differently, we can't expect the model to pick one.신중한 두 사람이 다르게 라벨링한다면 모델이 하나를 고르길 기대할 수 없습니다.

여섯 번째 문장은 등록 판단이 들어간 표현입니다. 라벨 품질 문제를 제기하면 그 작업을 한 사람이나 팀을 비판하는 것처럼 들리기 쉽습니다. 비판의 대상을 사람에서 지침으로 옮기면 같은 문제 제기가 훨씬 안전하게 전달됩니다. 그리고 대개 이게 사실이기도 합니다. 라벨이 흔들리는 진짜 원인은 작업자의 성실성이 아니라 정의의 모호함인 경우가 많습니다.

일곱 번째 문장은 논증으로도 강합니다. 사람 사이에 일치하지 않는 기준을 모델에게 요구하는 건 불가능한 요구라는 지적이라, 반박하기가 어렵습니다.

상황 5: 비용과 지연 시간 트레이드오프

이 대화에서 엔지니어가 자주 하는 실수는 기술적 판단과 사업적 판단을 섞어 말하는 것입니다. 둘을 분리해서 말하면 발언의 무게가 오히려 올라갑니다. 내 영역에서는 단정하고, 남의 영역에서는 정보만 주는 형태입니다.

표현
What's our latency budget? I'd rather design to a number than to a feeling.지연 시간 예산이 어떻게 되나요? 감이 아니라 숫자에 맞춰 설계하고 싶습니다.
The p50 is fine. It's the p99 that would break the experience.p50은 괜찮습니다. 문제는 사용자 경험을 깨는 p99입니다.
That's roughly three times the cost for two points.2점 얻자고 비용이 대략 세 배입니다.
I don't think that trade is worth it, but that's a product call, not a technical one.그 교환은 아깝다고 보는데, 이건 기술 판단이 아니라 제품 판단입니다.
We could cache the common cases, which covers maybe sixty percent of traffic.흔한 케이스를 캐시하면 트래픽의 60퍼센트 정도는 덮입니다.
If we halve the context, we lose about a point and save about forty percent.컨텍스트를 절반으로 줄이면 1점쯤 잃고 40퍼센트쯤 아낍니다.
I can give you cheaper or faster. Cheaper and faster is a different model.더 싸게 또는 더 빠르게는 됩니다. 둘 다는 다른 모델 얘기입니다.
Those numbers are my estimate, not a measurement. I haven't load-tested this.이 숫자는 측정값이 아니라 제 추정입니다. 부하 테스트는 안 했습니다.

네 번째 문장이 이 절의 핵심입니다. that's a product call, not a technical one은 내 의견을 밝히면서 결정 권한은 상대에게 남겨 둡니다. 의견을 안 내는 것도 아니고 남의 결정을 대신하는 것도 아닌 자리를 만들어 줍니다.

마지막 문장은 세 조각 중 근거에 해당합니다. 추정치와 측정치를 구분하지 않고 말하면, 나중에 실측이 다르게 나왔을 때 신뢰가 깎입니다. 추정임을 밝히고 말한 추정은 틀려도 비용이 없습니다.

상황 6: 연구 일정과 제품 일정이 부딪칠 때

연구는 언제 결과가 나올지 모르고 제품은 날짜가 필요합니다. 이 충돌은 해결되지 않고 관리될 뿐이라, 관리 가능한 형태로 바꿔 말하는 것이 이 대화의 전부입니다.

표현
I can commit to a date for the experiment. I can't commit to a date for the result.실험 일정은 약속할 수 있는데 결과 일정은 못 합니다.
What I can give you is a decision date, not a delivery date.납기일 대신 판단 시점을 드릴 수 있습니다.
By the twentieth I'll know whether this approach is viable.20일이면 이 접근이 가능한지 아닌지는 압니다.
If we need a fixed date, we need a fallback that doesn't depend on this landing.날짜를 고정해야 한다면 이게 성공하는 데 의존하지 않는 대안이 필요합니다.
I'd rather give you a date I can actually hold.지킬 수 있는 날짜를 드리고 싶습니다.
Here's what I'd cut if the date can't move.날짜를 못 옮긴다면 제가 뺄 것은 이겁니다.
Two weeks gets us an answer. Six weeks might get us a good one.2주면 답은 나오고, 6주면 좋은 답이 나올 수도 있습니다.

decision datedelivery date를 구분하는 프레임이 여기서 가장 실용적입니다. 연구가 언제 끝날지는 모르지만, 언제쯤이면 계속할지 접을지 판단할 수 있는지는 대개 말할 수 있습니다. 그리고 제품 쪽이 실제로 필요한 것도 대부분 그것입니다.

등록에 대한 판단을 하나 붙이겠습니다. I'd rather give you a date I believe than a date you want to hear처럼 상대를 살짝 지목하는 변형이 돌아다니는데, 이건 팀에 따라 원칙적으로 들리기도 하고 수동공격적으로 들리기도 합니다. 상대를 문장에서 빼고 나에 대해서만 말하는 형태가 어느 팀에서든 안전합니다. 위 표의 다섯 번째 문장이 그 형태입니다.

상황 7: 비기술 이해관계자에게 불확실성 설명하기

이 자리에서 실패하는 방식은 두 가지입니다. 지나치게 단순화해서 사실이 아닌 말을 하거나, 정확하게 말하려다 아무것도 전달하지 못하거나.

세 가지 규칙이 대체로 잘 통합니다. 숫자는 하나만 주고, 전문 용어를 빼고, 실패 모드를 반드시 함께 말합니다.

표현
It's right about eight times out of ten. The question for us is what happens the other two.열 번 중 여덟 번쯤 맞습니다. 우리가 볼 문제는 나머지 두 번입니다.
Think of it like a weather forecast. Seventy percent is useful even though it isn't a promise.일기예보처럼 보시면 됩니다. 70퍼센트는 약속이 아니어도 쓸모가 있습니다.
It works well on the cases we've seen. We don't yet know how it behaves on the ones we haven't.본 적 있는 경우에는 잘 됩니다. 못 본 경우는 아직 모릅니다.
I don't want to give you a number that sounds more solid than it is.실제보다 단단하게 들리는 숫자를 드리고 싶지 않습니다.
We can make it more accurate or more predictable. Those are two different projects.더 정확하게 만들 수도 있고 더 예측 가능하게 만들 수도 있는데, 서로 다른 일입니다.
Short version: good enough to pilot, not good enough to run unattended.요약하면 시범 운영은 되고 무인 운영은 아직입니다.
The failure mode isn't that it breaks. It's that it's confidently wrong.문제는 멈추는 게 아니라 확신에 차서 틀린다는 겁니다.

마지막 문장은 이 청중에게 특히 중요합니다. 비기술 이해관계자가 가장 자주 놓치는 것이 틀리는 방식입니다. 실패하면 멈출 거라고 생각하고 계획을 짜는데, 언어 모델의 실패는 대개 그렇지 않습니다. 이걸 미리 말하지 않으면 나중에 놀라움이 사고가 됩니다.

빼야 할 단어의 예를 들면 이렇습니다. eval, harness, ablation, in-distribution, held-out, seed. 이 단어들이 나쁜 게 아니라 이 자리에서 안 통할 뿐입니다.

팀 안에서밖에서는 이렇게
The eval set is small.We only tested it on a few hundred examples.
It's out of distribution.It's the kind of input we didn't plan for.
We need another ablation.We need one more experiment to isolate the cause.
The judge is biased toward longer answers.Our automatic scoring prefers longer answers, which isn't always better.

상황 8: 아직 모른다고 말하되 다음 단계를 함께 주기

앞의 세 조각이 가장 압축적으로 필요한 순간입니다. 회의 중에 갑자기 질문이 들어오고, 답을 모르고, 방에는 여덟 명이 있습니다.

표현어떤 조각
I don't know yet. Here's what would tell us, and here's when.아직 모릅니다. 무엇으로 알 수 있고 언제 알 수 있는지 말씀드리겠습니다.경계 더하기 다음 단계
I don't have that number in front of me. Let me get it and follow up today.지금 그 숫자가 없습니다. 확인해서 오늘 중에 드리겠습니다.다음 단계
Honestly, I don't know. My best guess is around 60 percent, but I wouldn't plan around it.솔직히 모르겠습니다. 대략 60퍼센트로 추측하는데 이걸 근거로 계획하지는 마세요.근거 더하기 경계
I can give you a guess or a good answer. Which do you need right now?추측을 드릴까요, 정확한 답을 드릴까요? 지금 필요한 게 어느 쪽인가요?경계
That's a fair question and I don't want to guess at it. Can I come back to you tomorrow?좋은 질문인데 추측하고 싶지 않습니다. 내일 다시 말씀드려도 될까요?다음 단계
We'll know after the ablation. That's about two days of compute.실험 하나 돌리면 압니다. 컴퓨트로 이틀 정도입니다.다음 단계
I know the answer for English. I don't know it for Korean, and I'd expect it to be different.영어는 압니다. 한국어는 모르고, 다를 거라고 봅니다.경계

네 번째 문장이 특히 유용합니다. 즉답을 요구받았을 때 그 요구를 거절하지 않으면서 조건을 되돌려 주는 형태입니다. 상대가 추측이라도 필요하다고 하면 추측을 주면 되고, 정확한 답이 필요하다고 하면 시간을 얻습니다. 어느 쪽이든 내가 말한 것의 성격이 명확해집니다.

마지막 문장은 이 분야에서 자주 필요합니다. 영어 벤치마크 결과를 한국어나 일본어로 그대로 옮길 수 없는 경우가 많은데, 그 경계를 미리 그어 두면 나중에 놀랄 일이 줄어듭니다.

상황 9: 남의 실험 결과에 이견 제기하기

여기서의 원칙은 하나로 정리됩니다. 결론을 반박하지 말고 조건을 확인하십시오.

이유는 단순합니다. 조건 질문에는 답이 있고 공격에는 답이 없습니다. That's wrong에는 방어 말고 할 수 있는 게 없지만, What was the baseline?에는 답이 있습니다. 그리고 답하는 과정에서 대개 문제가 저절로 드러납니다.

표현
Before I push back, let me make sure I understand the setup.이견 내기 전에 설정부터 정확히 이해하고 싶습니다.
What was the baseline here?여기 베이스라인이 뭐였나요?
Was that on the same eval set as last time?지난번과 같은 평가 세트였나요?
How many seeds?시드는 몇 개였나요?
I might be misreading the table. Is that averaged or best-of-n?표를 잘못 읽은 걸 수도 있는데, 평균인가요 최고값인가요?
I'm not doubting the number. I'm trying to work out what it's comparable to.숫자를 의심하는 게 아니라 무엇과 비교 가능한지 보려는 겁니다.
That's interesting, because it contradicts what I saw last week. One of us has a bug.흥미롭네요, 지난주에 제가 본 것과 반대라서요. 둘 중 하나에 버그가 있습니다.
Can you say more about how you picked the threshold?임계값을 어떻게 고르셨는지 조금 더 설명해 주실 수 있나요?
Does this hold if we swap the grader?심사자를 바꿔도 같은 결과가 나오나요?

One of us has a bug는 이 분야에서 특히 잘 작동하는 문장입니다. 두 사람의 결과가 다를 때 누가 틀렸는지를 미정으로 남기면서 원인이 있다는 사실만 확정합니다. 상대가 방어할 이유가 없어지므로 대화가 바로 기술적인 쪽으로 넘어갑니다.

Before I push back도 유용합니다. 이견이 온다는 것을 미리 알려서 상대가 준비할 시간을 줍니다. 다만 이 정도의 예고가 필요한지는 팀에 따라 다릅니다. 직설적인 토론이 기본값인 팀에서는 불필요한 완충으로 느껴질 수 있고, 반대로 조심스러운 팀에서는 이 한 문장이 갈등을 막습니다. 어느 쪽인지 모르겠으면 붙이는 편이 손해가 적습니다.

상황 10: 재현이 안 될 때

재현 실패는 거의 항상 누군가의 실수인데, 그 누군가가 누구인지는 아직 모르는 상태입니다. 먼저 내 쪽을 의심하는 문장으로 시작하면 대화가 조사로 흐르고, 상대 쪽을 먼저 의심하면 대화가 방어로 흐릅니다.

표현
I couldn't reproduce this. That's probably on my end.재현이 안 됩니다. 아마 제 쪽 문제일 겁니다.
Can we walk through your setup?설정을 같이 짚어 볼 수 있을까요?
Same commit, same data, different number, which usually means something isn't pinned.같은 커밋 같은 데이터인데 숫자가 다릅니다. 보통 뭔가 고정이 안 됐다는 뜻이죠.
What seed did you use?시드는 뭘 쓰셨나요?
Is there anything in your environment that isn't in the repo?저장소에 없는 환경 설정이 있을까요?
Could you share the exact command you ran?실행하신 명령을 그대로 공유해 주실 수 있나요?
I'd like us to get to the same number before we decide anything on top of it.이걸 근거로 뭘 결정하기 전에 같은 숫자에 도달하고 싶습니다.
I'm getting 74, you're getting 81. Let's find the difference before we argue about the meaning.저는 74, 그쪽은 81입니다. 해석을 다투기 전에 차이부터 찾죠.

That's probably on my end에 대해서는 등록 판단을 하나 덧붙여야 합니다. 이건 상대의 체면을 지키면서 문제를 제기하는 장치이고, 나중에 상대 쪽 문제로 밝혀져도 아무도 손해를 보지 않습니다. 다만 이 표현을 습관적으로 남발하면 실제로 자신 없어 보일 수 있습니다. 자기 비하를 낮게 평가하는 팀에서는 특히 그렇습니다. 재현 실패처럼 원인이 정말로 불확실한 상황에서는 쓰고, 명백히 상대 코드의 문제일 때는 쓰지 않는 편이 좋습니다.

헤지의 세기 눈금

불확실성 표현을 잘 쓰려면 세기를 조절할 수 있어야 합니다. 아래는 대략의 눈금입니다. 정확한 확률로 매핑되는 것은 아니고 관행적인 인상에 가깝습니다.

표현대략의 확신적합한 상황
I'm confident that...높음여러 번 확인한 것
I'd expect...중상근거는 있지만 직접 재지는 않음
It looks like...관찰했지만 확인 전
My best guess is...추측임을 명시
I suspect...중하근거가 약함
I could be wrong, but...반대 의견을 조심스럽게
I genuinely don't know.없음정직한 종료

여기에 하나의 규칙을 붙이겠습니다. 한 주장에 헤지는 하나면 충분합니다. I could be wrong, but I think maybe it might be the tokenizer처럼 네 개를 쌓으면 주장 자체가 사라지고, 듣는 사람은 내용 대신 화자의 자신감 없음을 기억합니다.

다만 이 판단은 팀 문화에 따라 달라집니다. 완곡함이 기본값인 조직에서는 헤지 두 개가 예의의 하한선일 수 있습니다. 반대로 직설적인 엔지니어링 팀에서는 하나도 많게 느껴집니다. 회의 두세 번만 관찰하면 그 팀의 기본값이 보입니다. 가장 빠른 방법은 그 팀에서 신뢰받는 시니어 한 명의 말투를 기준선으로 잡는 것입니다.

미국식과 영국식이 갈리는 지점

회의에서 실제로 오해를 만드는 몇 가지가 있습니다.

table 동사는 뜻이 정반대입니다. 미국에서 Let's table this는 나중으로 미루자는 뜻이고, 영국에서 to table an item은 안건으로 올리자는 뜻입니다. 회의 진행 중에 정확히 반대로 알아들을 수 있는 단어라 위험합니다. 양쪽이 섞인 회의에서는 그냥 Let's come back to this laterLet's add this to the agenda로 풀어 말하는 게 안전합니다.

quite의 세기가 다릅니다. 미국에서 quite good은 꽤 좋다는 칭찬에 가깝고, 영국에서 quite good은 나쁘지는 않다는 정도의 미지근한 평가로 읽히는 경우가 많습니다. 평가 결과를 논평할 때 오해가 나기 쉬운 단어입니다. 확실하게 말하고 싶으면 quite를 빼고 숫자를 말하는 편이 낫습니다.

momentarily도 갈립니다. 미국에서는 곧이라는 뜻으로, 영국에서는 잠깐 동안이라는 뜻으로 쓰이는 경향이 있습니다. in a momentbriefly로 바꾸면 오해가 없습니다.

Sorry의 무게가 다릅니다. 영국에서 Sorry는 사과보다 대화의 완충재에 가깝게 훨씬 자주 쓰입니다. 영국 동료가 사과를 남발하는 것처럼 보여도 그건 사과가 아닙니다. 반대로 미국 회사 문화에서는 지연을 사과하는 대신 Thanks for your patience로 바꾸는 것을 선호하는 경우가 많습니다. 다만 이건 회사 문화 차이가 나라 차이만큼 크므로 일반화하지는 않겠습니다.

하지 않는 편이 나은 것들

결과를 부풀리는 문장. 이 글에는 그런 예시가 없고 앞으로도 넣지 않겠습니다. 이유는 윤리적인 것만이 아닙니다. AI 개발에서는 부풀린 숫자가 대개 그 분기 안에 검증됩니다. 다른 팀이 같은 평가를 돌리고, 제품에 붙이고, 사용자가 씁니다. 부풀린 문장의 유효기간은 그 숫자가 재현되기까지의 시간뿐입니다.

회귀를 다음 보고까지 미루기. 위와 같은 이유로 결국 드러나고, 늦게 드러날수록 그 위에 쌓인 결정이 많아집니다.

It should work. 엔지니어링 영어에서 이 문장은 사실상 테스트를 안 했다는 신호로 읽힙니다. should는 보고가 아니라 예측이기 때문입니다. 대신 무엇을 테스트했고 무엇을 안 했는지 말하는 편이 낫습니다. I tested the happy path and the empty input. I haven't tested concurrency.

단독으로 쓰는 I'm not sure. 내부 상태만 보고하고 상대에게 아무것도 남기지 않습니다. 뒤에 경계나 다음 단계 중 하나만 붙어도 완전히 달라집니다.

조건 없는 숫자. 82라고만 말하면 회의록에는 82만 남고, 그 82는 몇 주 뒤에 근거로 인용됩니다. 어떤 데이터와 몇 번 실행이었는지는 그때 아무도 기억하지 못합니다.

질문으로 끝내는 불확실성 보고. Let me know if you have any questions.로 끝내면 다음 행동을 상대에게 미루는 셈입니다. 불확실성을 보고할 때는 내가 다음 단계를 제시하고 끝내는 편이 낫습니다.

남의 실험을 결론으로 반박하기. 조건 확인이 먼저입니다. 조건을 확인하고 나서도 이견이 남으면 그때 결론을 말하면 되고, 그 시점의 이견은 훨씬 강합니다.

실전 흐름: 주간 모델 리뷰

앞의 조각들이 실제 회의에서 어떻게 붙는지 보겠습니다.

PM:  So the new checkpoint is better, right? Can we ship Friday?

나:  Two caveats before the number.
     It is a single run, and it is on our internal set,
     so it is not comparable to the vendor numbers.
     With that said, we are up two points.
     Two points is inside the noise for this set,
     so I would not call it a win yet.

PM:  Okay. When will you know?

나:  What I can give you is a decision date, not a delivery date.
     Three more seeds is about two days of compute.
     By Wednesday I will know whether the gain is real.

PM:  And if it is real?

나:  Then there is a second question. I read twenty outputs by hand
     and some of them got longer without getting better.
     The metric moved, but I am not sure the task did.
     I would want a human check on a small sample before we ship.

동료: I saw the opposite last week - my run showed a clear gain.

나:  That is interesting, because it contradicts mine.
     One of us has a bug. Before I push back,
     what was your baseline, and how many seeds?

동료: Baseline was the March checkpoint. One seed.

나:  Okay, mine is against last month. That could be the whole difference.
     Can we get to the same number before we decide anything on top of it?

PM:  What do I tell leadership on Thursday?

나:  Short version: it looks promising, we do not know yet if it is real,
     and we will know by Wednesday.
     The failure mode to watch is longer answers that are not better ones.
     No decision needed from them this week.

이 대화에서 나는 좋은 소식을 한 번도 확정하지 않았습니다. 그런데 회의는 막히지 않았습니다. 매번 경계를 긋고, 근거를 밝히고, 다음 단계와 시점을 함께 줬기 때문입니다.

오늘 바로 해볼 것

표현을 전부 외울 필요는 없습니다. 세 개만 자동으로 나오게 만드세요.

  1. 한계를 앞에 두는 한 문장: Two caveats before I give you the number. 숫자를 말하기 전에 조건을 먼저 놓는 습관이 이 글 전체에서 가장 값이 큽니다.
  2. 모른다를 완성하는 한 문장: I don't know yet. Here's what would tell us, and here's when. 경계와 다음 단계가 한 문장에 들어 있습니다.
  3. 이견을 조건 확인으로 바꾸는 한 문장: I'm not doubting the number. I'm trying to work out what it's comparable to.

그리고 다음 회의에서 한 번만 확인해 보세요. 내가 말한 숫자에 조건을 붙였는가. 붙이지 않았다면, 그 숫자는 지금 회의실에서 내가 아는 것보다 단단한 것으로 유통되고 있습니다.

이어서 읽기

English for AI Engineering Meetings: Saying You Do Not Know Without Losing Credibility

Introduction: the thing this field says most

Skim transcripts of AI team meetings and one thing stands out. The more senior the speaker, the fewer sentences of certainty they use. What they use instead are sentences that draw the line between what is known and what is not. Those are two completely different abilities.

Most English-for-meetings material teaches the opposite. Speak with confidence, it says; cut down the vague expressions. In a general office environment that may be sound advice. In AI development, following it literally costs you credibility instead.

The reason is that every number in this field is conditional. Any eval score depends on which harness it was run on, which grader scored it, and which data it was measured against. As How to Read Leaderboards and Benchmarks sets out, self-reported figures, harness differences and grader bias are not exceptions but the baseline conditions. In a field like that, someone who states a number with no conditions attached does not look confident; they look like someone who does not know the conditions.

So this guide has one subject. The English an AI meeting needs most often is not the language of certainty but the language that expresses uncertainty credibly. Being able to say you do not know without sounding incompetent is the core meeting skill in this field.

Why saying you do not know, early, is a credit asset

First, why this is not a loss. If you do not convince yourself of the logic, the sentence will not come out in an actual meeting.

Someone who states uncertainty accurately can be believed when they state certainty. Conversely, someone who is confident every time gets a discount applied to every number they produce from the first time they are wrong. That is the real cost. The next time you carry in a result you are genuinely sure of, the certainty will not land.

The second reason comes from the structure of the field. A hidden regression always surfaces. Someone else runs the same eval, another team wires up the same model, and eventually a user sees it. The later it surfaces, the more expensive it is, because other decisions have already been stacked on top of it.

The third is time. Waiting to report until you have the cause fully identified looks conscientious, but in the meantime the team plans its schedule on a false premise. A decision you could not make for lack of information and a decision already made on wrong information have very different recovery costs.

There are no sentences in this guide for making a result look good. They were left out on purpose. Sentences like that work once or twice and then stop working, and by the time they stop, the career cost has already been incurred.

The three pieces of credible uncertainty

Saying you do not know sounds incompetent not because you do not know. It sounds incompetent because you did not say how far the not-knowing extends. Compare two sentences from the same situation.

[the form that loses trust]
I'm not sure. I'll look into it.

[the form that earns trust]
I don't know yet whether it's the model or the harness.
What I do know is that it started after Tuesday's deploy,
and it only shows up on the long-context slice.
I'm bisecting the commits and I'll have an answer Thursday.

The second one still contains the words for not knowing. Only the amount of information differs. The three pieces that went into it are the spine of this guide.

PieceWhat it doesThe shape it takes in English
BoundaryDraws the line between what you know and what you do notI don't know whether A or B. What I do know is C.
BasisStates where what you currently know came fromThis is from a single run on our internal set.
Next stepGives the action and the date that will narrow the boundaryI'm running X. I'll have an answer by Thursday.

Drop the boundary and it sounds incompetent. You have reported nothing but a state of not knowing, so there is nothing the listener can do with it.

Drop the basis and it sounds light. The same score of 82 is a completely different sentence depending on whether it is one run or an average of five, and that difference has vanished.

Drop the next step and it sounds evasive. This is the piece most often missing. Go silent after saying you do not know and the other person has to construct the next action, which means they pay the price for your not knowing.

Every situational phrase below is a variation on these three pieces. Training your eye to see which piece is missing lasts longer than memorising the phrases.

Situation 1: presenting eval results with their limits attached

The most frequent situation, and the one most often got wrong. The common mistake is to give the number first and attach the limits afterwards. Then the limits sound like excuses. State the limits first and then give the number, and the same content reads as care.

PhraseWhat it means
Two caveats before I give you the number.Two conditions before the number.
This is on our internal set, so it's not comparable to the published numbers.Internal set, so not directly comparable to published figures.
The headline number is 82. I'd read that as a range, not a point.The headline is 82, best read as a range rather than a point.
This eval doesn't capture multi-turn behavior at all.This eval catches nothing about multi-turn behaviour.
What this measures is retrieval quality. What it doesn't measure is whether the answer is useful.It measures retrieval quality, not whether the answer is useful.
I'd treat this as directional rather than decisive.Directional guidance, not a number to conclude from.
The interval overlaps with the baseline, so I wouldn't call it a win yet.The interval overlaps the baseline, so it is early to call it a win.
We're up two points, and two points is inside the noise on this set.Two points up, and two points is inside the noise on this set.
I ran it once. I'd want a second seed before we plan around it.One run. I want a second seed before anyone plans around it.

There are four conditions worth attaching to a number: which data, how many runs, which grader, compared to what. You do not have to say all four, but say none of them and the number circulates in the room as something firmer than it is.

Phrase that attaches a conditionWhat it means
That's on the held-out set, not the one we tuned on.Held-out set, not the set we tuned on.
Three runs, and that's the mean. The spread was about a point and a half.Mean of three runs, spread about a point and a half.
The grader is the same model family, which I'd flag as a possible bias.The grader is from the same model family, so I flag the possible bias.
Compared to last month's checkpoint, not to the vendor's number.Compared with last month's checkpoint, not with the vendor's figure.

Situation 2: a regression whose cause you do not know yet

The biggest temptation here is to speak only after you have found the cause. That is choosing to save one day and burn three days of the whole team's time.

PhraseWhat it meansTone
Heads up: we've got a regression on the summarization eval.Heads up, there is a regression on the summarisation eval.Casual, team channel
I want to flag a regression before it gets buried in the sprint update.Flagging a regression before the sprint update buries it.Formal, documents or senior people
I don't have a cause yet.I have not found the cause yet.Neutral fact
I'd rather flag this early than wait until I have the full picture.I chose to flag early rather than wait for the full picture.Explaining the stance
It started somewhere between Tuesday and Thursday. I'm bisecting now.It began between Tuesday and Thursday; I am bisecting.Boundary and next step
I can't rule out that it's the harness rather than the model.I cannot rule out the harness rather than the model.Honest range
No action needed from you yet. I'll have more by Thursday.Nothing for you to do yet; more by Thursday.Removes the listener's burden
I'd rather over-report this than sit on it.I would rather over-report than sit on it.Stating a principle

Heads up is close to standard in engineering conversation, but it is colloquial. In documents that go to executives or in a formal email, I want to flag fits better. That judgement depends on the company culture. On a team that writes documents in a casual tone, there is no need to change it.

No action needed from you yet is short but does a lot of work. The first reaction of someone who has heard bad news is usually to wonder what they have to do, and answering that question first makes the report land far more lightly.

Situation 3: the metric went up but the actual quality did not

This may be the hardest thing to say in this field, because it is the moment you cancel your own good news. But if it goes unsaid, the whole team spends a quarter heading the wrong way.

PhraseWhat it means
The metric moved, but I don't think the task did.The metric moved; I do not think the task did.
The score went up and the samples got worse. One of those is lying.Score up, samples worse. One of the two is lying.
This looks like the grader rewarding length rather than quality.It looks like the grader is scoring length, not quality.
I think we optimized the scorer, not the task.I think we optimised the scorer rather than the task.
Before we call this a win, I'd like to read twenty outputs by hand.Before calling it a win, I want to read twenty outputs by hand.
I'd call this a metric win, not a product win.A win on the metric, not a win for the product.
The gain disappears when I change the prompt format, which makes me suspicious.The gain vanishes when I change the prompt format, which makes me suspicious.

This situation already has a name: reward hacking. It points at the state where the metric genuinely went up while the task failed, and the key part is that it is not a system breaking but the exact result of optimising the goal we defined.

On English-speaking ML teams, shorthand for Goodhart's law works, as in That might be Goodhart. But this is shorthand that only travels inside an ML team, so in a meeting with other departments in the room it is safer to say it in full. Something like We may be optimizing the measure instead of the thing we care about gets through to anyone.

Situation 4: data quality and disagreements about labelling

Labelling arguments run long mostly because people are arguing about different things. You think you are arguing about whether the model is wrong, and often you are actually arguing about the label definition. The sentence that names what is being argued about saves the most time of anything here.

PhraseWhat it means
I think we're disagreeing about the label definition, not about the model.I think we disagree about the label definition, not the model.
Can we look at ten examples together?Shall we look at ten examples together?
What's our inter-annotator agreement on this slice?What is inter-annotator agreement on this slice?
I'd want to re-check the gold labels before we treat this as model error.I want to recheck the gold labels before calling this model error.
Some of these look mislabeled to me. Can I flag a batch for review?Some look mislabelled to me. May I flag a batch for review?
I'm not questioning the annotators. I'm questioning whether the instructions were answerable.I am not questioning the annotators but whether the instructions were answerable.
If two careful people label this differently, we can't expect the model to pick one.If two careful people label it differently, the model cannot be expected to pick one.

The sixth sentence carries a judgement about register. Raise a label quality problem and it easily sounds like criticism of the person or team who did the work. Move the target of the criticism from the people to the instructions and the same objection lands far more safely. And usually it is also the truth. What really makes labels wobble is more often ambiguity in the definition than a lack of diligence in the annotators.

The seventh sentence is strong as an argument too. It points out that demanding of a model a criterion two people cannot agree on is an impossible demand, which is hard to rebut.

Situation 5: the cost and latency trade-off

The mistake engineers often make in this conversation is mixing technical judgements with business judgements. Separate the two and the weight of what you say goes up rather than down. The form is definite inside your own domain, information only inside someone else's.

PhraseWhat it means
What's our latency budget? I'd rather design to a number than to a feeling.What is the latency budget? I would rather design to a number than a feeling.
The p50 is fine. It's the p99 that would break the experience.The p50 is fine; the p99 is what would break the experience.
That's roughly three times the cost for two points.Roughly three times the cost for two points.
I don't think that trade is worth it, but that's a product call, not a technical one.I do not think the trade is worth it, but that is a product call, not a technical one.
We could cache the common cases, which covers maybe sixty percent of traffic.Caching the common cases covers maybe sixty percent of traffic.
If we halve the context, we lose about a point and save about forty percent.Halving the context loses about a point and saves about forty percent.
I can give you cheaper or faster. Cheaper and faster is a different model.Cheaper or faster, yes. Cheaper and faster is a different model.
Those numbers are my estimate, not a measurement. I haven't load-tested this.Those are my estimates, not measurements. I have not load-tested it.

The fourth sentence is the core of this section. that's a product call, not a technical one states your opinion while leaving the decision authority with the other side. It builds a position that is neither withholding an opinion nor making someone else's decision for them.

The last sentence corresponds to the basis, of the three pieces. Speak without distinguishing estimates from measurements and your credibility takes the hit later, when the measured figure comes out different. An estimate given as an estimate costs nothing when it turns out wrong.

Situation 6: when the research schedule collides with the product schedule

Research does not know when a result will arrive and product needs a date. This collision is not resolved, only managed, so turning it into a manageable shape is the whole of this conversation.

PhraseWhat it means
I can commit to a date for the experiment. I can't commit to a date for the result.I can commit to a date for the experiment, not for the result.
What I can give you is a decision date, not a delivery date.I can give you a decision date instead of a delivery date.
By the twentieth I'll know whether this approach is viable.By the twentieth I will know whether the approach is viable.
If we need a fixed date, we need a fallback that doesn't depend on this landing.If the date must be fixed, we need a fallback that does not depend on this landing.
I'd rather give you a date I can actually hold.I would rather give you a date I can actually hold.
Here's what I'd cut if the date can't move.Here is what I would cut if the date cannot move.
Two weeks gets us an answer. Six weeks might get us a good one.Two weeks gets an answer; six weeks might get a good one.

The frame that separates a decision date from a delivery date is the most practical thing here. You do not know when the research will finish, but you can usually say when you will be able to judge whether to continue or stop. And that is mostly what the product side actually needs.

One judgement about register to add. A variant that points slightly at the other person circulates, as in I'd rather give you a date I believe than a date you want to hear, and depending on the team it sounds principled or passive-aggressive. Taking the other person out of the sentence and speaking only about yourself is safe on any team. The fifth sentence in the table above is that form.

Situation 7: explaining uncertainty to non-technical stakeholders

There are two ways to fail in this room. Simplify so hard that you say something untrue, or aim for accuracy and communicate nothing at all.

Three rules generally work well. Give one number only, strip the jargon, and always state the failure mode alongside it.

PhraseWhat it means
It's right about eight times out of ten. The question for us is what happens the other two.Right about eight times in ten; our question is the other two.
Think of it like a weather forecast. Seventy percent is useful even though it isn't a promise.Treat it like a forecast. Seventy percent is useful even without being a promise.
It works well on the cases we've seen. We don't yet know how it behaves on the ones we haven't.It works well on cases we have seen; we do not yet know about the ones we have not.
I don't want to give you a number that sounds more solid than it is.I do not want to hand you a number that sounds firmer than it is.
We can make it more accurate or more predictable. Those are two different projects.We can make it more accurate or more predictable, and those are different projects.
Short version: good enough to pilot, not good enough to run unattended.Short version: fine for a pilot, not for running unattended.
The failure mode isn't that it breaks. It's that it's confidently wrong.The failure is not that it stops but that it is confidently wrong.

The last sentence matters especially for this audience. What non-technical stakeholders most often miss is the way it goes wrong. They plan on the assumption that failure means stopping, and failure in a language model usually is not that. Say it in advance or the surprise becomes an incident later.

Examples of words to strip out: eval, harness, ablation, in-distribution, held-out, seed. There is nothing wrong with these words; they simply do not travel in this room.

Inside the teamOutside, say this
The eval set is small.We only tested it on a few hundred examples.
It's out of distribution.It's the kind of input we didn't plan for.
We need another ablation.We need one more experiment to isolate the cause.
The judge is biased toward longer answers.Our automatic scoring prefers longer answers, which isn't always better.

Situation 8: saying you do not know yet, with the next step attached

This is the moment where the three pieces are needed in their most compressed form. A question lands mid-meeting, you do not have the answer, and there are eight people in the room.

PhraseWhat it meansWhich piece
I don't know yet. Here's what would tell us, and here's when.I do not know yet; here is what would tell us and when.Boundary plus next step
I don't have that number in front of me. Let me get it and follow up today.I do not have that number here. I will get it and come back today.Next step
Honestly, I don't know. My best guess is around 60 percent, but I wouldn't plan around it.Honestly I do not know. My guess is around 60 percent, but do not plan on it.Basis plus boundary
I can give you a guess or a good answer. Which do you need right now?A guess or an accurate answer — which do you need right now?Boundary
That's a fair question and I don't want to guess at it. Can I come back to you tomorrow?Fair question, and I do not want to guess. May I come back tomorrow?Next step
We'll know after the ablation. That's about two days of compute.One experiment tells us. About two days of compute.Next step
I know the answer for English. I don't know it for Korean, and I'd expect it to be different.I know it for English, not for Korean, and I expect it to differ.Boundary

The fourth sentence is especially useful. It is the form that hands the condition back without refusing the demand for an immediate answer. If they say a guess will do, give the guess; if they say they need an accurate answer, you have bought time. Either way, the nature of what you said is now explicit.

The last sentence comes up often in this field. English benchmark results frequently cannot be carried across to Korean or Japanese unchanged, and drawing that boundary in advance removes a lot of later surprise.

Situation 9: pushing back on someone else's experiment

The principle here condenses into one line. Do not rebut the conclusion; verify the conditions.

The reason is simple. Condition questions have answers; attacks do not. There is nothing to do with That's wrong except defend, whereas What was the baseline? has an answer. And in the course of answering, the problem usually reveals itself.

PhraseWhat it means
Before I push back, let me make sure I understand the setup.Before objecting, let me make sure I have the setup right.
What was the baseline here?What was the baseline here?
Was that on the same eval set as last time?Was that the same eval set as last time?
How many seeds?How many seeds?
I might be misreading the table. Is that averaged or best-of-n?I may be misreading the table. Is that an average or best-of-n?
I'm not doubting the number. I'm trying to work out what it's comparable to.I am not doubting the number; I am working out what it is comparable to.
That's interesting, because it contradicts what I saw last week. One of us has a bug.Interesting, because it contradicts what I saw last week. One of us has a bug.
Can you say more about how you picked the threshold?Could you say more about how the threshold was picked?
Does this hold if we swap the grader?Does this hold if the grader is swapped?

One of us has a bug works particularly well in this field. When two people's results differ, it leaves who is wrong undecided while fixing the fact that there is a cause. The other person has no reason to defend, so the conversation moves straight to the technical side.

Before I push back is useful too. It tells the other person an objection is coming, which gives them time to prepare. Whether that much warning is needed depends on the team. On a team where blunt debate is the default it can feel like unnecessary cushioning; on a cautious team this single sentence prevents conflict. If you cannot tell which one you are on, attaching it costs less.

Situation 10: when it will not reproduce

A failure to reproduce is nearly always somebody's mistake, and which somebody is not yet known. Start with a sentence that suspects your own side and the conversation flows into investigation; suspect their side first and it flows into defence.

PhraseWhat it means
I couldn't reproduce this. That's probably on my end.I could not reproduce it. It is probably on my side.
Can we walk through your setup?Could we walk through your setup together?
Same commit, same data, different number, which usually means something isn't pinned.Same commit, same data, different number, which usually means something is unpinned.
What seed did you use?Which seed did you use?
Is there anything in your environment that isn't in the repo?Is there anything in your environment that is not in the repo?
Could you share the exact command you ran?Could you share the exact command you ran?
I'd like us to get to the same number before we decide anything on top of it.I want us on the same number before we decide anything on top of it.
I'm getting 74, you're getting 81. Let's find the difference before we argue about the meaning.I get 74, you get 81. Let us find the difference before arguing about meaning.

One judgement about register has to be attached to That's probably on my end. It is a device for raising a problem while protecting the other person's face, and nobody loses anything if it later turns out to be their side. Overuse it as a habit, though, and you can genuinely start to look unconfident. That is especially true on teams that mark self-deprecation down. Use it where the cause really is uncertain, as with a failure to reproduce, and not where the problem is plainly in their code.

The strength scale for hedging

Using uncertainty language well means being able to adjust the strength. Below is a rough scale. It does not map onto exact probabilities; it is closer to a conventional impression.

PhraseRoughly this much confidenceWhere it fits
I'm confident that...HighSomething checked several times
I'd expect...Medium-highGrounds exist but you did not measure it yourself
It looks like...MediumObserved but not confirmed
My best guess is...MediumExplicitly a guess
I suspect...Medium-lowWeak grounds
I could be wrong, but...LowA dissenting view offered carefully
I genuinely don't know.NoneAn honest full stop

One rule goes on top of this. One hedge per claim is enough. Stack four, as in I could be wrong, but I think maybe it might be the tokenizer, and the claim itself disappears; what the listener remembers is the speaker's lack of confidence rather than the content.

That judgement shifts with team culture, though. In an organisation where indirectness is the default, two hedges may be the floor of politeness. On a blunt engineering team, even one feels like a lot. Two or three meetings of observation is enough to see a team's default. The fastest method is to take one trusted senior person on that team as your baseline.

Where American and British usage split

A few of these genuinely create misunderstandings in meetings.

The verb table means the exact opposite. In the US, Let's table this means to postpone it; in the UK, to table an item means to put it on the agenda. It is dangerous precisely because it can be understood backwards while a meeting is running. In a meeting with both sides in the room, it is safer to say it in full: Let's come back to this later or Let's add this to the agenda.

The strength of quite differs. In the US quite good is close to a compliment meaning fairly good; in the UK quite good is often read as the lukewarm assessment of not bad. It is an easy word to be misread on when commenting on eval results. If you want to be unambiguous, drop quite and give the number.

momentarily splits too. In the US it tends to mean soon; in the UK it tends to mean for a moment. Swapping in in a moment or briefly removes the ambiguity.

The weight of Sorry differs. In the UK, Sorry is used far more often as conversational padding than as an apology. When a British colleague seems to be apologising constantly, that is not an apology. In the other direction, American company culture often prefers replacing an apology for a delay with Thanks for your patience. That said, company culture varies as much as country here, so I will not generalise it.

Things better left undone

Sentences that inflate the result. There are no examples of that in this guide and there will not be. The reason is not only ethical. In AI development, an inflated number usually gets checked within the same quarter. Another team runs the same eval, it gets wired into a product, and users use it. The shelf life of an inflated sentence is only the time it takes for that number to be reproduced.

Holding a regression until the next report. For the same reason it surfaces in the end, and the later it surfaces the more decisions have been stacked on top of it.

It should work. In engineering English this sentence reads, in effect, as a signal that it was not tested. should is a prediction rather than a report. Say what you tested and what you did not instead: I tested the happy path and the empty input. I haven't tested concurrency.

I'm not sure. used on its own. It reports an internal state and leaves the other person with nothing. Attach just one of the boundary or the next step and it changes completely.

Numbers with no conditions. Say only 82 and the minutes record only 82, and that 82 gets cited as evidence weeks later. Which data it was and how many runs it was, nobody remembers by then.

An uncertainty report that ends with a question. Finish with Let me know if you have any questions. and you have handed the next action to the other person. When reporting uncertainty, it is better to propose the next step yourself and stop there.

Rebutting someone else's experiment with a conclusion. Verifying the conditions comes first. If an objection remains after you have verified them, state the conclusion then, and at that point the objection is far stronger.

A worked example: the weekly model review

Here is how the pieces above attach to each other in a real meeting.

PM:   So the new checkpoint is better, right? Can we ship Friday?

Me:   Two caveats before the number.
      It is a single run, and it is on our internal set,
      so it is not comparable to the vendor numbers.
      With that said, we are up two points.
      Two points is inside the noise for this set,
      so I would not call it a win yet.

PM:   Okay. When will you know?

Me:   What I can give you is a decision date, not a delivery date.
      Three more seeds is about two days of compute.
      By Wednesday I will know whether the gain is real.

PM:   And if it is real?

Me:   Then there is a second question. I read twenty outputs by hand
      and some of them got longer without getting better.
      The metric moved, but I am not sure the task did.
      I would want a human check on a small sample before we ship.

Them: I saw the opposite last week - my run showed a clear gain.

Me:   That is interesting, because it contradicts mine.
      One of us has a bug. Before I push back,
      what was your baseline, and how many seeds?

Them: Baseline was the March checkpoint. One seed.

Me:   Okay, mine is against last month. That could be the whole difference.
      Can we get to the same number before we decide anything on top of it?

PM:   What do I tell leadership on Thursday?

Me:   Short version: it looks promising, we do not know yet if it is real,
      and we will know by Wednesday.
      The failure mode to watch is longer answers that are not better ones.
      No decision needed from them this week.

In that conversation I never once confirmed the good news. And yet the meeting did not stall. Because every time, a boundary was drawn, a basis was stated, and a next step and a date were given alongside it.

Try this today

You do not need to memorise all the phrases. Make three of them automatic.

  1. One sentence that puts the limits first: Two caveats before I give you the number. The habit of laying the conditions down before the number is the highest-value thing in this whole guide.
  2. One sentence that completes an I-do-not-know: I don't know yet. Here's what would tell us, and here's when. The boundary and the next step are both inside one sentence.
  3. One sentence that turns an objection into a check on conditions: I'm not doubting the number. I'm trying to work out what it's comparable to.

And check one thing in your next meeting. Did I attach conditions to the numbers I said? If you did not, that number is right now circulating in the room as something firmer than what you actually know.