Split View: AI가 쓴 글이 무너지는 지점들 — 여섯 가지 실패와 각각의 가드레일

Where AI-Written Posts Fall Apart — Six Failure Modes and a Guardrail for Each

• Introduction — The Sentences Are Smooth, the Post Is Wrong
• The Failure Modes at a Glance — A Map Sorted by Detectability
• Fabricated Sources and Quotes — Perfect Format, Missing Substance
• Stale Information — Instead of Saying It Doesn't Know, the Model States the Past in the Present Tense
• Stretched-Out Paragraphs and Unsupported Assertions — Two Sentence-Level Failures
  • A Paragraph That Stretches the Same Point
  • A Sentence That Asserts Without Evidence
• Thin Content and Repeated Phrasing — What Readers and Search Actually Penalize
• Plagiarism and Copyright — How Closely Does the Output Resemble the Source
• Closing — The Point of a Guardrail Isn't Passing, It's Deciding Where to Look
• References

Introduction — The Sentences Are Smooth, the Post Is Wrong

A bad post written by a human and a bad post written by AI are bad in different ways. A human-written post usually falls apart at the sentence level first. There's an awkward construction, the flow breaks, and reading it you can quickly point to where it's off. An AI-written post is the opposite. The sentences stay smooth all the way through, the paragraphs are balanced, and the tone is confident from start to finish. And yet the quoted sentence isn't on that page, three paragraphs say the same thing, and it asserts things with no evidence given.

This gap matters in practice because we verify writing using smoothness as our cue. We have a habit of passing something if nothing snags us while reading, and AI output has none of that signal from the start. So "I read it" doesn't count as verification — the only thing that does is a checklist of exactly what to look for, decided in advance.

This post is that checklist. It splits six failure modes into how to detect each and how to fix it, and separates which parts you can hand to a machine and which a human has to do to the end. Where to place these checks within the pipeline is covered in the post on the full pipeline; here we look at the checks themselves.

The Failure Modes at a Glance — A Map Sorted by Detectability

Let's put the six into a table first. It's sorted not by severity but by how much a machine can catch, because that's the information you actually need when designing a guardrail.

What you should read is the relationship between the two right-hand columns. Even an item with high automated detection still leaves something for a human to look at, and what's left is always judgment. A machine can tell you "this quote isn't in the source" — it cannot tell you "this quote flips the meaning of the source." Blur that boundary when building a guardrail, and the illusion creeps into the pipeline that passing a check means everything's fine.

Fabricated Sources and Quotes — Perfect Format, Missing Substance

This is the most common failure mode, and the most damaging one. A model is very good at generating things with a regular format — URLs, paper titles, author names, document numbers. A regular format means it's easy to learn, which also means it's easy to generate a new, plausible-looking combination. So a paper that doesn't exist shows up in a precise citation format, and a URL with a dead anchor arrives with a perfectly natural-looking path structure.

There's one misconception to clear up here. Attaching a search feature doesn't make this problem go away. Even when the model has actually pulled up a real document via search, a sentence that isn't in the source can end up sitting where the quote goes, during the summarizing step. What actually shows up most often isn't a total fabrication — it's several sentences from the source blended into one. The content is mostly right, but that exact sentence isn't in the source. The moment you put quotation marks around it, it's an error.

Detection, fortunately, is something a machine does well. Require a verbatim quote and cross-check whether that string exists in the source. The cross-check script is included in the pipeline post, so here we'll just cover how to handle the result.

• The quote isn't in the source: Remove the quotation marks and turn it into a paraphrase, or discard the claim entirely. Don't go hunting for a similar-sounding sentence to patch in — that's not verification, it's after-the-fact justification.
• The link doesn't open: Treat that claim as unsupported. Searching for a different link to attach is a separate task, unrelated to whether the original claim was correct.
• It's not a primary source: Trace back from the summary article to the original. Along the way you'll often find that the original actually says something different.

The single highest-return habit on the fixing side is reversing the order — write the claim first, look for the source second, becomes look at the source first, write only from what's in it. There's no room left to fabricate anything.

Stale Information — Instead of Saying It Doesn't Know, the Model States the Past in the Present Tense

The knowledge cutoff is well known, but the actual incidents don't come from the cutoff itself — they come from the way the cutoff goes unmarked. Rather than saying "I don't know anything past the point I know," the model states the last state it knows in the present tense. Since the sentence carries no timestamp, the reader has no way to tell whether it's old information.

The risk items are fixed: prices and pricing plans, product and model names, API parameters and defaults, version numbers and end-of-support dates, a company's policy language, and people's affiliations and titles. What they have in common is that they change often and are easy to verify. Easy to verify also means a reader notices immediately.

Partial automation is possible. Flag time-sensitive phrasing by machine, and have a human check only those spots.

# Pull out time-sensitive phrasing to narrow down what needs review. The key is not treating it as a failure.
# A low-precision check turned into a hard gate soon gets ignored, and an ignored gate is worse than none at all.
rg -n --no-heading \
  -e 'latest|currently|right now|this year|last year|recently' \
  -e 'the (fastest|cheapest|largest|most)|first in the industry|the only' \
  -e '(is free|free of charge)|[0-9]+ *(dollars|USD|\$)' \
  -e 'v?[0-9]+\.[0-9]+(\.[0-9]+)? *(version|release)?' \
  data/blog/**/*.mdx

The fix is to nail the sentence down to a point in time. Write "as of July 2026, a free tier is offered" instead of "it's currently free," and even if it later turns out to be wrong, the post itself isn't a lie. A sentence you're reluctant to pin to a timestamp is usually one you haven't actually verified.

One more thing. Asking the model to "update this with the latest information" isn't a fix. The model can't know anything past its own cutoff, so given that request, it just produces a sentence that looks up to date. Stale information turns into plausible-looking stale information — that's all.

Stretched-Out Paragraphs and Unsupported Assertions — Two Sentence-Level Failures

A Paragraph That Stretches the Same Point

Under pressure to hit a length target, a model repeats the same information in different words instead of adding new information. This is hard to spot because each paragraph looks fine on its own. Read three paragraphs individually and each one makes sense; read them in sequence and the second and third are just restating the first.

A machine can catch this. Cut paragraphs into fragments and measure the overlap ratio.

// find-duplicate-paragraphs.mjs — pulls near-duplicate paragraphs, ranked by similarity.
// The goal is finding "substantially the same point," not exact matches, so shingling + Jaccard is enough.
import { readFileSync } from 'node:fs'

const K = 5 // a 5-word shingle works well for English prose; for CJK text drop this to 3-4.
const THRESHOLD = 0.35

const shingles = (text) => {
  const words = text
    .replace(/`[^`]*`/g, ' ') // exclude inline code
    .replace(/[^\p{L}\p{N}\s]/gu, ' ')
    .split(/\s+/)
    .filter(Boolean)
  const set = new Set()
  for (let i = 0; i + K <= words.length; i++) set.add(words.slice(i, i + K).join(' '))
  return set
}

const jaccard = (a, b) => {
  if (!a.size || !b.size) return 0
  let shared = 0
  for (const s of a) if (b.has(s)) shared++
  return shared / (a.size + b.size - shared)
}

const body = readFileSync(process.argv[2], 'utf8')
  .replace(/^---[\s\S]*?\n---\n/, '')
  .replace(/```[\s\S]*?```/g, '') // exclude fenced code blocks
  .replace(/^\|.*\|$/gm, '') // exclude table rows: tables naturally repeat their format

const paras = body
  .split(/\n{2,}/)
  .map((p) => p.trim())
  .filter((p) => p.length > 120 && !p.startsWith('#') && !p.startsWith('-'))

const pairs = []
const sets = paras.map(shingles)
for (let i = 0; i < paras.length; i++) {
  for (let j = i + 1; j < paras.length; j++) {
    const score = jaccard(sets[i], sets[j])
    if (score >= THRESHOLD) pairs.push({ score, i, j })
  }
}

pairs.sort((a, b) => b.score - a.score)
for (const { score, i, j } of pairs.slice(0, 5)) {
  console.log(`\nsimilarity ${score.toFixed(2)} — paragraph ${i + 1} and paragraph ${j + 1}`)
  console.log(`  A: ${paras[i].slice(0, 90)}...`)
  console.log(`  B: ${paras[j].slice(0, 90)}...`)
}
console.log(`\n${pairs.length} pair(s) at or above the threshold, out of ${paras.length} paragraphs`)

The threshold is a matter of taste. Start around 0.35 and raise it if you get too many false positives. That said, this tool only flags, it doesn't judge. Repetition can be deliberate — revisiting a principle set out earlier with a concrete example later is structure, not repetition.

The fix is to delete the paragraph, not polish it. If two paragraphs say the same thing, keep the more specific one and remove the rest. Merging them into one long paragraph leaves the problem intact.

A Sentence That Asserts Without Evidence

When a human writes, the degree of confidence leaks into the sentence. If they're not sure, hedges show up — "maybe," "depending on the case," "in my experience." A model's output has none of this signal, uniformly. A fact confirmed a thousand times and a sentence that was just invented come out in the same tone.

So the assertion itself becomes the signal. Below are sentence patterns that show up repeatedly in unverified drafts.

The left column of this table is searchable by machine. It's realistic to pull candidates with regex and have a human judge each one. Don't let it auto-fix. Tell the model "soften the assertion" and it just tacks a hedge onto an unsupported sentence — a wrong statement becomes a cautiously wrong statement.

Thin Content and Repeated Phrasing — What Readers and Search Actually Penalize

Let's first clear up a widespread misconception. Google does not penalize content just because it was made with AI. Search Central's official position is that it looks at quality, not the means of production, and this standard predates automated generation becoming an issue at all.

What actually gets penalized is something else. The spam policy documentation explicitly defines mass-producing content that adds no value to the reader, purely to chase search rankings, as scaled content abuse. The important part is that the standard here isn't "did AI write it" but "did it add value." Hire ten people to do the same thing by hand and you run into the same policy.

And something penalizes it even before the search engine does: the reader. The symptoms of thin content are clear. The scroll is long, and by the end, the reader hasn't gained anything beyond what they knew before they searched. A post like this loses return visits before it ever loses ranking.

Here are three self-diagnostic questions worth keeping around.

1. What does this post have that nothing else does? If there isn't at least one thing you actually did, a failure you actually ran into, or a fact you confirmed yourself, it's thin.
2. Does someone who already read the top three results have a reason to read this one too? If it's the same content in different words, there's no reason.
3. If you deleted an entire section, would the reader lose anything? If not, that section exists only to pad length.

Repeated phrasing is the visible surface of this same problem. Force a keyword to repeat and the sentence gets unnatural, and a reader notices that unnaturalness first. The check itself is simple — count word frequency across the body and see if the top items appear only as often as the context actually calls for. That said, Korean particles attach to words and split their surface forms, so a naive string count will undercount actual frequency. Counting by stem, or just reading through by eye, is more accurate.

Plagiarism and Copyright — How Closely Does the Output Resemble the Source

This item is the hardest to detect automatically, and that's exactly why it needs to be handled with rules.

Let's get the actual issue right first. What matters in practice isn't what the model was trained on — it's how closely the output reproduces a specific source. The legality of training data is a matter for litigation and legislation, and it's still being sorted out; whether someone else's exact phrasing ended up verbatim in your post is a problem you control right now.

The conditions that raise the risk are predictable: when the source is a famous, widely quoted text; when the request is tightly bound to a single source, like "summarize this piece"; and in domains where phrasing is formulaic (definitions, legal text, product descriptions). Under these conditions, a model actually does reproduce the source's sentence structure almost verbatim.

It's better to keep the practical rules simple.

• Always attach quotation marks and a source to a quote, and keep it short. Paraphrase instead wherever you can.
• Don't feed in a single source and have it summarized. Feeding in at least two or three sources together lowers the odds of locking onto one specific phrasing.
• Translation is not a way to avoid plagiarism. Substantial similarity of expression survives translation.
• Apply the same standard to images, tables, and diagrams. When you carry over a table that has a source, cite the source.
• Follow the terms on anything under a stated license. This is a matter of compliance, not judgment.

Detection is only partially possible. A manual check — searching for a distinctive sentence — is still the most reliable method, and automated tools are reference-level at best. That said, following the rules above sharply lowers the odds of an incident in the first place. This is a domain where prevention is overwhelmingly cheaper than detection.

Closing — The Point of a Guardrail Isn't Passing, It's Deciding Where to Look

Lay the six back out and a common thread appears. All six fail on facts outside the sentence, not the quality of the sentence. Whether the quote is in the source, whether the information still holds now, whether this paragraph says something different from the last one, whether this assertion has evidence. Reading the sentences alone settles none of it, so no matter how carefully you read, none of it gets filtered out.

That's also where the purpose of a guardrail comes from. It's not there to say a check was passed so it's safe — it's there to narrow down where a human needs to look. A verbatim-quote cross-check clears out the fabricated quotes so you only have to read what's left; duplicate detection shows you only the suspicious paragraph pairs. The judgment after that is still a human's to make, and try to eliminate that part and the guardrail becomes a substitute for verification instead of a support for it.

Boiled down to one line — most of what needs fixing in AI-written text isn't the sentences. It's the grounds for believing those sentences are true.

References

• Google Search Central — Google's stance on AI-generated content
• Google Search Central — Spam policies: scaled content abuse
• Google Search Central — Creating helpful, reliable, people-first content
• Survey of Hallucination in Natural Language Generation — a taxonomy and evaluation methods for hallucination
• A Survey on Hallucination in Large Language Models — causes, detection, and mitigation techniques
• Building an AI Blog-Writing Pipeline — the full flow, centered on verification (related post)