Split View: AI로 블로그 글 파이프라인 만들기 — 초안이 아니라 검증이 병목입니다

Building an AI Blog-Writing Pipeline — Verification, Not Drafting, Is the Bottleneck

• Introduction — Three Minutes for a Draft, Ninety for Verification
• The Pipeline's Seven Stages and the Cost of Failure at Each One
• Topic and Research — AI Expands the Candidates, a Human Discards Them
• Drafting — Kill the Blank Screen, But Don't Hand Off the Structure
• Fact-Checking — The Center of Gravity of This Pipeline
  • A Confidently Written Sentence and a Verified Sentence Are Different Things
  • Making the Model Cite Only What It Actually Read
  • The Pre-Publish Checklist
• Editing, Publishing, and Checking Performance — Unify the Voice, Make It Reversible
• An Automation Sketch — A Workflow That Leaves in the Places It Can Stop
• Closing — What's Worth Automating Is Judgment, Not Generation
• References

Introduction — Three Minutes for a Draft, Ninety for Verification

Once you start writing with AI, almost everyone passes through the same illusion: since the draft comes out in three minutes, the time spent on a single post should drop to a tenth. In practice, the draft really does take three minutes — and then everything after it takes ninety. That's how long it takes to check whether the numbers are right, click every link to see if it's alive, cross-check whether a quote actually appears in the source, and strip out the paragraph that says the same thing three different ways.

So if you design the pipeline around draft generation, you build it wrong, because you've automated the part that was never the bottleneck. The stage that actually eats time is verification, and it's also where there's the most room to improve. Double the drafting speed and three minutes becomes a minute and a half; cut verification in half and ninety minutes becomes forty-five.

This isn't a collection of prompts. It's a record of actually running all seven stages, from topic collection to checking performance, and separating what the model is good at from what causes an incident if you hand it off. The individual failure types that make AI-written posts fall apart, and how to detect each one, are covered separately in the next post; here we look at the overall flow that filters them out.

The Pipeline's Seven Stages and the Cost of Failure at Each One

Let's start with the full map. The table below splits out, for each stage, where AI genuinely creates value and where things become hard to undo the moment you hand them off.

What you should read out of this table isn't the individual cells — it's the distribution of cost. Stages 2 and 4 concentrate the cost. Those two should be automated, but blocked from passing without human sign-off; the rest can be handed off boldly. Most of pipeline design is just respecting this distribution.

Topic and Research — AI Expands the Candidates, a Human Discards Them

In topic collection, the model is strong at divergence. Pulling twenty angles out of one seed is faster than a human and never gets tired. It's weak at convergence, in the opposite direction. Ask "which of these is best?" and it picks one with a plausible-sounding rationale, but that judgment doesn't contain what your readers were disappointed by last month. Use it to expand candidates; do the discarding yourself.

There's one automation here that's genuinely valuable in practice: a duplicate check against posts already written. Cross-reference new topic candidates against existing post titles and summaries, flag the overlaps, and you can prevent the incident of rewriting the same story from a slightly different angle. This is comparison, not judgment, so a machine does it well.

The research stage has a different character. An error that gets in here survives to the end of the pipeline, and every later stage just polishes it to look more plausible. So if there's exactly one rule to set, it's this: don't use what the model summarized as your research finding. All the model does is point you to where to read; the reading itself is the human's job.

Put into a form that's actually easy to follow, it looks like this.

• Open every URL the model produces, yourself. If it doesn't open, discard that claim entirely.
• It's fine to start from a secondary source (a blog roundup, a news summary), but rewrite the citation against a primary source (official docs, the paper, the announcement, the release notes).
• Don't use a page with no date as a source. A fact you don't know when it was true can't be verified.
• When you run into conflicting claims, don't ask the model which one is right — write the fact that they conflict directly into the post.

That last item is especially useful in practice. While researching Office keyboard shortcuts for this post's companion piece, I actually ran into cases where Microsoft's own official docs pointed to different keys for the same action. The right answer there isn't to pick one — it's to write "the docs disagree."

Drafting — Kill the Blank Screen, But Don't Hand Off the Structure

The model's value at the drafting stage is clear. It removes the time you'd spend agonizing over the first sentence in front of a blank screen. That alone is useful enough, and this stage is cheap to get wrong — if something's off, it shows, and if it shows, it gets fixed.

There's one thing you should never hand off, though: structure. Leave the outline to the model and you almost always get the same shape. Definition, then pros, then cons, then examples, then conclusion. That structure isn't wrong, but it doesn't argue anything. What a reader usually remembers from a post is the order the structure created, and if you fill that order with an average, nothing is left after reading.

In practice, split it like this. Decide the H2 headings and their order yourself, write two or three lines about what goes under each H2, and hand that to the model. Then the model's job is to fill in paragraphs, and the throughline stays intact. Say "write me a post on this topic" instead, and you get a post with no throughline — not because the model can't do it, but because you never gave it one.

Here are a few constraints that make a noticeable difference when you put them in the drafting prompt.

• Have it leave what it doesn't know blank and flagged. Unless you explicitly say not to fill it in, it will fill it in.
• Have it use only numbers and proper nouns that are in the material you provided.
• Give a paragraph-length cap. Without one, it stretches out the same point.
• Have it write, first, one line for each H2 on "the one new thing the reader learns in this section," and if that overlaps the previous section, merge the sections.

Fact-Checking — The Center of Gravity of This Pipeline

This is the most important section in this post. Do the other six stages sloppily and the post is still publishable as long as this stage is intact; skip this stage and no matter how well you do everything else, an incident eventually happens.

A Confidently Written Sentence and a Verified Sentence Are Different Things

A model's output carries no marker of how confident it is. A fact seen thousands of times in training data, and a sentence that sounds plausible and was just invented, come out in exactly the same tone. In human writing, a sentence wavers when the writer isn't sure, and we read that signal unconsciously; a model's writing has no such signal. So "it reads naturally" doesn't count as verification. If anything, the more natural it reads, the more dangerous it is.

The items to watch closely are fixed: numbers, dates, versions, prices, quotes, people's names and affiliations, and URLs. These are exactly the items a model is most likely to plausibly fabricate, and they're also the items a reader can most easily verify. Get every other sentence right and get just one of these wrong, and the trust in the whole post drops with it.

The knowledge cutoff causes trouble at the exact same spot. Instead of saying "I don't know the latest on this," the model states the last state it knows in the present tense. For anything that changes often — prices, plans, product names, API parameters, policy language — treat the model's answer only as a starting point, and always re-verify it against the current documentation.

Making the Model Cite Only What It Actually Read

Ask for "sources" and you get sources. The problem is that the source might be somewhere plausible, not somewhere it actually read. URL format is easy to learn, so a document number that doesn't exist or a dead anchor gets generated in a perfectly natural-looking shape.

The way to prevent this isn't to change the request — it's to force the output into a verifiable form. The key is a verbatim quote. Have it copy the source's actual characters rather than summarize, and a machine can cross-check whether that string actually exists in the source.

Have it produce these four things together for every single claim.

1. The claim sentence itself
2. The source URL
3. One or more sentences quoted verbatim, character for character from that URL's body
4. One line explaining why that quote supports the claim

Item 3 is the key to automated verification. If the quote isn't on the page, the claim was manufactured with no basis, and a script can catch that before a human ever reads it. Below is a minimal implementation that does that cross-check.

// verify-claims.mjs — cross-checks whether the verbatim quotes in the claims ledger (claims.jsonl) actually appear in the source.
// Each line's format: {"claim": "...", "url": "https://...", "quote": "...verbatim from the source...", "why": "..."}
import { readFileSync } from 'node:fs'

// Differences in whitespace/quotes/case aren't treated as a quote failure.
// We only want to catch cases where the content actually differs.
const normalize = (s) =>
  s
    .replace(/<[^>]+>/g, ' ')
    .replace(/&[a-z]+;|&#\d+;/gi, ' ')
    .replace(/[‘’“”]/g, "'")
    .replace(/\s+/g, ' ')
    .trim()
    .toLowerCase()

const rows = readFileSync('claims.jsonl', 'utf8')
  .split('\n')
  .filter(Boolean)
  .map((l) => JSON.parse(l))

const cache = new Map()
let failed = 0

for (const [i, row] of rows.entries()) {
  const label = `[${i + 1}] ${row.claim.slice(0, 50)}`

  if (!row.url || !row.quote) {
    console.error(`MISSING ${label} — url or quote is empty`)
    failed++
    continue
  }

  if (!cache.has(row.url)) {
    try {
      const res = await fetch(row.url, { redirect: 'follow' })
      // A dead link is already a failure before we even get to checking the quote.
      cache.set(row.url, res.ok ? normalize(await res.text()) : null)
    } catch {
      cache.set(row.url, null)
    }
  }

  const page = cache.get(row.url)
  if (page === null) {
    console.error(`DEAD    ${label} — could not open ${row.url}`)
    failed++
  } else if (!page.includes(normalize(row.quote))) {
    // This is the whole reason this script exists: a plausible-looking quote that isn't in the source.
    console.error(`NOQUOTE ${label} — quote not found in source: "${row.quote.slice(0, 60)}"`)
    failed++
  }
}

console.log(`\nChecked ${rows.length}, failed ${failed}`)
process.exit(failed ? 1 : 0)

It's also worth being clear about what this script can't catch. It'll pass a quote stripped of context even when it's present in the source verbatim, and it'll pass a quote that's accurate but doesn't actually support the claim. So you still need a human to read item 4 (the one-line rationale). That said, once a machine filters the front end, the volume a human needs to read drops sharply. In practice, since adding this cross-check, human verification time has visibly dropped — because we no longer have to comb through source text hunting for a fabricated quote.

The Pre-Publish Checklist

Below is the list actually run before hitting publish. It's split into what a machine can do and what a human must do.

The last item looks like a formality but actually catches the most in practice. If you try to state the core claim without looking at the post and get stuck, that part wasn't verified — it was just let through.

Editing, Publishing, and Checking Performance — Unify the Voice, Make It Reversible

At the editing stage, what you can hand to the model is detection; what you can't is the standard. The model can't judge "is this my voice." Because it tends to converge on the average texture of what it was trained on, hand it off and the sentences get smoother while the post stops being yours.

Instead, if you write your voice down as rules, a machine can check it. Things like a list of words you don't use, a paragraph-length cap, a ban on exclamatory phrasing, a ban on overusing a particular conjunction. These aren't taste — they're checkable conditions, so they can be automated. "Make this sound more natural," on the other hand, isn't a checkable condition — it's outsourcing your taste.

What matters at the publishing stage isn't speed — it's being reversible. Unlike code, an error in a post that surfaces after publishing doesn't roll back quietly. People have already read it, and sometimes it's already been cited elsewhere. So even if you automate publishing itself, it's better to build the rollback path first. Deciding in advance on a rule to leave a correction note inside the post itself helps you avoid the temptation to quietly fix things when something goes wrong.

At the performance-checking stage, the trick is cutting down the metrics. View count doesn't change your next post. What actually changes decisions is the distribution of dwell time (the rate people bounce off the first screen), the gap between search queries that bring people in and the post title, and roughly which section is where scrolling stops. In particular, the gap between search query and content directly tells you the topic for your next post, so if you're only going to look at one thing, look at that.

Here it's worth touching on Google's policy once. Google states explicitly that it doesn't penalize content just because it was made with AI, and that the standard is not the means of production but whether it gives value to the reader, as laid out in its Search Central documentation. That said, mass-producing valueless content purely to chase rankings is explicitly banned as scaled content abuse under its spam policies. To summarize: the moment you set the pipeline's goal to "crank out a lot," you've put yourself on the wrong side of the policy.

An Automation Sketch — A Workflow That Leaves in the Places It Can Stop

Put it all together and you get something like the shape below. What matters isn't the scope of automation — it's where it stops. This workflow doesn't let a draft move to the publish stage unless it clears the verification gate, and a draft never reaches the publish branch without a human's approval.

# .github/workflows/content-gate.yml
# When a draft PR opens, run every check a machine can do — but never auto-merge, even if it all passes.
name: content-gate

on:
  pull_request:
    paths: ['data/blog/**/*.mdx']

jobs:
  verify:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '22'

      - name: Build safety — does the MDX actually compile
        run: node scripts/check-posts.mjs

      - name: Internal links — do the paths they point to exist
        run: node scripts/check-links.mjs

      - name: Verbatim quote cross-check — do quotes actually exist in the source
        run: node scripts/verify-claims.mjs

      - name: Time-sensitive phrasing — just flags spots a human needs to judge
        run: |
          # Doesn't fail the build. This is a signal to prompt review, not verification.
          grep -nE 'latest|currently|this year|last year|recently|fastest yet|first in the industry' data/blog/**/*.mdx \
            | tee /tmp/time-sensitive.txt || true
          echo "The phrases above may state facts that are stale past the cutoff. The reviewer should confirm them directly."

      - name: Duplicate paragraphs — pull only the top-similarity pairs and report
        run: node scripts/find-duplicate-paragraphs.mjs --top 5 || true

  # Human-in-the-loop: even if every check above is green, it stops here.
  # Approval comes from a GitHub review approval — the workflow never approves on its own.

There are two things deliberately left undone here.

First, passing verification was not used as the publish trigger. Machine verification can only tell you "there's no obvious disaster" — never "it's fine to publish." The moment you attach auto-merge, the last three items on the checklist (context distortion, unsupported assertions, whether you can explain it) disappear from the pipeline.

Second, the time-sensitive-phrasing check was not made a failing check. This check has low precision, so making it a hard failure means it gets ignored soon, and an ignored gate is worse than no gate at all. It lasts longer if all it does is tell a human where to look.

Closing — What's Worth Automating Is Judgment, Not Generation

When building an AI content pipeline, your hand reaches for generation first. The result is visible, and making it feels instantly faster. But what actually gives you back time and prevents incidents is the judgment side. Have a machine filter, first, whether a quote is real, whether a link is alive, whether you repeated yourself — and a human only spends time on what the machine can't do.

And what the machine can't do turns out to be smaller and clearer than you'd think: whether a quote distorts its context, whether this topic has value for the reader, and whether the post's argument is actually yours. These three stay a human's job to the end, and if the pipeline is well built, there should be time left over for exactly these three.

Boiled down to one line — it's not that you can't trust a draft. It's that there's no basis to trust an unverified one.

References

• Google Search Central — Google's stance on AI-generated content
• Google Search Central — Creating helpful, reliable, people-first content
• Google Search Central — Spam policies (including scaled content abuse)
• Google Search Central — Search quality rater guidelines overview
• Where AI-Written Posts Fall Apart — failure types and guardrails (related post)