필사 모드: Diffusion LLMs That Write CUDA Kernels — DICE and Why Parallel Generation Might Help

• Introduction — A Diffusion Model That Writes Kernels
• Autoregressive vs Diffusion — Why Left-to-Right Fits Code Poorly
• What DICE Actually Did — CuKe and Two-Stage RL
• Why CUDA Kernels Are a Good Testbed — and Exactly Where the Trap Is
• Closing — What's Interesting, What's Uncertain
• References

Introduction — A Diffusion Model That Writes Kernels

In February 2026, Haolei Bai and five co-authors released the preprint "DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels" (v1 2026-02-12, v2 2026-06-16). The title is provocative on its face: a diffusion LLM is good at generating CUDA kernels. Nearly every code-generation model we know is autoregressive (AR) — it appends tokens one at a time, left to right. DICE instead adapts a diffusion-family LLM to this task and claims to outperform both autoregressive and diffusion LLMs of comparable scale, setting a new state of the art (SOTA).

This post reads that preprint's abstract through an engineer's lens: why the parallel, global nature of diffusion generation plausibly helps on a hard-but-verifiable task like writing fast GPU kernels, what DICE actually did, and where honest skepticism belongs — what to believe and what to leave open. One nail up front: this is a non-peer-reviewed preprint, and the abstract carries no concrete benchmark numbers. So words like "SOTA" and "outperforms" below are all the authors' claims, not independently verified figures.

Autoregressive vs Diffusion — Why Left-to-Right Fits Code Poorly

An autoregressive model predicts one next token, commits it, and moves on. That is natural for prose but awkward for code, because it fixes the early part too soon. A wrong data structure or tile size chosen near the top of a kernel gets baked in as every later token stacks on top of it. Even when the model later realizes "ah, I should have laid out shared memory differently," a strictly left-to-right generator has no natural path back to what it already wrote.

A diffusion LLM has a different generation shape entirely. Just as image diffusion starts from noise and refines the whole picture over several steps, a discrete diffusion language model roughly starts from a fully masked sequence and, over several steps, fills every position in parallel — and rewrites positions it has already filled. Because the order is not fixed left to right, it is natural to lay down global structure first and fill in details later, or to look ahead and fix what came before: non-sequential refinement. This is exactly why the abstract argues diffusion suits code — in the authors' words, code generation is a setting where "holistic structural planning and non-sequential refinement are critical."

Autoregressive (left -> right):
  tok1 -> tok2 -> tok3 -> ... -> tokN     one token at a time
  a mistake in tok2 is frozen; later tokens only build on it

Diffusion (parallel refinement):
  [every position masked]
     |  step 1: fill a rough draft everywhere at once
     v  step 2: revise any position, in any order
  [coherent kernel]     global structure first, details later

Let me be clear about one thing. The "why it might help" above is the motivation the abstract offers plus my own reading — not a causal result the abstract proves. Whether diffusion actually won because of this property, or because of the data and training recipe, cannot be separated out from the abstract alone.

What DICE Actually Did — CuKe and Two-Stage RL

The intuition that diffusion should fit code is one thing; making it actually work on CUDA kernels is another. The abstract names two concrete obstacles: the high specialization of the CUDA domain, and a severe lack of high-quality training data. Good CUDA kernels are not lying around the internet the way ordinary Python is.

DICE attacks this on two fronts.

• CuKe — an augmented supervised fine-tuning (SFT) dataset built for high-performance CUDA kernels. It aims squarely at the data-scarcity problem.
• BiC-RL (bi-phase curated reinforcement learning) — a two-stage RL framework. Stage one is CUDA kernel infilling: filling in the blanks of a kernel (which meshes naturally with diffusion's mask-filling behavior). Stage two is end-to-end kernel generation: producing the whole thing from scratch.

Trained this way, DICE is presented at three scales — 1.7B, 4B, and 8B — and, per the abstract, on KernelBench it significantly outperforms comparable autoregressive and diffusion LLMs, establishing a new SOTA for CUDA kernel generation. To repeat: the abstract does not state which scale beat what by how many points. "SOTA" and "significantly outperforms" are the authors' wording, and the magnitude is unknown until you read the body and tables directly.

Why CUDA Kernels Are a Good Testbed — and Exactly Where the Trap Is

Part of what makes CUDA kernel generation an interesting testbed is that the task is verifiable. A benchmark like KernelBench (Stanford) scores an LLM-written kernel on three mechanical criteria: does it compile and run, does it match a reference PyTorch implementation numerically (correctness), and how much faster is it than that reference (speedup). All three can be measured automatically, with no human judgment.

That automatic measurability is decisive. Reinforcement learning works best when it has a clean, dense reward signal, and "compiles + correct + N-times faster" is exactly such a signal. On top of that, diffusion's iterative refinement — take a draft and rewrite it over several steps — pairs well with this kind of verification loop. In short, CUDA kernels present nearly ideal conditions for diffusion plus RL to shine.

But that same fact is the trap in reading the result. Doing well where a verifiable reward is crisp is a very different thing from doing well on open-ended code — business logic, API design, fuzzy requirements — where correctness does not reduce to compile-and-measure. Whether DICE's edge is an advantage of the diffusion paradigm itself, or the happy alignment of a "verifiable plus RL" setup, cannot be told apart from the abstract alone.

Closing — What's Interesting, What's Uncertain

What is interesting. First, the data point itself: a diffusion LLM outperforming a same-size autoregressive model on a genuinely hard systems task. It suggests the "code generation is autoregressive" default is not the only answer. Second, the infilling stage is conceptually clean — it resembles how a human refines a kernel, keeping a skeleton and filling and fixing the inside. Third, it sits on the same line as a broader trend several recent results are converging on: pairing verifiable tasks with RL.

What is uncertain. Every quantitative phrase here — "SOTA," "outperforms" — is a claim from a non-peer-reviewed preprint, and the abstract has no concrete numbers. Generalization beyond kernels is not addressed, and the verifiability trap above remains. Until the code and weights are released and the community re-measures on varied hardware and varied kernels, read it with care. Still, the direction is clear. There is no law that code generation must go left to right, and global, parallel generation can yield a real edge on verifiable tasks — DICE at least shows the hypothesis is worth testing seriously.

References

• Bai et al. (2026), "DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels"
• The same paper, PDF
• Ouyang et al. (2025), "KernelBench: Can LLMs Write Efficient GPU Kernels?"
• KernelBench overview (Stanford Scaling Intelligence Lab)