Skip to content
Published on

What LLMs Cannot Do Is Not the Proof, It Is Setting Up the Premise

🇰🇷KO🇺🇸EN🇯🇵JA🇨🇳ZH
Split View필사 모드
Share
Authors

The Project Where We Swapped the Model Three Times and Got the Same Answer

Suppose you once built an internal tool that finds anomalous patterns in logs and proposes causal hypotheses. Every time you swap in the newest model the sentences get better, but the list of hypotheses that comes out stays almost identical. Deploy timing, traffic spike, cache miss, dependency latency. The same list you were getting six months ago.

Before you spend a few weeks treating this as a prompt problem, it is worth examining another possibility. The model may not be lazy. Producing an item that was not on the list may simply be a kind of work that does not come out of the current structure.

Induction, Deduction, and the Missing Slot

Position: LLMs can not jump, the position paper Tom Zahavy of Google DeepMind submitted to ICML 2026, splits reasoning into three branches. Induction, which extracts rules from data; deduction, which draws conclusions from premises; and abduction, which erects a new hypothesis to explain an observation.

The paper's claim draws a clear line between the first two and the remaining one. Generative AI has already mastered statistical pattern matching, which is to say induction, and it is rapidly conquering deduction, which corresponds to formal proof. But for abduction, the making of new explanatory hypotheses, the paper's position is that no such mechanism exists at all. The jump in the title refers to this move, from experience across to axiom.

This distinction is useful because in practice the two failures look completely different. A model weak at deduction gives you wrong answers. A model without abduction is not wrong and hands you the same list every time. The latter barely shows up in performance metrics.

What the Einstein Case Does for the Paper

The paper uses the formulation of general relativity as a case study. That choice is not decoration, it is the core of the argument.

The observational data of the time was sparse. The precession of the perihelion of Mercury was close to the whole of it, and what could be statistically extracted from that data was roughly a small correction to Newtonian mechanics. The premise called the equivalence principle did not come out of compressing data. It was erected outside the data and then verified against data.

Here is the point the paper makes. Once that premise is given, the deductive stretch from there to the field equations is something a modern LLM can perform plausibly. What it cannot do is the side that erects the premise.

Where Creativity as Compression Falls Apart

This case aims directly at one widely held view: that creativity is ultimately the ability to explain data more briefly, and therefore that a bigger, better-compressing model will eventually make discoveries too.

The paper holds that this view does not survive in discoveries where the observational data is sparse. Compression works only when there is something to compress. Erecting a new explanation at a point where the data is thin is a different operation from compression, and so it does not automatically follow from scaling the model.

What the paper identifies as the bottleneck is the stretch that translates simulation into formal axioms, and the direction it proposes is to supply sensory grounding from a physically consistent multimodal world model. A model trained on text alone learns how the world works from sentences written about the world. When it has to erect an explanation nobody has written yet, that training signal has nothing to lean on. That is the core of the thesis.

This Is a Position Paper, Not a Results Paper

One thing about how to read it needs to be made explicit. The ICML Position track is not a venue for new experimental results but a venue for claims about which direction the field should go. The prefix in the title states that fact.

So this paper does not prove "LLMs cannot do abduction" experimentally. It takes one historical discovery as a computational case study and argues that current training objectives give no grounds for reaching that stretch. This is not a weakness but a property of the format, and an opposing position could come out of the same track. It is also an individual author expressing a position, and there is no basis for reading it as the official view of their institution.

For a practitioner, the useful way to read it is this. Rather than trying to adjudicate whether it is true or false, look at what changes in your design if you assume it is true.

Supplying the Hypothesis Space from Outside the Model

The first change is the placement of roles. Instead of telling an LLM to "come up with" a list of hypotheses, the system builds an enumerable space and the LLM sweeps it, translates it, and ranks it.

Back to the log analysis example: instead of accepting causal hypotheses as free-form prose, you expand the service dependency graph, the recent deploy list, the configuration change history, and infrastructure events into a cross product to produce candidates. The candidate count runs into the hundreds and no human can read them all. Here the LLM works not as a proposer but as a filter and a translator. This is not using the model beneath its abilities, it is placing it where it is strong.

"""Hypotheses come from an enumerable space; the model sweeps and ranks."""
from dataclasses import dataclass
from itertools import product


@dataclass(frozen=True)
class Hypothesis:
    subject: str      # what
    change: str       # changed how
    mechanism: str    # and produces the symptom by which path

    def as_claim(self) -> str:
        return f"{self.change} in {self.subject} produces the symptom via {self.mechanism}"


def enumerate_space(services, changes, mechanisms):
    """The size and the boundary of the hypothesis space are owned by code."""
    return [Hypothesis(*t) for t in product(services, changes, mechanisms)]


def refutable(h: Hypothesis, evidence: dict) -> bool:
    """Falsifiability check: does an observation that would confirm it actually exist?"""
    return h.subject in evidence and h.mechanism in evidence[h.subject]


space = enumerate_space(
    services=["payment-api", "session-cache", "search-index"],
    changes=["a deploy", "a config change", "a capacity reduction"],
    mechanisms=["connection pool exhaustion", "a cache miss spike", "timeout propagation"],
)
evidence = {"payment-api": {"connection pool exhaustion", "timeout propagation"}}

testable = [h for h in space if refutable(h, evidence)]
print(len(space), "candidates, of which", len(testable), "are checkable against observations")
for h in testable[:3]:
    print("-", h.as_claim())

What matters here is that enumerate_space is code. The ceiling on what the system can discover is written in this function rather than in a prompt, which means it can be reviewed and it can be raised.

Do Not Hand Proposal and Falsification to the Same Model

The second change is the separation of roles. Put the side that produces hypotheses and the side that breaks them in the same call, and it tilts easily toward hypotheses that can pass on their own. This is not dishonesty on the model's part but a problem with the objective function. Demand a good answer in a single shot and an answer with low falsifiability looks like the better answer.

So you cut the pipeline like this. The proposal stage draws candidates broadly and does not score them. The falsification stage makes the model write, for each candidate, both "the observation that must be visible if this is true" and "the observation that must never be visible if this is true," and then queries that observation against real data. Humans only look at the few that survive at the end.

Putting falsification in a separate call brings one more side effect. Hypotheses for which no "must never be visible" observation can be written get filtered out automatically. Such hypotheses are usually unfalsifiable statements, sentences that survive no matter what you observe. Those sentences, plausible to a human reader and carrying no information, eat the most time in practice.

The nice thing about this arrangement is that you lose nothing if the paper turns out to be wrong. If the next generation of models really does perform abduction, the candidate quality at the proposal stage simply goes up. The structure stays as it is.

Where Our Problem Ends and the Paper Problem Begins

Finally the scope needs pinning down. What the paper addresses is scientific invention, jumps at the level of erecting new axioms. Most of the systems we build every day do not require a jump at that level.

So summarizing this paper as "LLMs are not creative" goes too far. The more accurate summary is this one. Searching within a known hypothesis space and widening the space itself are different jobs, and current tools are far stronger at the former. So the first question is whether our system needs the latter at all, and if it does, the design has to state explicitly that a human or a simulator owns that part.

Returning to the log tool from the beginning, the reason the same list came out for six months is quite likely not the model but the fact that there was nowhere at all that defined that list.

References

Discuss on Twitter • View on GitHub