A Paper With Fake Authors Got Accepted for an Oral — a Systems Analysis of Peer Review's Failure in the AI Era

• Introduction — 15 of 22, and Two Papers Given Oral Slots
• What Was Actually Found — Nonexistent Authors on Real Papers
• What's Confirmed and What's One Person's Testimony
• The Scale of Contamination, in Numbers
• Why the Review Pipeline Can't Filter This Out
• LLM Use on the Reviewer Side — What ICML's Watermark Caught, and What It Missed
• The Integrity Mechanisms on the Table, and What Limits Each One
• Conclusion — Detection Is Already Automated; the Decision Isn't

Introduction — 15 of 22, and Two Papers Given Oral Slots

On July 30, 2026, Caleb Robinson and Isaac Corley posted Q&A from the slop trenches. Between the two of them, they reviewed 22 conference submissions this summer, and 15 of those (68 percent) had fabricated citations, fake author lists attached to real papers, or unmistakable signs of LLM generation. The assignments spanned NeurIPS (the Datasets and Benchmarks track and the Position Paper track), WACV, and TerraBytes, an ECCV workshop. Individually, Caleb found issues in 6 of his 11, and Isaac in 9 of his 11.

The post landed on Hacker News under the title "I flagged two research papers for fake authors and both were accepted as orals," and picked up 248 points. What the title points to is a single paragraph from Isaac's answer. He found two submissions that cited papers by researchers he personally knew, where only the author names had been swapped for fictitious people; he recommended rejection and reported it directly to the organizers. Both were accepted as oral presentations, on the condition that only the hallucinated references be fixed.

Reading this as a story about individual misconduct gets you very little. What's interesting is the system side — detection succeeded, the reporting channel worked, and yet the outcome didn't change. This post traces the point in that chain where the signal gets lost.

What Was Actually Found — Nonexistent Authors on Real Papers

The hardest type to catch isn't a reference fabricated wholesale. It's an entry where the title, the venue, and the other co-authors are all real, but one or two of the author names have been swapped for a plausible-sounding fake. Caleb's example is exactly that shape. One WACV submission listed the authors of the SatMAE paper as "Yuyang Cong, Saurabh Khanna, Chen Meng," when the actual author list begins with Yezhen Cong, Samar Khanna, and Chenlin Meng (arXiv:2207.08051).

# What actually needs to be compared to verify a single reference entry
Value printed in the submission:
  Cong, Yuyang; Khanna, Saurabh; Meng, Chen; et al.
  "SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery"

Record at the registry (arXiv:2207.08051):
  Cong, Yezhen; Khanna, Samar; Meng, Chenlin; et al.
  Same title, same year, same identifier

# A check that only confirms the title or DOI exists lets this entry pass.
# You only catch it by diffing the full author list field by field.

This distinction matters because it's exactly where the design requirements for an automated verification tool diverge. A check that asks "does this reference exist" lets it through; only a check that asks "does the printed value match the registry record" catches it. bib-audit, which the two co-authors released, automates the latter — it looks up every entry against Crossref, arXiv, DataCite, and Semantic Scholar, diffs field by field including the full author list, and reports entries in order starting with citations that don't exist at all. Entries with no DOI or arXiv ID that can only be matched by title aren't flagged definitively; they come back as an advisory warning to "check by hand."

Another detail worth noting is the phrasing the two recommend for a review. Don't assert motive — report only the observation: write "the first author on the arXiv record is Yezhen Cong, not Yuyang Cong" and leave the judgment to the editor. That's the practically correct posture. A reviewer isn't in a position to prove manipulation, and trying to turns the review into an argument.

What's Confirmed and What's One Person's Testimony

There's something worth making explicit here. The post above is a first-person account from two reviewers, and I could not find confirmation from the conferences involved. Specifically:

Verifiable: the scale of the assignment — 22 papers — and the 15 that had issues are laid out in a table in the post. The SatMAE author-list mismatch can be verified by anyone against the arXiv record. bib-audit is up in a public repository under the MIT license. The external studies the post cites (covered below) each have their own original sources.

Unconfirmed: the claim that "we flagged fake authors, and both papers were accepted as orals anyway" rests entirely on Isaac's account. Which conference it was isn't stated in the post. His assignments were 2 NeurIPS Position papers, 5 TerraBytes, and 4 WACV, so it's likely a workshop with oral slots or WACV — but that's my inference, not a stated fact. How the organizers actually decided, whether the report reached an ethics officer, what the final decision-maker actually saw — none of this is public. Since the paper isn't identified, there's no counter-account to weigh against it either.

I'm drawing this distinction not purely out of caution. In a systems analysis, this is precisely the candidate point of failure — the fact that neither an outside observer nor the reviewer himself can tell whether the reviewer's report actually factored into the final decision is itself an observability problem in the pipeline.

The Scale of Contamination, in Numbers

To check whether one person's experience is representative, you need other observations. Translating the measurements from the past year that the original post compiled, sourced back to the originals where possible:

Every source in the table above comes from following what the original post itself cited; items with no link attached are ones I could not open the original of directly. The citation relationships can be confirmed in the original post's reference list.

Ansari's taxonomy is especially useful in practice. Breaking the 100 entries down by type: 66 percent fully fabricated, 27 percent partial-attribution contamination, 4 percent identifier hijacking, 2 percent placeholders, and 1 percent semantic hallucination — and all 100 were compound failures. In other words, "a paper that doesn't exist" never shows up alone; it's usually accompanied by semantic hallucination (63 percent) or identifier hijacking (29 percent), which is what makes it look plausible. The per-paper distribution is bimodal too — 92 percent have 1–2 instances, 8 percent have 4–13. It's natural to read the first group as carelessness and the second as wholesale generation.

Why the Review Pipeline Can't Filter This Out

Three pressures overlap here.

The first is submission volume. The post notes that AAAI 2027 received 48,000 abstract submissions. Ansari's figure of 1 percent contamination among accepted papers, in absolute terms, comes out to dozens per conference. At this scale, doing desk rejection by hand is impossible, and in practice reviewers are now crowdsourcing desk rejection in its place. Isaac phrases it exactly that way — that it's not a fair use of time for the reviewers who are still acting in good faith.

The second is mandatory reciprocal reviewing. A system that requires reviewing 4–5 papers to submit one automatically ties reviewer supply to submission volume. In exchange, the motivation for reviewing shifts from "I want to read this field" to "a toll I pay to get my own paper judged." ICML 2026's review FAQ shows a target load of 5–6 papers per reviewer, with no load relief allowed in 2026. In exchange, more than 99 percent of submissions had received at least 3 reviews by the start of rebuttals. Coverage gets secured, but coverage and attention are different things.

The third is information loss at the decision stage. When a reviewer writes "the references are fabricated," the area chair receives it as one axis among several. Once novelty, experiments, clarity, and an integrity flag sit on the same plane, the integrity flag gets absorbed as a "fixable flaw" whenever the other scores are high. This is the structure that produces conditional acceptances like "just fix the hallucinated references." Integrity needs to be a gate, not a scoring axis — and most review forms don't have a gate.

LLM Use on the Reviewer Side — What ICML's Watermark Caught, and What It Missed

Contamination isn't limited to submissions. Pangram's analysis of ICLR 2026 reviews reported that 21 percent were fully AI-generated, and Gartenberg et al. (2026) measured, at the journal Organization Science, that submissions rose 42 percent after ChatGPT's arrival and more than 30 percent of reviews use AI in some form.

The way ICML 2026 responded to this is technically worth a look. It embedded an instruction, invisible to human eyes but readable only by an LLM, into every submitted PDF. The instruction directs the model to insert two phrases, drawn at random from a dictionary of roughly 170,000 phrases, into the review. The probability of two such phrases co-occurring by chance is less than 1 in 10 billion; every flagged case was manually inspected by a human to filter out instances where "the reviewer merely mentioned that a watermark existed," and the resulting family-wise error rate is 0.0001. In pre-deadline testing, the detection rate exceeded 80 percent for most models.

The result was 795 flagged reviews from 506 reviewers, 398 of whom violated the reciprocal-reviewing requirement, leading to 497 desk rejections. Of the violators detected, 51 — 10 percent — had more than half of their own reviews deleted.

The organizers wrote the limitations down themselves. This technique only catches someone who dumps the entire PDF into an LLM and pastes the output back verbatim. Excerpt the text instead, touch up the output, or use a local model, and you won't be caught. In other words, 795 is a floor, and the method can't distinguish an honest slip from a sophisticated violation.

And there's worse news. Li et al. (2026) showed that adversarially rewriting only the abstract — without changing a paper's scientific content or how it's communicated — improves AI review outcomes. Even without knowing which review model was in play, the strongest attack succeeded roughly 38 percent of the time, lifting a Gemini 3 Flash reviewer's score by 1.31 points and a GPT 5.4 Mini reviewer's score by 0.88 points on a 10-point scale. Meaning: the more the review side gets automated, the more gameable it becomes.

The Integrity Mechanisms on the Table, and What Limits Each One

Four kinds of mechanism are actually running right now.

Automated citation verification at submission time. The direction Ansari recommends and bib-audit implements. Cross-checking against a registry is mechanical, automatable, and something authors can run as a CI gate before submitting. It only catches whether a citation is factual; it does nothing about fabrication in experiments or the body text.

Desk rejection at the camera-ready stage. ACL 2026 actually enforced this. Per ACL's statement, it desk-rejected more than 100 accepted papers that cited nonexistent literature, in a two-stage process: an automated system flagged suspicious entries in camera-ready submissions, and program chairs and senior area chairs then confirmed by human eye that "this reference does not exist." The statement defines a citation as "a formal claim about the relationship between two works," and states plainly that when that relationship is fabricated, the integrity of both the document itself and the scholarly record is compromised. That said, an update is attached to the same statement — the ACL executive committee met and decided the papers in question would go through the normal resubmission process. That's a reversal, not a penalty.

Detection and sanctions for reviewer-side LLM use. ICML's watermark is the leading example, and the policy language differs by conference. The ECCV 2026 review policy the original post cites bans writing a review or meta-review with an LLM, local or via API, and separately bans sharing a substantial portion of a submission with an LLM. WACV's reviewer guidelines classify an LLM-generated review as "highly irresponsible behavior" and allow sanctioning it with a desk rejection of the reviewer's own paper. There's a practical snag here — even a verification tool like bib-audit sends part of a submission to a hosted LLM, so reviewers need to check the relevant conference's policy before using it. Integrity tooling and confidentiality rules collide head-on.

A controlled experiment in AI-assisted review. NeurIPS 2026 chose measurement over prohibition. Participating reviewers are randomly assigned, per paper, to one of three conditions — no LLM, free-form LLM assistance, or structured LLM assistance. The interface is integrated into OpenReview and applies only to papers whose authors consented to participate in the experiment. Once reviews are in, area chairs blind to condition evaluate review quality. Every use of an LLM outside this experiment is subject to desk rejection as an integrity-policy violation. Personally, I think this is the most honest approach on the table — prohibition doesn't get enforced, and looking away destroys the signal, so instead it's trying to settle, with data, what form of assistance actually improves review quality.

Recalibrating anonymity is layered on top of this. ICLR has begun revealing author names on OpenReview. It's a move that raises the reputational cost of low-effort mass submission, but it also brings back the very bias double-blind review was meant to block. There's no free exchange here.

Conclusion — Detection Is Already Automated; the Decision Isn't

What failed in this incident wasn't detection. The reviewer found the fake authors, recommended rejection, and reported it directly to the organizers. The signal was accurate and arrived on time. And the outcome still didn't change.

Three things remain.

• Integrity has to be a gate, not a scoring axis. The moment it sits on the same plane as novelty, experiments, and clarity, it gets absorbed whenever everything else looks good. ACL desk-rejecting more than 100 papers at the camera-ready stage is a case of building a gate — late, but a gate.
• Hand mechanical checks to machines, and spend the reviewer's time on the science. Cross-checking references against a registry field by field is automatable and can move to a pre-submission CI gate. Right now that work is being done after the fact by unpaid reviewers.
• Don't mix one person's testimony with a conference's confirmation. The central anecdote in this post has not yet been confirmed by any conference, and the paper hasn't been identified. That unconfirmed state is itself worth pointing out — if nobody can see how a reviewer's integrity report actually fed into the final decision, the reporting channel has no observability.

The review pipeline was designed as a trust system, not a spam filter. Facing inputs whose generation cost converges to zero, we're watching, in real time, how a system built on the assumption of trust comes apart.