Chaos and Order

💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction
Why this article does not rank models
What text LLM reports are actually arguing about
Nine technical reports worth reading
Limitations the authors state themselves
What a practitioner should read first
How and when this was verified
Try it yourself
References

Introduction

This series works through one domain at a time and answers a single question: what is worth reading right now. Part one is text LLMs.

There are two reasons to read a technical report. One is to pick a model to deploy. The other is to understand which design decisions the field is currently arguing about. Reports are surprisingly weak for the first purpose and irreplaceable for the second. This article is a guide for the second.

Paper details were verified directly from the primary sources on 2026-08-12. This field moves fast, so check the current state yourself.

Why this article does not rank models

The benchmark tables in a technical report are, with few exceptions, the authors measuring their own model. They are not independent evaluations. Change the eval harness, the prompt format, the number of few-shot examples, or the regex that extracts the answer, and the number changes. Putting report A's 82 next to report B's 80 and declaring a winner is mostly meaningless.

So this series looks at three things instead of score tables: the problem the authors set out to solve, what they gave up to solve it, and the scope limit the authors admit in their own abstract.

What text LLM reports are actually arguing about

[Four axes the reports actually contest]
 Scale     : total parameters vs parameters activated per token
 Reasoning : how long to think, and when to stop
 Context   : the gap between training length and inference demand
 Openness  : weights only, or data and checkpoints too

These four are coupled. Cutting activated parameters lowers serving cost but introduces routing instability and load imbalance. Thinking longer raises accuracy on verifiable tasks but raises latency and token cost with it. When reading a report, the fastest move is to find which axis the authors gripped and which one they let go.

Nine technical reports worth reading

DeepSeek-V3 Technical Report

arxiv.org/abs/2412.19437 — DeepSeek-AI, submitted 2024-12-27, v2 on 2025-02-18.

A Mixture-of-Experts model with 671B total parameters that activates 37B per token. The abstract leads with Multi-head Latent Attention and the DeepSeekMoE architecture, an auxiliary-loss-free load balancing strategy, and a multi-token prediction objective. Pre-training consumed 14.8 trillion tokens over 2.788M H800 GPU hours, and the authors state that no irrecoverable loss spikes occurred and no rollbacks were performed. For anyone who has actually run a large training job, that last sentence is worth more than the score table.

DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

arxiv.org/abs/2501.12948 — DeepSeek-AI, submitted 2025-01-22, v2 on 2026-01-04. The page shows publication in Nature, volume 645, pages 633–638.

The claim is that reasoning ability can be incentivized by pure reinforcement learning, without human-labeled reasoning trajectories. Patterns such as self-reflection, verification and strategy switching are reported to emerge during training and to transfer into smaller models. The abstract is explicit about the scope of that result, though: it covers tasks where an answer can be checked mechanically, such as mathematics, coding competitions and STEM.

Qwen3 Technical Report

arxiv.org/abs/2505.09388 — Qwen team, submitted 2025-05-14.

A family spanning 0.6B to 235B in both dense and MoE forms. The interesting design decision is folding a thinking mode and a non-thinking mode into one framework, plus a thinking-budget mechanism that allocates inference compute adaptively. Language coverage is stated as expanding from 29 to 119 languages and dialects, released under Apache 2.0.

Gemma 3 Technical Report

arxiv.org/abs/2503.19786 — Gemma team, submitted 2025-03-25.

Lightweight open models from 1B to 27B, now with vision understanding and a minimum 128K context. The part that lands hardest in production is the reduction of KV-cache memory pressure by raising the ratio of local attention. At long context, the thing that decides your bill is often not parameter count but cache.

Olmo 3

arxiv.org/abs/2512.13961 — Team Olmo, submitted 2025-12-15, v2 on 2026-04-14.

7B and 32B released fully open, where "fully" means something specific. Not just weights, but the entire model flow: every stage, checkpoint, data point and dependency used to build the family. For a researcher who needs reproducibility, that single decision matters more than any score. The predecessor report, 2 OLMo 2 Furious (arxiv.org/abs/2501.00656), points the same direction.

Kimi K2: Open Agentic Intelligence

arxiv.org/abs/2507.20534 — Kimi Team, submitted 2025-07-28, v2 on 2026-02-03.

An MoE with 1 trillion total parameters activating 32B per token, trained on 15.5 trillion tokens with an optimizer called MuonClip and, the authors report, no training instability. The numbers in the abstract are 66.1 on Tau2-Bench and 65.8 on SWE-Bench Verified, and the authors themselves scope that comparison to non-thinking models. Quoting the number without carrying the scope along is not citation, it is distortion.

MiniMax-01: Scaling Foundation Models with Lightning Attention

arxiv.org/abs/2501.08313 — MiniMax, submitted 2025-01-14.

Lightning attention combined with MoE, 456B total with 45.9B activated. The focus is context length: the authors state 1 million tokens during training with extrapolation to 4 million tokens at inference. If you want to see how far attention approximation can push long context, this report is a good specimen.

Mellum2 Technical Report

arxiv.org/abs/2605.31268 — submitted 2026-05-29.

A 12B MoE specialized for software engineering, activating 2.5B parameters per token. Roughly 10.6 trillion tokens through a three-phase curriculum, with Grouped-Query Attention, Sliding Window Attention and Multi-Token Prediction. The authors phrase it as competitive with open-weight baselines in the 4B to 14B range, and the emphasis is that it gets there at the per-token compute of a 2.5B dense model.

s1: Simple test-time scaling

arxiv.org/abs/2501.19393 — submitted 2025-01-31, v3 on 2025-03-01.

Fine-tuning on just 1,000 questions curated for difficulty, diversity and quality, combined with budget forcing, which cuts off or extends the thinking process. The authors self-report gains of up to 27% from this technique on competition mathematics such as MATH and AIME24. Read it as evidence that the lever can be inference-time compute allocation rather than data scale.

Limitations the authors state themselves

The DeepSeek-R1 abstract limits its result to verifiable tasks. Carrying that claim over to open-ended writing or taste-dependent judgment is a reading error.
The Kimi K2 benchmark figures are explicitly scoped to a comparison among non-thinking models.
The s1 result centers on competition mathematics, and the abstract does not claim broad generalization.
The performance statements in Mellum2, Qwen3 and Gemma 3 are all author-run measurements. Nowhere in those abstracts is there a third-party evaluation.
The Olmo 3 abstract describes its flagship as the strongest fully-open thinking model released to date. That is the authors' claim, not an independently verified finding.

What a practitioner should read first

If cutting serving cost is the goal, start with DeepSeek-V3 and Mellum2. You come away with the habit of thinking about total parameters and activated parameters as separate quantities.

If raising reasoning quality is the goal, read DeepSeek-R1 and s1 back to back. The first approaches it through training, the second through inference-time budget. The cost structures are completely different.

If your organization needs reproducibility and auditability, start with Olmo 3. A model that published its data lineage and one that did not are different objects when a regulator asks.

If long context is the bottleneck, read Gemma 3's cache reduction next to MiniMax-01's extrapolation. They are two different answers to the same problem.

How and when this was verified

All nine reports here were verified on 2026-08-12 by opening the arXiv abstract page directly and reading the title, identifier, submission date, version history and abstract claims. Candidates that could not be opened were not cited. Every number quoted is as reported by the authors and is not a value I reproduced.

Try it yourself

AI Benchmark Collection — leaderboard sites grouped by category. Put a report's self-reported numbers next to a public leaderboard and compare.
Neural Network Architecture Explorer — if the MoE discussion feels abstract, walking through the transformer structure by hand first is the faster path.

Next in the series: OCR and Document Understanding Technical Reports: What to Read

References

DeepSeek-V3 Technical Report (arXiv 2412.19437): arxiv.org/abs/2412.19437
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning (arXiv 2501.12948): arxiv.org/abs/2501.12948
Qwen3 Technical Report (arXiv 2505.09388): arxiv.org/abs/2505.09388
Gemma 3 Technical Report (arXiv 2503.19786): arxiv.org/abs/2503.19786
Olmo 3 (arXiv 2512.13961): arxiv.org/abs/2512.13961
2 OLMo 2 Furious (arXiv 2501.00656): arxiv.org/abs/2501.00656
Kimi K2: Open Agentic Intelligence (arXiv 2507.20534): arxiv.org/abs/2507.20534
MiniMax-01: Scaling Foundation Models with Lightning Attention (arXiv 2501.08313): arxiv.org/abs/2501.08313
Mellum2 Technical Report (arXiv 2605.31268): arxiv.org/abs/2605.31268
s1: Simple test-time scaling (arXiv 2501.19393): arxiv.org/abs/2501.19393