Chaos and Order

💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

Introduction
Why this article does not declare a winner
What video reports are actually contesting
Ten technical reports worth reading
Limitations the authors state themselves
What a practitioner should read first
How and when this was verified
Try it yourself
References

Introduction

Part three is video. Few domains have a wider gap between what the reports suggest and what using the models feels like. The demo reel always looks good; run it yourself and you hit duration limits, resolution limits and wait times first.

So this article reads constraints instead of demos. The seconds of output, the frame rate, the resolution, the VRAM required and the token budget written in the abstract are what decide what you can build with a model.

Paper details were verified directly from the primary sources on 2026-08-12. This field moves fast, so check the current state yourself.

Why this article does not declare a winner

Video generation is the domain where automatic metrics align worst with human judgment. The VBench authors, introduced below, start from exactly that concern: existing metrics do not fully align with human perception. Add that every report uses a different prompt set and a different pool of raters, and cross-report score comparison barely holds together.

So this article never says one model is better. It records what each report made possible and what it gave up in exchange.

What video reports are actually contesting

[The constraint axes in video reports]
 Duration     : how many seconds in one generation
 Compression  : the VAE ratio decides detail and speed together
 Time axis    : frame-to-frame consistency and camera motion
 Understanding: how many tokens to represent a long video with
 Accessibility: does it run on a consumer GPU

Generation reports mostly grip the first three lines, understanding reports the fourth. The fifth only recently became a first-class objective.

Ten technical reports worth reading

CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer

arxiv.org/abs/2408.06072 — submitted 2024-08-12, v3 on 2025-03-26.

The output specification in the abstract is refreshingly concrete: 10-second continuous video aligned with the text prompt, 16 fps, 768x1360 resolution. The core pieces are a 3D VAE that compresses space and time together and an expert transformer with adaptive LayerNorm for text-video alignment. The authors frame the previous generation's problem as limited movement and short durations.

Movie Gen: A Cast of Media Foundation Models

arxiv.org/abs/2410.13720 — Meta, submitted 2024-10-17, v2 on 2025-02-26.

The largest model is 30 billion parameters with a 73K-token transformer context, generating 16-second video at 16 frames per second with synchronized audio. It bundles not just text-to-video synthesis but personalization, editing and audio generation, which makes it useful reading when you are designing a pipeline rather than picking a model.

HunyuanVideo: A Systematic Framework For Large Video Generative Models

arxiv.org/abs/2412.03603 — submitted 2024-12-03, v6 on 2025-03-11.

An open model of roughly 13 billion parameters. The value here is less the model than the framework write-up: data curation, architecture, scalable training and dedicated infrastructure in a single document. The motivation the authors state is that the leading video generation models remain closed-source.

LTX-Video: Realtime Video Latent Diffusion

arxiv.org/abs/2501.00103 — submitted 2024-12-30.

A report that makes speed a first-class objective, reporting 5 seconds of 24 fps video at 768x512 in 2 seconds on a single H100. The video VAE compresses at 1:192, packing 32x32x8 pixels into a token. And this report names its own limitation in the abstract: high compression inherently limits the representation of fine details. Their remedy is to have the VAE decoder perform the final denoising in pixel space.

Wan: Open and Advanced Large-Scale Video Generative Models

arxiv.org/abs/2503.20314 — Team Wan, submitted 2025-03-26, v2 on 2025-04-19.

Two sizes, 1.3B and 14B, covering text-to-video, image-to-video, editing and personalization. For a practitioner the most important sentence is not about quality: the 1.3B model requires only 8.19 GB of VRAM. Whether something runs on a consumer GPU changes your entire experiment cycle.

Seedance 1.0: Exploring the Boundaries of Video Generation Models

arxiv.org/abs/2506.09113 — submitted 2025-06-10, v2 on 2025-06-28.

Multi-shot generation is supported natively in the training paradigm, with text-to-video and image-to-video learned jointly. The authors report roughly 10x inference speedup from distillation and system optimization, and 1080p 5-second video in 41.4 seconds on an NVIDIA L20.

Seedance 2.0: Advancing Video Generation for World Complexity

arxiv.org/abs/2604.14148 — Team Seedance, submitted 2026-04-15.

The successor moves toward joint audio-video generation. It accepts four input modalities — text, image, audio and video — with reference inputs of up to 3 video clips, 9 images and 3 audio clips. Output runs 4 to 15 seconds at native 480p and 720p. Those numbers are not boasts, they are the current boundary, which is exactly why they are worth reading.

Qwen2.5-VL Technical Report

arxiv.org/abs/2502.13923 — submitted 2025-02-19.

The first understanding-side entry. Dynamic resolution processing and absolute time encoding are stated to enable second-level event localization in videos up to hours long. A native dynamic-resolution ViT with window attention keeps the compute cost manageable.

VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding

arxiv.org/abs/2501.13106 — submitted 2025-01-22, v4 on 2025-06-03.

Interesting because it runs the other direction: a vision-centric approach that prioritizes high-quality image-text data over massive video datasets. Token representations are variable-length depending on image content, with similarity-based token reduction applied to video.

HY-Himmel Technical Report

arxiv.org/abs/2605.08158 — submitted 2026-05-04.

Attacks three long-video problems together: decoding cost, growing token counts and weak motion recognition. Only sparse anchor I-frames go through the visual transformer, while a lightweight compressed-domain tri-stream adapter extracts motion information. The authors report Video-MME moving from 61.2 to 63.5 while cutting context tokens by roughly 3.6x.

Limitations the authors state themselves

LTX-Video admits directly in its abstract that high compression inherently limits fine detail. Reading the compression ratio purely as a performance figure is a mistake.
The durations, resolutions and reference-input counts Seedance 2.0 specifies are the current ceilings of that system.
The VBench authors concede that existing automatic metrics do not fully align with human perception (arxiv.org/abs/2311.17982).
HY-Himmel states that the observed gains depend on the full tri-stream. Taking one part in isolation will not reproduce the result.
Every performance statement above is an author-run measurement, and several include internally constructed benchmarks.

What a practitioner should read first

If getting an experiment loop running locally comes first, look at Wan's 1.3B requirements and LTX-Video's speed design.

If you need to design an entire pipeline, HunyuanVideo and Movie Gen. Documents that walk from data curation to infrastructure in one place are rare.

If you need to build an evaluation process, VBench is the starting point. The way it decomposes quality into 16 dimensions is itself useful for deciding what to measure.

If long-video understanding is the job, read Qwen2.5-VL and HY-Himmel together. One clears the wall with time encoding, the other with token budget.

How and when this was verified

All ten reports here were verified on 2026-08-12 by opening the arXiv abstract page directly. Candidates that could not be opened were not cited. Every specification and number quoted is as reported by the authors.

Try it yourself

ML Training Data Explorer — see by example what training data video generation models consume.
Neural Network Architecture Explorer — following the diffusion transformer discussion is easier once the transformer block is in your hands.

Previous in the series: OCR and Document Understanding Technical Reports: What to Read

Next in the series: Speech Recognition and Synthesis Technical Reports: What to Read

References

VBench: Comprehensive Benchmark Suite for Video Generative Models (arXiv 2311.17982): arxiv.org/abs/2311.17982
CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer (arXiv 2408.06072): arxiv.org/abs/2408.06072
Movie Gen: A Cast of Media Foundation Models (arXiv 2410.13720): arxiv.org/abs/2410.13720
HunyuanVideo: A Systematic Framework For Large Video Generative Models (arXiv 2412.03603): arxiv.org/abs/2412.03603
LTX-Video: Realtime Video Latent Diffusion (arXiv 2501.00103): arxiv.org/abs/2501.00103
VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understanding (arXiv 2501.13106): arxiv.org/abs/2501.13106
Qwen2.5-VL Technical Report (arXiv 2502.13923): arxiv.org/abs/2502.13923
Wan: Open and Advanced Large-Scale Video Generative Models (arXiv 2503.20314): arxiv.org/abs/2503.20314
Seedance 1.0: Exploring the Boundaries of Video Generation Models (arXiv 2506.09113): arxiv.org/abs/2506.09113
HY-Himmel Technical Report (arXiv 2605.08158): arxiv.org/abs/2605.08158
Seedance 2.0: Advancing Video Generation for World Complexity (arXiv 2604.14148): arxiv.org/abs/2604.14148