HunyuanVideo and LTX-2 Complete Analysis: Architecture, Performance, and Practical Guide to Open-Source Video Generation Models

[AI Video Generation Model Development Timeline]

2024 Q1-Q2          2024 Q3-Q4          2025 Q1-Q2          2025 Q3-Q4          2026 Q1
    |                   |                   |                   |                   |
    v                   v                   v                   v                   v
 Sora Preview       HunyuanVideo        Wan 2.1             HunyuanVideo 1.5    LTX-2 Open
 Runway Gen-3       CogVideoX           LTX-Video 1.0       Wan 2.2 (MoE)      Wan 2.6
 Pika 1.0           Kling 1.0           Mochi 1             LTX-2 Preview      Veo 3.1
                    Mochi Preview                            Sora 2

[Open Source vs Commercial Model Competition]

Commercial:   Sora --> Sora 2 --> Veo 3.1 --> Kling 3.5
                 \       \           \           \
                  \       \           \           v
Open Source:  CogVideoX -> HunyuanVideo -> Wan 2.1 -> LTX-2
                 \           \           \           \
                  v           v           v           v
             Quality Gap:  Gap Narrows:  On Par:     Surpassing:
             Commercial    Rapid         Benchmark   Speed/Access
             Advantage     Catch-up      Parity      Advantage

[HunyuanVideo Input/Output Pipeline]

Text-to-Video:
  "A golden retriever running       +----------+     +--------+
   through a sunlit meadow"  -----> | Hunyuan  | --> | Video  |
                                    | Video    |     | Output |
Image-to-Video:                     | Pipeline |     | (MP4)  |
  [Input Image] + Prompt    -----> |          | --> |        |
                                    +----------+     +--------+
                                         |
                                    MLLM Encoder
                                    3D VAE
                                    DiT Denoiser

[HunyuanVideo Full Architecture Diagram]

                    Text Prompt
                         |
                         v
                  +-------------+
                  | MLLM Text   |
                  |   Encoder   |
                  | (Decoder-   |
                  |  Only LLM)  |
                  +------+------+
                         |
                  Text Tokens (with bidirectional refiner)
                         |
                         v
+--------+    +---------------------+    +--------+
| Gaussian| -> | Dual-Stream to      | -> | Denoised|
| Noise   |    | Single-Stream DiT   |    | Result  |
+--------+    |                     |    +---+----+
              | [Dual Phase]        |        |
              |  - Video Tokens     |        v
              |  - Text Tokens      |  +----------+
              |  (independent)      |  | 3D VAE   |
              |                     |  | Decoder  |
              | [Single Phase]      |  +----+-----+
              |  - Concat & Fuse    |       |
              +---------------------+       v
                                      Final Video

# Dual-Stream Phase Pseudocode
class DualStreamBlock(nn.Module):
    def __init__(self, dim, num_heads):
        self.video_attn = MultiHeadAttention(dim, num_heads)
        self.text_attn = MultiHeadAttention(dim, num_heads)
        self.video_ffn = FeedForward(dim)
        self.text_ffn = FeedForward(dim)
        self.video_norm = AdaLayerNorm(dim)
        self.text_norm = AdaLayerNorm(dim)

    def forward(self, video_tokens, text_tokens, timestep):
        # Independent video token processing
        video_tokens = self.video_norm(video_tokens, timestep)
        video_tokens = video_tokens + self.video_attn(video_tokens)
        video_tokens = video_tokens + self.video_ffn(video_tokens)

        # Independent text token processing
        text_tokens = self.text_norm(text_tokens, timestep)
        text_tokens = text_tokens + self.text_attn(text_tokens)
        text_tokens = text_tokens + self.text_ffn(text_tokens)

        return video_tokens, text_tokens

# Single-Stream Phase Pseudocode
class SingleStreamBlock(nn.Module):
    def __init__(self, dim, num_heads):
        self.attn = MultiHeadAttention(dim, num_heads)
        self.ffn = FeedForward(dim)
        self.norm = AdaLayerNorm(dim)

    def forward(self, video_tokens, text_tokens, timestep):
        # Concatenate video + text tokens
        combined = torch.cat([video_tokens, text_tokens], dim=1)

        # Unified processing (Full Attention)
        combined = self.norm(combined, timestep)
        combined = combined + self.attn(combined)
        combined = combined + self.ffn(combined)

        # Split and return
        video_out = combined[:, :video_tokens.shape[1]]
        text_out = combined[:, video_tokens.shape[1]:]

        return video_out, text_out

Input Video:    720 x 1280 x 129 frames x 3 channels
                = ~356M values

Latent:         90 x 160 x 33 x 16 channels
                = ~7.6M values

Compression:    ~47:1 (by element count)

[Text Encoding Pipeline]

Text Prompt
     |
     v
+----------+     +--------------+     +------------------+
| MLLM     | --> | Bidirectional| --> | Final Text       |
| (Causal  |     | Token        |     | Embedding        |
|  Attn)   |     | Refiner      |     | (DiT condition)  |
+----------+     +--------------+     +------------------+
  Rich            Bidirectional        Diffusion-optimized
  semantics       context boost        text representation

# Flow Matching Training Pseudocode
def flow_matching_loss(model, x_0, text_cond):
    """
    x_0: original video latent
    text_cond: text condition
    """
    # Random timestep sampling
    t = torch.rand(x_0.shape[0], device=x_0.device)

    # Noise sampling
    noise = torch.randn_like(x_0)

    # Linear interpolation for intermediate state
    x_t = (1 - t) * x_0 + t * noise

    # Target vector field: noise direction
    target = noise - x_0

    # Model's vector field prediction
    predicted = model(x_t, t, text_cond)

    # Loss computation
    loss = F.mse_loss(predicted, target)
    return loss

# HunyuanVideo original model (13B)
pip install huggingface_hub
huggingface-cli download tencent/HunyuanVideo --local-dir ./HunyuanVideo

# HunyuanVideo 1.5 (8.3B, lighter version)
huggingface-cli download tencent/HunyuanVideo-1.5 --local-dir ./HunyuanVideo-1.5

# Image-to-Video model
huggingface-cli download tencent/HunyuanVideo-I2V --local-dir ./HunyuanVideo-I2V

import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import export_to_video

# Load model
model_id = "tencent/HunyuanVideo"
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    model_id,
    subfolder="transformer",
    torch_dtype=torch.bfloat16,
)
pipe = HunyuanVideoPipeline.from_pretrained(
    model_id,
    transformer=transformer,
    torch_dtype=torch.float16,
)
pipe.vae.enable_tiling()
pipe.to("cuda")

# Generate video
output = pipe(
    prompt="A cat walks on the grass, realistic style, natural lighting",
    height=720,
    width=1280,
    num_frames=129,
    num_inference_steps=30,
    guidance_scale=6.0,
).frames[0]

# Save video
export_to_video(output, "hunyuan_output.mp4", fps=24)

import torch
from diffusers import HunyuanVideoPipeline, HunyuanVideoTransformer3DModel
from diffusers.utils import export_to_video
from transformers import BitsAndBytesConfig

# INT4 quantization config
quant_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.float16,
    bnb_4bit_quant_type="nf4",
)

# Load quantized transformer
transformer = HunyuanVideoTransformer3DModel.from_pretrained(
    "tencent/HunyuanVideo",
    subfolder="transformer",
    quantization_config=quant_config,
)

pipe = HunyuanVideoPipeline.from_pretrained(
    "tencent/HunyuanVideo",
    transformer=transformer,
    torch_dtype=torch.float16,
)
pipe.vae.enable_tiling()

# CPU offload for additional VRAM savings
pipe.enable_model_cpu_offload()

output = pipe(
    prompt="A beautiful sunset over the ocean, cinematic",
    height=544,
    width=960,
    num_frames=65,
    num_inference_steps=30,
    guidance_scale=6.0,
).frames[0]

export_to_video(output, "quantized_output.mp4", fps=24)

[Generation Speed Comparison (5-second video)]

Model              Gen Time    vs Real-time
LTX-Video 1.0:    ~2 sec       2.5x faster
LTX-2:            ~3-5 sec     ~real-time
HunyuanVideo:     ~2-5 min     60x slower
CogVideoX:        ~3-8 min     100x slower
Mochi:            ~5-10 min    120x slower

(H100 GPU, 768x512 resolution)

[LTX-2 Full Architecture]

Text Prompt
     |
     v
+-------------+
| Text Encoder |  (Gemma-based)
| + Prompt     |
|   Enhancer   |
+------+------+
       |
       v
+------------------------------------------+
|        Asymmetric Dual Stream DiT         |
|                                          |
|  +------------------+  +-------------+   |
|  | Video Stream      |  | Audio Stream|   |
|  | (wide channels,   |  | (narrow,   |   |
|  |  high capacity)   |  |  lightweight)|  |
|  +--------+---------+  +------+------+   |
|           |      Cross-Attention  |       |
|           +----------+-----------+       |
+------------------------------------------+
       |                    |
       v                    v
+-------------+      +-------------+
| Video VAE   |      | Audio VAE   |
| Decoder     |      | Decoder     |
| (3D spatio- |      | (1D temporal)|
|  temporal)  |      |             |
+------+------+      +------+------+
       |                    |
       v                    v
   Video Output        Audio Output
       |                    |
       +--------+-----------+
                |
                v
         Final AV Output (MP4)

# 1. Python environment (3.10+ recommended)
conda create -n ltx2 python=3.10
conda activate ltx2

# 2. Install official LTX-2 package
pip install ltx-pipelines

# 3. Or install from source
git clone https://github.com/Lightricks/LTX-2.git
cd LTX-2
pip install -e "packages/ltx-pipelines[all]"
pip install -e "packages/ltx-core[all]"

# 4. Download model weights
huggingface-cli download Lightricks/LTX-2 --local-dir ./models/ltx2

from ltx_pipelines import TI2VidTwoStagesPipeline

# Initialize pipeline
pipe = TI2VidTwoStagesPipeline.from_pretrained(
    "Lightricks/LTX-2",
    device_map="auto",
    enable_fp8=True,  # Save VRAM with FP8
)

# Generate video
result = pipe(
    prompt="A serene mountain lake at sunrise, mist rising from the water, "
           "birds flying overhead, cinematic quality",
    negative_prompt="blurry, low quality, distorted",
    height=704,
    width=1216,
    num_frames=121,
    frame_rate=24,
    num_inference_steps=30,
    cfg_guidance_scale=7.5,
    seed=42,
    enhance_prompt=True,  # Auto prompt enhancement
)

# Save
result.save("ltx2_output.mp4")

from ltx_pipelines import TI2VidTwoStagesPipeline
from PIL import Image

pipe = TI2VidTwoStagesPipeline.from_pretrained(
    "Lightricks/LTX-2",
    device_map="auto",
    enable_fp8=True,
)

# Load input image
input_image = Image.open("input_photo.jpg")

# I2V generation
result = pipe(
    prompt="The scene comes alive with gentle wind blowing through the trees",
    images=[input_image],
    height=704,
    width=1216,
    num_frames=121,
    frame_rate=24,
    num_inference_steps=30,
    cfg_guidance_scale=7.5,
    seed=42,
)

result.save("ltx2_i2v_output.mp4")

[Recommended Model by Scenario]

"I need the highest quality video"
  --> HunyuanVideo (v1, 13B)
  Reason: Full Attention + 13B for best visual quality

"I want to run locally on a consumer GPU"
  --> LTX-2 (FP8) or HunyuanVideo 1.5
  Reason: LTX-2 runs on 12GB, HV 1.5 on 14GB

"I need fast iterative work"
  --> LTX-2 (Distilled)
  Reason: Near real-time generation speed

"I need video with audio"
  --> LTX-2
  Reason: Only model with simultaneous AV generation

"I need specific character/style training"
  --> HunyuanVideo + LoRA
  Reason: Rich LoRA ecosystem

"I need 4K high resolution"
  --> LTX-2
  Reason: Native 4K support

"Human/face generation is important"
  --> HunyuanVideo
  Reason: Excellent Human Fidelity benchmark

[Subject] + [Action] + [Environment] + [Camera/Cinematography]

S (Subject):      Subject - what/who is the main focus
A (Action):       Action - what is happening
E (Environment):  Environment - where, what atmosphere
C (Camera):       Camera - how it is filmed

# Basic quality control
"blurry, low quality, distorted, deformed, ugly, bad anatomy,
watermark, text overlay, logo, grainy, noisy"

# Additional for human generation
"extra fingers, mutated hands, poorly drawn hands, poorly drawn face,
mutation, deformed, extra limbs, missing limbs"