Skip to content
Published on

Live Migration 2: The Real Meaning of Pre-copy, Post-copy, Dirty Pages, and Auto-converge

🇰🇷KO🇺🇸EN🇯🇵JA
Split View
Authors

Introduction

Now let us enter the truly difficult phase of live migration. Creating the target Pod was a Kubernetes control plane problem. From this point on, it becomes a data transfer problem -- how libvirt and QEMU decide to move the guest's memory, CPU state, and some disk state.

The core code for this phase is pkg/virt-launcher/virtwrap/live-migration-source.go. Here, KubeVirt builds libvirt migration flags, monitors progress, and triggers auxiliary strategies like post-copy or pause when necessary.

What Actually Gets Moved in the Migration Data Plane

Live migration does not mean "copying the entire VM at once." In practice:

  • Guest memory pages
  • CPU execution state
  • Some device state
  • Local volume-related data if needed

These are what get transferred.

If shared storage is available, the entire disk data does not need to be copied, but memory and execution state must continue to be transferred. The hardest problem here is that the guest keeps dirtying memory continuously.

What Is Pre-copy?

Pre-copy is the most basic migration strategy.

  1. The source VM continues running.
  2. Simultaneously, memory pages are copied to the target.
  3. Pages modified during copying are re-copied in the next round.
  4. When sufficiently converged, a brief stop-and-copy transition occurs.

The advantage is that the guest continues running for most of the time. The disadvantage is that if the guest modifies memory too quickly, the copy may never catch up.

This is where the concept of dirty page rate comes in.

Why Dirty Page Rate Matters

Looking at logMigrationInfo in live-migration-source.go, the following information is read from libvirt job stats:

  • DataRemaining
  • MemProcessed
  • MemRemaining
  • MemDirtyRate
  • Downtime

KubeVirt does not view migration as just "success or failure." It continuously observes how much remains and how fast the guest is re-dirtying memory.

If a VM is very write-heavy, copied pages may immediately become dirty again, preventing pre-copy from converging. High-performance databases and memory-intensive workloads are challenging here.

What Do the libvirt Migration Flags Tell Us?

Looking at generateMigrationFlags, you can see what modes KubeVirt requests from libvirt during migration.

  • MIGRATE_LIVE
  • MIGRATE_PEER2PEER
  • MIGRATE_PERSIST_DEST
  • MIGRATE_NON_SHARED_INC for block migration
  • MIGRATE_AUTO_CONVERGE when auto-converge is allowed
  • MIGRATE_POSTCOPY when post-copy is allowed
  • MIGRATE_PARALLEL for parallel migration

In other words, the migration method is not a vague concept but is ultimately concretized as a combination of libvirt flags.

What the Migration Monitor Does

migrationMonitor is the heartbeat that watches the data plane. Here KubeVirt:

  • Tracks remaining data
  • Determines if progress has stalled
  • Checks if acceptable completion time has been exceeded
  • Triggers post-copy or pause strategy if needed
  • Aborts if completely stuck

This structure is very practical. Migration is not "start and it goes to completion" -- it requires progress-based adaptive control.

When Does Post-copy Transition Happen?

The code shows that if migration has been stalled for a while, AllowPostCopy is enabled, and it is not a VFIO VMI, then dom.MigrateStartPostCopy is called. KubeVirt does not start with post-copy from the beginning -- it typically starts with pre-copy and transitions to post-copy if convergence fails.

Meaning of post-copy

  • The source no longer tries to pre-copy all pages
  • The target receives the active workload earlier
  • Required pages can be fetched later

The advantage is that even busy VMs are more likely to eventually complete the move. However, the downside is significant. After the active state moves to the target, if there is a network or target failure, the guest becomes more vulnerable.

That is why KubeVirt documentation and settings treat post-copy as a powerful but risky option.

Why Auto-converge and Pause Are Needed

Not all environments allow post-copy, and it is not suitable for all workloads. So KubeVirt uses other auxiliary strategies.

Auto-converge

A strategy where libvirt and QEMU deliberately impact guest performance to lower the dirty rate. Simply put, it is the approach of "slow down a bit but finish the migration."

Pause

The code shows that if post-copy is not allowed or not appropriate, the guest is temporarily suspended to complete the migration. This is a strategy that sacrifices downtime for completion.

In other words, migration is a triangular trade-off between performance, availability, and success probability.

Why Are VFIO Workloads More Challenging?

live-migration-source.go treats VFIO-based VMIs separately. Post-copy may not be generally supported, and instead, a very large max downtime is set to trigger QEMU internal switchover.

This means device passthrough workloads are much more difficult than regular VMs that just need memory transfer. This is why network SR-IOV or GPU passthrough workloads have significant migration constraints.

Criteria for Judging Migration as Stuck

The code makes a very practical judgment: "if there is no progress and network or QEMU connection issues are suspected, abort." It distinguishes between simply being slow and being completely stalled.

Two representative cases:

  • DataRemaining has not decreased during the progress timeout
  • Overall acceptable completion time is exceeded

The first is usually stuck, the second is closer to convergence failure.

When Disk Migration and Memory Migration Are Mixed

When block migration or volume migration is involved, difficulty increases further. Logic like classifyVolumesForMigration and configureLocalDiskToMigrate exists for this reason. Moving memory and disk simultaneously increases bandwidth, completion time, and failure surface.

The source code even has special error message improvement logic for when the destination volume is smaller than the source, showing that storage is a common cause of migration failure.

Common Misconceptions

Misconception 1: Live migration is zero-downtime

Not entirely. It minimizes downtime but is not fully zero-downtime. If convergence fails, it transitions to pause or post-copy, which introduces trade-offs.

Misconception 2: Post-copy is always better

No. It can increase the probability of success but the risk during failures is greater.

Misconception 3: Migration failures are mostly due to network

Network is one cause, but dirty rate being too high, local volume conditions not matching, or device passthrough constraints can also be the issue.

Key Metrics Operators Must Watch

  • Is remaining data decreasing?
  • Is the dirty rate high?
  • Has post-copy been triggered?
  • Has migration mode changed to paused or post-copy?
  • Is volume migration involved?

Watching these metrics lets you separate "why is migration slow" from "why did migration fail."

Conclusion

The data plane of KubeVirt live migration defaults to pre-copy, but when convergence fails, it uses auxiliary strategies like auto-converge, pause, and post-copy as an adaptive system. This structure is a realistic compromise to keep busy VMs alive as long as possible while eventually completing the move. Therefore, to properly understand migration, you must look at libvirt flags, dirty page rate, progress timeout, and post-copy transition conditions together.

In the next post, we will examine the network paths, sockets, and ports through which this data transfer occurs from the migration proxy perspective.

Discuss on Twitter • View on GitHub