Split View: 폐쇄망 운영 플레이북 — 보안 패치를 따라가고, 롤백하고, CVE 지연을 관리하기

The Air-Gapped Operations Playbook — Keeping Up with Security Patches, Rolling Back, and Managing CVE Lag

• Opening — you cannot remove the lag, but you can manage it
• First check — is updateinfo still alive
• Counting what is backed up
• Applying it
• Turning the difference between two snapshots into review material
• Managing mirror capacity
• Rollback — what can be reversed and what cannot
• Designing the transfer cadence
• The operations checklist
• Closing — do not hide the lag
• Try it yourself
• The full series
• References

Opening — you cannot remove the lag, but you can manage it

Security response on a connected server looks like this. An advisory comes out, you run dnf update --security that same day, and you are done.

An air-gapped network looks like this. An advisory comes out, you wait for the next transfer cycle, you fetch the packages on the outside, you file for review, you carry the medium in, you refresh the repository, and you apply it to the servers. A few days at best, usually a few weeks.

That lag comes out of the structure. No amount of polishing the procedure drives it to zero. So the goal of this post is not to remove the lag but to make it measurable, keep a separate emergency path, and make sure the organization knows about it.

First check — is updateinfo still alive

Most of the operations chapter starts from dnf updateinfo. And in an air-gapped network it is extremely common for this command to print nothing at all.

# Count what is backed up
dnf updateinfo --summary

# If nothing comes out, it is usually a repository-side problem

The reason has the same shape as the module problem from post 5. Which package belongs to which advisory lives in the repository metadata, not inside the rpm file. What createrepo_c generates from rpms is primary, filelists, and other; the advisory information is not inside the rpm, so it cannot be manufactured. The fact that the createrepo_c man page carries a separate --keep-all-metadata ("Keep additional metadata during update (default)") is itself because this additional metadata is a separate thing.

So it splits like this.

• If you fetched with dnf reposync --download-metadata, the advisory information comes along with it. In that case, do not run createrepo_c again after the transfer. Rebuilding loses it
• If you gathered only rpms with dnf download and built the repository with createrepo_c, there is no advisory information. In that repository the entire --security family becomes meaningless

Run dnf update --security in the second state and you get "nothing to do" with no error. Silently doing nothing is what makes it dangerous. Put this one line into the verification step of your transfer procedure.

# Check whether the transferred repository holds advisory information — zero means the procedure is wrong
dnf updateinfo --summary | tail -5

Counting what is backed up

If the advisory information is alive, taking stock is a few command lines. The notation is taken straight from the official RHEL 9 security documentation.

# Security updates not yet applied
dnf updateinfo list updates security

# Security updates already applied (for audit response)
dnf updateinfo list security --installed

# The details of a specific advisory (includes the CVE number and description)
dnf updateinfo info RHSA-2019:0997

# Filter down to advisories that reference a CVE
dnf updateinfo list --with-cve

Per the documentation, --with-cve is "print only advisories referencing a CVE" and --with-bz is "print only advisories referencing a bugzilla". There are four availability specifiers: --available (the default), --installed, --updates, and --all.

The output of dnf updateinfo list updates security in the RHEL 9 security documentation has this shape.

RHSA-2019:0997 Important/Sec. platform-python-3.6.8-2.el8_0.x86_64
RHSA-2019:0997 Important/Sec. python3-libs-3.6.8-2.el8_0.x86_64
RHSA-2019:0990 Moderate/Sec.  systemd-239-13.el8_0.3.x86_64

The second column is the severity. You can set transfer priority from it.

Applying it

# Apply every available security update
sudo dnf update --security

# Apply only a specific advisory
sudo dnf update --advisory=RHSA-2019:0997

# Apply a specific advisory with the smallest possible version change
sudo dnf upgrade-minimal --advisory=RHSA-2019:0997

# Apply by naming a specific CVE
sudo dnf update --cve=CVE-2019-9636

# Narrow the scope by severity
sudo dnf update --sec-severity=Important

The dnf documentation defines --security as "Includes packages that provide a fix for a security issue", --advisory=<advisory> as "Include packages corresponding to the advisory ID", --cve=<cves> as "Include packages that fix a CVE ID", and --sec-severity=<severity> as "Includes packages that provide a fix for an issue of the specified severity".

upgrade-minimal is especially useful in an air-gapped network. The documentation describes it as "Updates each package to the nearest available version that provides a bugfix, enhancement or a fix for security issue". Because it takes the security fix while keeping the version change to a minimum, the scope of change is narrower, review is easier to pass, and the regression risk drops. It is worth making the default for scheduled air-gapped patching.

There is one thing left to do after applying.

# Check which processes need restarting
dnf needs-restarting

The RHEL 9 documentation attaches an important qualifier to this command. It lists only processes, not services, so "you cannot restart processes listed using the systemctl utility". Which means you must not hand what it prints straight to systemctl.

Turning the difference between two snapshots into review material

The most tedious part of a scheduled transfer is answering the question "what is changing this time". dnf repodiff does that job. The documentation defines it as "repodiff is a program which will list differences between two sets of repositories".

# Compare the previous transfer snapshot with this one
dnf repodiff \
  --repofrompath=old,/srv/repo-snapshots/2026-07-14/baseos \
  --repofrompath=new,/srv/repo-snapshots/2026-08-15/baseos \
  --repo-old=old --repo-new=new \
  --archlist=x86_64 \
  --size

# As a one-line summary (good for attaching to the review document)
dnf repodiff \
  --repofrompath=old,/srv/repo-snapshots/2026-07-14/baseos \
  --repofrompath=new,/srv/repo-snapshots/2026-08-15/baseos \
  --repo-old=old --repo-new=new \
  --archlist=x86_64 --simple --downgrade

There is one default here you absolutely have to know. The documentation states "Note that by default only source packages are compared". If you do not specify --archlist, binary packages are not compared at all. The documentation describes --archlist, -a as "Add architectures to change the default from just comparing source packages. Note that you can use a wildcard * for all architectures".

As for the rest of the options, --size, -s is "Output additional data about the size of the changes", --simple is "Output a simple one line message for modified packages", --downgrade is "Split the data for modified packages between upgraded and downgraded packages", and --compare-arch brings the architecture, not just the name, into the comparison.

Attach this output directly to the transfer review, and the reviewer can judge "what is changing" on the basis of evidence.

Managing mirror capacity

Repeat the transfers and the repository keeps growing. But delete all the old versions and rollback becomes impossible. dnf repomanage balances the two. The documentation describes it as "repomanage prints newest or older packages in a repository specified by <path> for easy piping to xargs or similar programs". The important part is that it only prints — it does not delete.

# Keep only the two newest versions of each package and list the rest
dnf repomanage --old --keep 2 /srv/repo/rhel9-baseos

# Actually delete them (only after you have looked the list over yourself)
dnf repomanage --old --keep 2 /srv/repo/rhel9-baseos | xargs -r rm -v

# Rebuild the metadata after the cleanup
sudo createrepo_c --update /srv/repo/rhel9-baseos

Per the documentation, --old is "Show older packages (for a package or a stream show all versions except the newest one)", --new is "Show newest packages", and -k, --keep is "Limit the resulting set to newest <keep-number> packages".

How many you keep is exactly how far back you can roll. With a monthly transfer cycle and three kept, you can roll back roughly three months. Put capacity and rollback range on the same scale when you decide.

Rollback — what can be reversed and what cannot

This is the procedure for when something breaks after a patch.

# Check the transaction history
dnf history

# Check the contents of a specific transaction
dnf history info 42

# Undo just that one transaction
sudo dnf history undo 42

# Undo everything after a given point (the named transaction itself is kept)
sudo dnf history rollback 41

The RHEL 9 documentation describes the behaviour of undo this way. If the transaction installed packages it removes them, if it removed packages it reinstalls them, and for upgraded packages it attempts a downgrade only when the earlier version is still available. It then states outright that "If an older package version is not available, the downgrade by using the dnf history undo command fails".

In an air-gapped network that sentence connects directly to the retention policy from the previous section. Delete the old packages and the rollback fails.

And there is one boundary you have to respect. The RHEL 9 documentation warns as follows.

"Downgrading RHEL system packages to an older version by using the dnf history undo and dnf history rollback command is not supported. This concerns especially the selinux, selinux-policy-*, kernel, and glibc packages, and dependencies of glibc such as gcc."

In other words, the kernel, glibc, and the SELinux policy must not be handled by history reversal. Regression response for those has to be prepared through a different path rather than through rollback. For the kernel there is booting into the previous kernel, and for the rest, recovery at the snapshot or system image level is the realistic answer. Write this boundary into the procedure document and the decision at the moment of an incident gets much faster.

Designing the transfer cadence

This is the core design of this post. Keep only one path and you will fail, guaranteed. Keep two.

The key to the emergency path is agreeing on it in advance, in peacetime. Propose building a shortened procedure after an incident has already happened and that discussion alone eats several days.

Do not forget to measure the lag either.

#!/usr/bin/env bash
# patch-lag.sh — count the backed-up security advisories by severity
set -euo pipefail

echo "== Surveyed at: $(date -u +%Y-%m-%dT%H:%M:%SZ)"
echo "== Repository snapshot"
grep -h '^name=' /etc/yum.repos.d/airgap-*.repo || true

echo "== Backed-up security advisory summary"
if ! dnf updateinfo --summary 2>/dev/null | grep -q .; then
  echo "Warning: no updateinfo metadata. Check the transfer procedure."
  exit 2
fi
dnf updateinfo --summary

echo "== Detail by severity"
for sev in Critical Important Moderate Low; do
  n="$(dnf updateinfo list updates security 2>/dev/null | grep -c "${sev}/Sec." || true)"
  printf '%-10s %s\n' "${sev}" "${n}"
done

Put this output into your regular reporting and patch lag stops being a feeling and becomes a number. Once it is a number, the discussion about shortening the transfer cycle becomes possible.

Most of the procedures in the official Red Hat security documentation state "A Red Hat subscription is attached to the host" as a precondition. Air-gapped servers frequently do not satisfy that condition, so advisory information has to arrive through the metadata of the transferred repository. That is exactly why the first section above matters.

Redistributing Red Hat content without a subscription may violate your agreement, so check the licensing terms that apply to your organization first. Advisory metadata is Red Hat content too.

The operations checklist

This is meant to be pasted at the front of the procedure document and used from there.

Transfer preparation (connected side)

• Did you pin the release version (--releasever or /etc/dnf/vars/releasever)
• Did you resolve dependencies against an empty --installroot
• If this is a system that uses modules, did you specify module_platform_id
• Did you add --download-metadata to reposync (preserves advisory and module information)
• Did dnf repoclosure pass
• Did you generate MANIFEST.tsv (NEVRA, checksum, snapshot date, releasever)
• Did you include the GPG public key in the bundle
• Did you record the generating commands in PROVENANCE.txt
• Did you attach the dnf repodiff output as review material

After the transfer (air-gapped side)

• Did you check the manifest and the checksums against each other
• Did every rpmkeys --checksig pass
• Did you check whether the medium holds files that are not in the manifest
• Did you decide whether the repository metadata needs to be rebuilt (no regeneration if you fetched with --download-metadata)
• Is gpgcheck=1 stated explicitly in the .repo file
• Did you run dnf clean expire-cache and then confirm the package count with dnf repolist -v
• Is dnf updateinfo --summary non-empty
• Did you run dnf repoclosure again against the real repository configuration

Applying and afterwards

• Did you minimize the scope of change with dnf upgrade-minimal --security
• Did you deal with the dnf needs-restarting results
• Did you record the dnf history transaction ID in the operations log
• Did you confirm the rollback range (the repomanage --keep value)
• Did you exclude the kernel, glibc, and SELinux from history rollback
• Did you reflect the patch lag figures in the regular report

Closing — do not hide the lag

The most dangerous attitude in air-gapped operations is being embarrassed by patch lag and leaving it out of the report. The lag is the price of choosing an air-gapped network, and that choice was usually made by regulation rather than by technology.

What you have to do is turn the lag into numbers, open the emergency path ahead of time, and show those numbers regularly. That is what makes the discussion of "let us shorten the transfer cycle from monthly to every two weeks" happen on the basis of evidence rather than instinct.

The conclusions of all seven posts gather into one. In an air-gapped network, only what is repeatable survives. Manifests and checklists are what create that repetition.

Commands and options were verified against the official documentation on 2026-08-15. They differ by RHEL version, so confirm against the documentation for the version you are running.

Try it yourself

• Linux Terminal — assemble the patch lag measurement script yourself
• Linux Command Quiz — review the dnf operations commands
• Container Lab — design the image refresh cycle alongside the package cycle

The full series

1. The real reason air-gapped installation is hard
2. Fetching on the outside — dnf download, reposync, yumdownloader
3. Building a local repository — createrepo_c and GPG keys
4. The transfer procedure and integrity — a reproducible bundle
5. Modularity and version pinning
6. Bringing container images in
7. The operations playbook (this post)

Air-gapped Kubernetes operations continue in the Day 2 operations post.

References

• Red Hat: Managing and monitoring security updates (RHEL 9)
• Red Hat: Managing software with the DNF tool (RHEL 9)
• DNF Command Reference — updateinfo, upgrade-minimal, history
• DNF repodiff plugin
• DNF repomanage plugin
• createrepo_c man page source