Split View: 폐쇄망 이미지 반입 파이프라인 설계 — skopeo, Harbor, 그리고 썩지 않는 재반입 런북

Designing an Air-Gapped Image Import Pipeline — skopeo, Harbor, and a Reimport Runbook That Doesn't Rot

• Introduction — A Single Import Always Rots
• Manage the Import List as Code
• External Collection — Pulling It Down With skopeo sync
  • Things That Aren't Images — oras
• The Inspection Stage — Bringing In the Trivy DB by Hand
• Signatures and SBOMs — cosign Where There's No Transparency Log
• Internal Distribution — the Two-Registry Pattern and Harbor
  • Carrying Helm Charts as OCI Artifacts
• The Periodic Reimport Runbook
• Closing — the Pipeline's Lifespan Is the List File's Lifespan
• References

Introduction — A Single Import Always Rots

There's no shortage of air-gapped installation guides. Most of them only cover getting through the first install, though. What actually wears a team down is everything that comes after.

Three months later, a new service ships and twelve more images turn out to be needed. Six months later, a vulnerability scan comes back, but the in-house scanner's vulnerability DB is half a year old, so the results can't be trusted. Nine months later, you need to bump the base image, and nobody can reconstruct who brought which tag in through which path, and when. The person who did the original import is already on a different team.

This post is about that problem. It's about not how to bring something in once, but how to design a structure that brings things in repeatedly. Here are the tools this was checked against.

Manage the Import List as Code

The first thing this pipeline needs isn't a tool — it's a list. If that list lives in someone's head or on a wiki page, six months from now it's guaranteed to be out of sync with reality. Keep it in a Git repository, and make sure an import job can only start from a commit to that file.

The most practical approach is to treat the format skopeo can read as the single source of truth, since it needs no separate conversion script.

# images.yaml — the source of truth for the import list. A diff to this file IS the import request.
docker.io:
  images:
    library/postgres:
      - '16.4'
      - '16.6'
    library/redis:
      - '7.4.1'
  images-by-tag-regex:
    library/busybox: ^1\.36.*$
registry.k8s.io:
  images:
    ingress-nginx/controller:
      - 'v1.12.0'
    metrics-server/metrics-server:
      - 'v0.7.2'
quay.io:
  tls-verify: true
  images:
    prometheus/prometheus:
      - 'v3.1.0'
    prometheus/node-exporter:
      - 'v1.8.2'
ghcr.io:
  images:
    aquasecurity/trivy:
      - '0.58.1'

This format matches the YAML source format defined in the skopeo sync manual exactly. It supports the keys images, images-by-tag-regex, images-by-semver, credentials, tls-verify, and cert-dir. It matters that you can specify tags by regex or semver range, but in an air-gapped setting, it's better to avoid range specifications as much as possible and pin tags exactly. Use a range and what actually comes in changes every time you import, and then the list file can no longer serve as the source of truth.

There's one more thing that absolutely must be recorded alongside the list: the digest.

# Pin, alongside the list, the digest that the tag points to
skopeo inspect docker://docker.io/library/postgres:16.6 \
  | jq -r '.Digest' \
  | tee -a DIGESTS.txt

Tags move. Pull the same tag again three months later and different content can come down. To answer the question an air-gap audit will ask — "is this the same thing you imported last time?" — you need a digest record.

External Collection — Pulling It Down With skopeo sync

The collection stage runs on a DMZ device that has internet access. The key thing here is not using a Docker daemon. skopeo talks directly to the registry with no daemon, so you can keep the staging device used for collection at a minimal footprint.

#!/usr/bin/env bash
# collect.sh — DMZ collection device
set -euo pipefail

BATCH="$(date +%Y%m%d)"
OUT="/staging/inbound/${BATCH}"
mkdir -p "${OUT}"

# Pull from several registries at once using a single list file.
# --scoped prefixes the destination path with the source registry path, so names don't collide.
skopeo sync \
  --src yaml \
  --dest dir \
  --scoped \
  --all \
  --keep-going \
  images.yaml "${OUT}"

du -sh "${OUT}"
find "${OUT}" -maxdepth 3 -type d | head -30

Each flag's meaning is exactly what's defined in the manual.

• --scoped — "since multiple images can share a name, prefix the source image path when storing at the destination." Essential when pulling same-named images from multiple registries.
• --all — when the source points to an image list (a multi-architecture manifest), copy all of them rather than just the one matching the current OS and architecture. Essential if your air-gapped network mixes amd64 and arm64 nodes.
• --keep-going — if a copy fails partway through, log it and keep going. Prevents a run pulling 200 images from stopping on the third one.
• --preserve-digests — preserves digests, and fails if it can't. Turn this on if import traceability matters.

The destination (--dest) is, per the manual, either docker or dir. Since you're carrying it out on physical media, use dir. Before putting it on the media, bundle it into an archive and attach a checksum.

# Archive and manifest for carrying on media
cd /staging/inbound
tar -cf "images-${BATCH}.tar" "${BATCH}"
sha256sum "images-${BATCH}.tar" > "images-${BATCH}.sha256"

# A snapshot of the list to attach to the import request
cp images.yaml "images-${BATCH}.manifest.yaml"
cp DIGESTS.txt "images-${BATCH}.digests.txt"

Things That Aren't Images — oras

Images aren't the only thing you need to bring in through the air gap. Helm charts, SBOMs, policy bundles, and vulnerability DBs are all needed too. Treat these as OCI artifacts and you can carry them through exactly the same path as images. The tool is oras.

# Push an arbitrary file in as an OCI artifact (as of oras 1.3)
oras push --artifact-type application/vnd.example.policy.v1+tar \
  registry.internal.example:5000/policies/kyverno:2026.07 \
  policies.tar.gz

# You can also specify a media type per file
oras push registry.internal.example:5000/bundles/edge:2026.07 \
  bundle.tar:application/vnd.example.bundle \
  README.md:text/markdown

# Pull down as an OCI layout directory — a form suited to carrying on media
oras push --oci-layout /staging/inbound/20260731/oci:policies-2026.07 policies.tar.gz

For copying between registries there's oras cp, and for checking a manifest there's oras manifest fetch. That said, as of this check, what I confirmed directly in the oras push documentation page's body was the push-family syntax and the OCI layout option. Check the exact flags for oras cp and oras pull on their own documentation pages before putting them in a script.

The Inspection Stage — Bringing In the Trivy DB by Hand

There needs to be an inspection stage between collection and distribution. What makes inspection hard in an air-gapped setting isn't the scanner itself — it's that the scanner's data normally updates from the internet. Trivy distributes its vulnerability DB as an OCI artifact, so it can be brought in the same way as an image.

#!/usr/bin/env bash
# collect-trivy-db.sh — DMZ collection device
set -euo pipefail
BATCH="$(date +%Y%m%d)"
mkdir -p "/staging/inbound/${BATCH}/trivy" && cd "/staging/inbound/${BATCH}/trivy"

# The repository and tags specified by the official documentation
oras pull ghcr.io/aquasecurity/trivy-db:2
oras pull ghcr.io/aquasecurity/trivy-java-db:1
oras pull ghcr.io/aquasecurity/trivy-checks:latest

ls -la
sha256sum ./* > TRIVY-DB.sha256

Once the media crosses back inside, push it up to the internal registry. The media type matters here. Trivy identifies its layers by custom media type, so if you push it in as a plain file, Trivy can't read it.

# Inside the air gap — push into the internal registry (in the form shown by the official docs)
oras push registry.internal.example:5000/trivy/trivy-db:2 db.tar.gz
oras push registry.internal.example:5000/trivy/trivy-java-db:1 javadb.tar.gz
oras push registry.internal.example:5000/trivy/trivy-checks:latest ./checks/

# Scan — point it at the internal repositories
trivy image \
  --db-repository registry.internal.example:5000/trivy/trivy-db \
  --java-db-repository registry.internal.example:5000/trivy/trivy-java-db \
  --checks-bundle-repository registry.internal.example:5000/trivy/trivy-checks \
  registry.internal.example:5000/apps/api:1.4.2

There's something to be honest about here. Flags like --skip-db-update, --skip-java-db-update, and --offline-scan are widely used, but as of this check, I could not confirm the exact names and behavior of these flags in the body of the Trivy air-gap documentation page. The same goes for the default cache directory path. Check these directly against the Trivy binary you actually imported before putting them into a pipeline.

# Confirm the actual flag names against the binary you imported — don't guess
trivy image --help | grep -iE 'db|offline|cache'
trivy --version

One more thing. The Trivy docs explain that the check bundle is embedded in the Trivy binary at build time, as a fallback used when the external DB isn't available. That means a configuration error can silently fall back to a stale embedded bundle for misconfiguration checks. If a scan result looks suspiciously clean, suspect this path.

Signatures and SBOMs — cosign Where There's No Transparency Log

Signature verification is only half-functional in an air gap. cosign's default behavior is to cross-check a signature against a transparency log, and in an air-gapped setting, that log is unreachable. So you have to take a different approach.

The most reliable method is to use your own key pair and include the public key in what you bring in.

# DMZ segment — sign an image that passed inspection with the in-house key
cosign sign --key /secure/cosign.key \
  registry.dmz.example:5000/apps/api@sha256:abc123...

# Include the public key together in the import manifest
cp /secure/cosign.pub "/staging/inbound/${BATCH}/cosign.pub"

# Inside the air gap — verify with the local public key
cosign verify --key /etc/cosign/cosign.pub \
  registry.internal.example:5000/apps/api:1.4.2

The situation is different if you need to verify something the original provider signed. The official documentation offers a path for verifying a locally downloaded image, and an option to skip the transparency-log check.

# Verify a locally downloaded image
cosign verify --key cosign.pub --local-image /staging/inbound/20260731/apps-api

# Skip the transparency-log check and verify only the key and payload
cosign verify --check-claims=false --key cosign.pub registry.internal.example:5000/apps/api:1.4.2

That said, as of this check, I could not confirm --insecure-ignore-tlog, --private-infrastructure, or the procedure for initializing an offline TUF root from a mirror in the body of the sigstore verification documentation. If your pipeline needs these three, check them directly against the help output of the cosign binary you imported and the sigstore documentation before applying them. Filling in a verification procedure with a guess is worse than not verifying at all. All you're left with is a record saying "we verified it," when in reality nothing was actually confirmed.

SBOMs take priority over signatures. Without an SBOM in an air-gapped setting, when a new vulnerability is disclosed six months from now, there's no way to answer "does our cluster have that package?" Re-scanning images requires re-importing the scanner DB, and you can't even investigate an image that's already been deleted.

# Generate the SBOM at collection time and carry it on the media
trivy image --format cyclonedx \
  --output "sbom/apps-api-1.4.2.cdx.json" \
  registry.dmz.example:5000/apps/api:1.4.2

# Import the SBOM together, as an OCI artifact
oras push --artifact-type application/vnd.cyclonedx+json \
  registry.internal.example:5000/sbom/apps-api:1.4.2 \
  "sbom/apps-api-1.4.2.cdx.json"

There's also a way to attach an SBOM to an image as a reference (referrer), but the registry has to support the referrers API. If support isn't certain, it's safer to upload it as a separate repository path with a tag, as shown above. Lookups stay simple and it works on any registry.

Internal Distribution — the Two-Registry Pattern and Harbor

The structure of an import pipeline boils down to two registries and an inspection segment between them.

The most common design mistake is leaving the distribution registry open for developers to push to directly. The moment you do that, the two-registry pattern collapses, and images that bypassed the inspection segment enter the cluster. The only entity with write access to the distribution registry should be a single import-pipeline account.

If you're using Harbor as the internal distribution registry, the latest release as of this check is 2.14.0. Installing Harbor in an air gap requires the offline installer, and Harbor's own container images are included in that installer. Don't forget to put the Harbor installer on your import list.

There's something that must be pointed out here. Harbor's proxy cache feature is useless in a fully air-gapped network. A proxy cache works by forwarding a request up to a parent registry and caching the result, which presupposes a network path to that parent registry. With no path, a cache miss is simply a failure. Where this feature is useful is a semi-air-gapped setting — "the internet works but we want to control it" — not an environment with no routing at all.

For the same reason, Harbor's replication feature doesn't cross the air-gap boundary either. Use replication to tidy things up between collection registries inside the DMZ, and only carry things across the boundary via physical media or a one-way gateway. As of this check, the individual documentation pages for Harbor's replication settings and proxy cache had moved URLs and I could not confirm the body text, so check details like supported source registry types and trigger methods directly in the Harbor 2.14 documentation.

Carrying Helm Charts as OCI Artifacts

Manage charts in a separate chart repository and you add one more import path. Unify on OCI artifacts and you use the same registry, the same authentication, and the same import procedure as images.

# DMZ collection — pull down external charts as .tgz
helm pull oci://registry-1.docker.io/bitnamicharts/postgresql --version 16.4.5 -d ./charts
helm pull https://prometheus-community.github.io/helm-charts/prometheus-25.27.0.tgz -d ./charts
sha256sum ./charts/*.tgz > CHARTS.sha256

# Inside the air gap — push to the internal registry
helm registry login registry.internal.example:5000
helm push ./charts/postgresql-16.4.5.tgz oci://registry.internal.example:5000/charts
helm push ./charts/prometheus-25.27.0.tgz oci://registry.internal.example:5000/charts

# Install — an oci reference requires a version to be specified
helm show all oci://registry.internal.example:5000/charts/postgresql --version 16.4.5
helm template pg oci://registry.internal.example:5000/charts/postgresql --version 16.4.5
helm install pg oci://registry.internal.example:5000/charts/postgresql --version 16.4.5

You can't stop at importing just the chart. The images the chart references have to be imported separately. This omission is the most common incident in air-gapped settings. Build a step into your pipeline that extracts image references the moment a chart is received and reflects them into the list file.

# Extract the list of images a chart references — the basis for updating the import list
helm template tmp ./charts/postgresql-16.4.5.tgz \
  | grep -E '^\s+image:' \
  | awk '{print $2}' \
  | tr -d '"' \
  | sort -u

Note that the referenced images depend on your values file. You have to extract using the actual values you'll deploy with, for this to be accurate. Miss a conditionally enabled sidecar or init container and you won't discover it until deployment day.

helm template tmp ./charts/postgresql-16.4.5.tgz -f values-prod.yaml \
  | grep -E '^\s+image:' | awk '{print $2}' | tr -d '"' | sort -u

The Helm documentation page is, as of this check, based on Helm 4.2.3, and that page carries a warning that it hasn't been fully updated for Helm 4 yet. If the Helm version you're importing differs from the documentation's version, verify command behavior in staging first.

The Periodic Reimport Runbook

Everything up to here has been the structure. From here on is this post's actual conclusion. No matter how well you build the pipeline above, if it doesn't run periodically, it's useless within six months. Different assets have different shelf lives, so you can't bundle them under a single cadence.

Put this table on a calendar and assign an owner. In an air-gapped setting, a plan of "we'll do it when we need to" always ends up as "it takes three weeks once we need it."

The skeleton of a script that automates reimport looks like this. The key is computing the diff against the last import first. Carry the full set across every time and neither the media capacity nor the review time can keep up.

#!/usr/bin/env bash
# reimport.sh — runs periodically on the DMZ collection device
set -euo pipefail

BATCH="$(date +%Y%m%d)"
PREV="$(ls -1d /staging/inbound/20* | sort | tail -1)"
OUT="/staging/inbound/${BATCH}"
mkdir -p "${OUT}"

echo "== 1. Computing digests for the current tags"
: > "${OUT}/DIGESTS.txt"
while read -r ref; do
  [ -z "${ref}" ] && continue
  d=$(skopeo inspect "docker://${ref}" 2>/dev/null | jq -r '.Digest') || d="ERROR"
  echo "${ref} ${d}" >> "${OUT}/DIGESTS.txt"
done < refs.txt

echo "== 2. Diff against the previous import"
if [ -f "${PREV}/DIGESTS.txt" ]; then
  diff "${PREV}/DIGESTS.txt" "${OUT}/DIGESTS.txt" > "${OUT}/CHANGES.diff" || true
  CHANGED=$(grep -c '^>' "${OUT}/CHANGES.diff" || true)
  echo "Changed references: ${CHANGED}"
  if [ "${CHANGED}" -eq 0 ]; then
    echo "No changes — skipping this round's import"
    exit 0
  fi
fi

echo "== 3. Collecting only the changed set"
skopeo sync --src yaml --dest dir --scoped --all --keep-going images.yaml "${OUT}/images"

echo "== 4. Scanning and SBOM"
mkdir -p "${OUT}/sbom" "${OUT}/scan"
while read -r ref _; do
  name=$(echo "${ref}" | tr '/:' '__')
  trivy image --format cyclonedx --output "${OUT}/sbom/${name}.cdx.json" "${ref}" || true
  trivy image --severity HIGH,CRITICAL --format json \
    --output "${OUT}/scan/${name}.json" "${ref}" || true
done < "${OUT}/DIGESTS.txt"

echo "== 5. Archive for review submission"
cd /staging/inbound
tar -cf "batch-${BATCH}.tar" "${BATCH}"
sha256sum "batch-${BATCH}.tar" > "batch-${BATCH}.sha256"
echo "Ready to submit: batch-${BATCH}.tar"

The internal-import side needs a script at the same level of rigor. Checksum verification, pushing to the distribution registry, and logging to the import ledger all need to finish in one shot, so a human never gets the chance to skip a step.

#!/usr/bin/env bash
# ingest.sh — inside the air gap
set -euo pipefail
BATCH="$1"
SRC="/media/inbound/batch-${BATCH}.tar"

sha256sum -c "/media/inbound/batch-${BATCH}.sha256"
mkdir -p "/opt/inbound" && tar -xf "${SRC}" -C /opt/inbound

# Push images that came in as a directory into the distribution registry
skopeo sync --src dir --dest docker \
  "/opt/inbound/${BATCH}/images" registry.internal.example:5000/mirror/

# Log to the import ledger — the only basis later for "when did what come in"
{
  echo "batch=${BATCH} at=$(date -Iseconds) by=${USER}"
  cat "/opt/inbound/${BATCH}/DIGESTS.txt"
} >> /var/log/airgap-ingest.log

Closing — the Pipeline's Lifespan Is the List File's Lifespan

What survives longest in an import pipeline isn't the scripts — it's the list file. Scripts get rewritten as tool versions change, but a list saying "these are the artifacts our cluster needs" lasts for years. Keep that list in Git with digests recorded alongside it, and the pipeline keeps running even after the person responsible has changed three times over.

And whenever you're evaluating a feature like a proxy cache or replication, always ask the same question first: does this feature presuppose a network path going upstream? If it does, it doesn't hold up in an air-gapped network. This one question alone will save you two hours of architecture-meeting time.

References

• skopeo-sync manual
• ORAS command documentation
• Trivy Air-Gapped Environment
• Trivy Self-Hosting Databases
• Harbor 2.14 documentation
• Helm OCI Registries
• Cosign signature verification