Three ways to put a proxy cache in front of Docker Hub — registry:2, Nexus, ECR pull-through cache

The problem is the limit, not the speed

Every CI run pulls FROM python:3.12-slim from Docker Hub. Ten nodes means ten pulls; a pipeline that runs a hundred times a day means a hundred. Docker Hub is counting.

The limits Docker documents are these. An unauthenticated client gets 100 pulls per 6 hours per IPv4 address (or IPv6 /64), a personal account gets 200, and Pro, Team and Business are unlimited. When several machines leave through one NAT address — an office, a cluster — they all share those 100.

I did not take the numbers on faith; I read the headers. Docker keeps a repository called ratelimitpreview/test for exactly this, and a manifest request with an anonymous token returns the remaining count.

TOK=$(curl -s "https://auth.docker.io/token?service=registry.docker.io&scope=repository:ratelimitpreview/test:pull" | jq -r .token)
curl -s -I -H "Authorization: Bearer $TOK" \
  https://registry-1.docker.io/v2/ratelimitpreview/test/manifests/latest | grep -i ratelimit

ratelimit-limit: 100;w=3600
ratelimit-remaining: 85;w=3600

Over a few experiments remaining went 91 → 88 → 85. The header's window says w=3600, i.e. 3,600 seconds, while the documentation says six hours. Whichever is right, one thing is certain: this check request itself counts as a pull.

A proxy cache is the device that protects this number. When every node in the cluster asks my cache instead of Docker Hub, the same image comes down from upstream once.

The smallest experiment: registry:2 as a pull-through cache

The registry:2 image Docker ships turns into a read-only proxy with one environment variable.

docker run -d --name regmirror -p 5001:5000 \
  -v $PWD/data:/var/lib/registry \
  -e REGISTRY_PROXY_REMOTEURL=https://registry-1.docker.io \
  registry:2

Pulling localhost:5001/library/nginx:1.27-alpine now makes the cache fetch missing layers from Docker Hub, store them on disk, and hand them to the client. After confirming the local daemon had no nginx layers at all (docker images | grep nginx gave zero lines), I pulled the same image three times, deleting the local copy each time.

docker images reports the image as 76.8MB while the cache is 22M. The cache keeps compressed layer blobs; the daemon reports the unpacked size.

What the experiment says

The first pull gets slower. Fetching from upstream, writing to disk and passing it on is two steps, so it took nearly three times as long as pulling directly (2.86 s).

Even the second pull was not faster than going direct. From a home connection, Docker Hub direct took 2.86 s; through the filled cache it took about 4 s. The proxy container ran on the same laptop, so there is one more hop. With a fast line and a single client, a proxy cache is not a speed device.

What you gain is upstream request count. On the 2nd and 3rd pulls the cache directory did not grow by a byte. The layers were not downloaded from upstream again. On a twenty-node cluster, what Docker Hub counts approaches one instead of twenty. That is the value of a proxy cache.

One thing to record honestly. registry:2 does not log its upstream requests at the info level, so I could not count how many manifest checks each pull made. From "the disk did not grow" I can say only that no layer was re-downloaded.

Nexus: proxy, hosted, and group

A registry:2 proxy sees one upstream. A company usually wants to see Docker Hub, push its own images, and use both through one address. Sonatype Nexus Repository has three kinds of Docker repository.

• proxy — caches a remote registry. For Docker Hub, Remote storage is https://registry-1.docker.io and Docker Index is "Use Docker Hub" (search is served by index.docker.io, so the two are set separately).
• hosted — where I push.
• group — bundles several proxy and hosted repositories under one URL. The documentation calls a group "the recommended way to expose all your repositories for read access to your users", and members are added "in the desired order". A repository added to the group is visible at once without touching client configuration.

Client-side setup ends with registering one Repository Connector port (say nexus.example.com:8082) as the daemon's mirror. From then on docker pull nginx searches the group's members in order, and if hosted lacks it, the proxy fetches from Docker Hub and caches it.

When Nexus proxies ECR as an upstream, authentication is peculiar. ECR tokens last 12 hours; according to the Nexus documentation it caches that token on the first pull and refreshes it every six hours. Give Nexus AWS credentials once and it does the rest.

ECR pull-through cache: caching inside AWS

If the cluster is on AWS there is no need to run a cache; ECR does it. Write a pull through cache rule with the upstream and a prefix, and pulling <account>.dkr.ecr.<region>.amazonaws.com/<prefix>/library/nginx:1.27-alpine makes ECR fetch from upstream and keep it in a repository in my account.

The rules confirmed in the documentation:

• Upstreams that need no authentication are ECR Public, the Kubernetes container image registry, and Quay. Docker Hub, Azure Container Registry, GitHub and GitLab Container Registry, and Chainguard require a Secrets Manager secret; another account's ECR authenticates with an IAM role.
• That secret's name must start with ecr-pullthroughcache/ and live in the same account and region as the rule. Docker Hub is then pulled as an account, so you use that account's limit rather than the anonymous 100.
• When the same tag is pulled again, if ECR validated it against upstream within the last 24 hours it serves the cache without contacting upstream. Once the window has expired it checks for a newer version and updates. If you deploy by overwriting tags, you can receive an old image for up to a day.
• If the upstream update fails, the last cached image is still served. Deployments from already-pulled images keep working even when Docker Hub is down.
• Turning on tag immutability on the cache repository blocks updates to the same tag.
• A multi-architecture image pulls the manifest list and every architecture in it. To get one, pull by that architecture's digest.
• The first pull may need a route to the internet because ECR must reach upstream. A VPC with only a PrivateLink endpoint can fail on the first pull, so the documentation recommends setting up a route. Later pulls do not need it.
• AWS Lambda does not support images created through a pull-through cache rule.

Which to choose

In our pipeline

LabHub's build has Jenkins build the image with kaniko, push it to a private Harbor, and ArgoCD deploy that tag. The base image is pinned by digest in the Dockerfile.

FROM python:3.12-slim@sha256:78387bc3…

A proxy cache in front changes nothing about this line. A digest is a hash of the content, so whichever cache it passes through, the bytes must be identical to pass. The cache protects the limit; the digest protects the content. The two devices do different jobs.

In one line

A proxy cache is not a device for faster pulls; it is a device for reducing what upstream counts to one. With a few nodes, registry:2 is enough; with several repositories, make one address with a Nexus group; inside AWS, let ECR do the caching.