💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

원문 렌더가 준비되기 전까지 텍스트 가이드로 표시합니다.

Introduction

The default state of a Kubernetes cluster is that every pod can talk to every pod. A payment service pod can reach the internal wiki pod, and a compromised frontend pod can connect straight to the database. The starting point of zero trust is flipping this default: only explicitly allowed communication is possible.

Standard NetworkPolicy gets you part of the way, but in practice you hit walls quickly. You cannot control HTTP paths, you cannot allow external APIs by domain name, and there is no way to see denied traffic. Cilium fills these gaps with CiliumNetworkPolicy (CNP) and CiliumClusterwideNetworkPolicy (CCNP). This article walks through the policy model, L3/L4/L7/DNS policy YAML, the default-deny rollout strategy, the Hubble-driven authoring workflow, and the common pitfalls — in the order you would tackle them in production.

Limits of Standard NetworkPolicy and What CNP Adds

| Capability | k8s NetworkPolicy | CiliumNetworkPolicy |

| --- | --- | --- |

| L3/L4 (pod selectors, ports) | Yes | Yes |

| L7 HTTP (method, path) | No | Yes |

| Kafka topics, gRPC methods | No | Yes |

| DNS-name based egress | No | Yes (toFQDNs) |

| Explicit deny rules | No (allowlist only) | Yes (ingressDeny/egressDeny) |

| Cluster-wide policy | No (namespace scoped) | Yes (CCNP) |

| Host (node) policy | No | Yes (nodeSelector) |

| Visibility into denied traffic | Implementation dependent | Immediate via Hubble |

| Entity concepts (world, host, etc.) | No | Yes |

An important premise: both CNP and standard NetworkPolicy are evaluated identity-based on the same eBPF datapath. When you mix the two kinds, the result is additive — if either allows, traffic is allowed — so deciding as a team which resource type is your standard reduces operational confusion.

The Policy Model — Identity and Direction

The mental model for Cilium policy evaluation is as follows.

ingress policy egress policy

"who may come to me" "where may I go"

[src identity] ----> (endpoint) ----> [dst identity/CIDR/FQDN]

| |

| O(1) verdict in the per-endpoint policy map

| key: (identity, port, proto, direction)

Verdict: ALLOW / DENY / (no policy) default allow*

* BUT: as soon as any policy selects an endpoint in a

given direction, that direction flips to default deny (!)

That last line is the rule that causes the most incidents in practice. The moment you apply even one ingress policy to a pod, that pod ingress becomes allowlist mode. Its egress remains fully open until an egress policy is applied. Remember that the flip happens independently per direction.

L3/L4 Policy — Practical YAML

Namespace isolation (allow only the same namespace)

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: payments

spec:

endpointSelector: {} # applies to all pods in the namespace

ingress:

- fromEndpoints:

- {} # allow all pods in the same namespace

An empty endpointSelector means all endpoints in this namespace, and an empty fromEndpoints entry means all endpoints in the same namespace. This single policy blocks all traffic arriving from outside the namespace.

Allowing specific services (frontend → backend 8080)

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: shop

spec:

endpointSelector:

matchLabels:

app: backend

ingress:

- fromEndpoints:

- matchLabels:

app: frontend

toPorts:

- ports:

- port: "8080"

protocol: TCP

Allowing pods from another namespace

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: shop

spec:

endpointSelector:

matchLabels:

app: backend

ingress:

- fromEndpoints:

- matchLabels:

k8s:io.kubernetes.pod.namespace: monitoring

app: prometheus

toPorts:

- ports:

- port: "9090"

protocol: TCP

To select across namespaces, use the `k8s:io.kubernetes.pod.namespace` label. It is more direct than the namespaceSelector of standard NetworkPolicy.

Explicit deny — egressDeny

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: shop

spec:

endpointSelector: {}

egressDeny:

- toCIDR:

- 169.254.169.254/32 # block the cloud metadata endpoint

Deny rules always take precedence over allow rules. They are suited to items that must be blocked no matter what, such as the cloud metadata server.

L7 Policy — HTTP, Kafka, gRPC

How it works: Envoy integration

To operate L7 policy you must understand how the traffic path changes.

Up to L4: client pod --eBPF--> server pod (handled in kernel)

L7 policy: client pod --eBPF--> [Envoy proxy] --> server pod

embedded in cilium-agent (or dedicated pod)

eBPF redirects only the affected flows

parses HTTP, matches rules, 403 on violation

eBPF hands only the flows carrying L7 rules to Envoy, so traffic without L7 policy is still processed entirely in the kernel. For the traffic that is L7-inspected, you must accept the proxy cost: added latency and connection termination.

Restricting HTTP methods and paths

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: shop

spec:

endpointSelector:

matchLabels:

app: order-api

ingress:

- fromEndpoints:

- matchLabels:

app: frontend

toPorts:

- ports:

- port: "8080"

protocol: TCP

rules:

http:

- method: GET

path: /api/v1/orders.*

- method: POST

path: /api/v1/orders

- method: GET

path: /healthz

Requests that do not match (for example DELETE, or the /admin path) still establish a connection but are rejected with HTTP 403. Unlike an L4 block, from the application log perspective this looks like "connection works but 403" — keep that difference in mind when troubleshooting.

Restricting Kafka topics

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: streaming

spec:

endpointSelector:

matchLabels:

app: kafka

ingress:

- fromEndpoints:

- matchLabels:

app: order-service

toPorts:

- ports:

- port: "9092"

protocol: TCP

rules:

kafka:

- role: produce

topic: orders

- fromEndpoints:

- matchLabels:

app: settlement-service

toPorts:

- ports:

- port: "9092"

protocol: TCP

rules:

kafka:

- role: consume

topic: orders

This example lets the order service only produce to the orders topic, and the settlement service only consume from it. In multi-tenant environments sharing a message broker, you can enforce topic-level isolation at the network layer.

Restricting gRPC methods

gRPC calls run over HTTP/2 in the form "POST /package.Service/Method", so they are expressed with HTTP rules.

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: shop

spec:

endpointSelector:

matchLabels:

app: inventory-grpc

ingress:

- fromEndpoints:

- matchLabels:

app: order-api

toPorts:

- ports:

- port: "50051"

protocol: TCP

rules:

http:

- method: POST

path: /inventory.InventoryService/CheckStock

- method: POST

path: /inventory.InventoryService/ReserveStock

DNS-Based Egress Policy — toFQDNs

Allowing external SaaS APIs by IP is unmaintainable (the IPs change constantly). toFQDNs allows egress by domain name.

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: payments

spec:

endpointSelector:

matchLabels:

app: pg-gateway

egress:

1) Allow DNS itself and make it observable via the DNS proxy

- toEndpoints:

- matchLabels:

k8s:io.kubernetes.pod.namespace: kube-system

k8s-app: kube-dns

toPorts:

- ports:

- port: "53"

protocol: UDP

rules:

dns:

- matchPattern: "*"

2) The external domains to allow

- toFQDNs:

- matchName: api.stripe.com

- matchPattern: "*.tosspayments.com"

toPorts:

- ports:

- port: "443"

protocol: TCP

How the DNS proxy works

pod --(DNS query)--> [Cilium DNS proxy] --> CoreDNS/external DNS

| intercepts A/AAAA records in the response

| "this pod learned api.stripe.com = 54.187.x.x"

registers that IP under the toFQDNs identity

in the ipcache/policy maps

pod --(TCP 443 to 54.187.x.x)--> eBPF allows based on the IP

The key insight: toFQDNs does not inspect the SNI of packets. It remembers **the pairing of the name this pod resolved via DNS and the answer IPs**, then allows by IP. Therefore, without the DNS rule (block 1 in the YAML above), toFQDNs never works. This is the single most common configuration mistake.

The Default-Deny Rollout Strategy — A Four-Stage Roadmap

Turning on default deny all at once in a live cluster causes outages. The proven incremental sequence is:

Stage 1: Observe Stage 2: Allow core Stage 3: Audit-mode Stage 4: Enforce

Watch all current paths. Turn findings deny. Deploy default Disable audit,

traffic via Hubble → into policies, apply → deny + policy-audit → actually block

(2-4 weeks) without any deny mode (violations (namespace by

(per-service PRs) only logged) namespace)

1. **Observe**: collect the real communication matrix per namespace from Hubble metrics and flow logs. Allow at least a month so you do not miss rarely-running traffic such as cron batch jobs.

2. **Author allow policies for core paths**: deploy allow policies first, with no deny policies. Nothing is blocked at this stage, so it is safe.

3. **Audit mode**: switching endpoints to policy-audit-mode means that even with a default deny policy in place, nothing is actually blocked — traffic that would have been blocked is recorded as a verdict.

Agent-wide audit mode (helm: policyAuditMode=true)

or for a specific endpoint only

kubectl -n kube-system exec ds/cilium -- cilium endpoint config 1234 PolicyAuditMode=Enabled

Observe audit verdicts: find traffic that would have been blocked

hubble observe --verdict AUDIT --namespace payments

4. **Enforce**: once audit shows no violations over a set period (e.g. two weeks), disable audit namespace by namespace. Never flip the entire cluster at once.

Deploy default deny itself explicitly, like this:

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: payments

spec:

endpointSelector: {}

ingress:

- fromEndpoints: [] # matches nothing = explicit default deny

egress:

- toEndpoints: []

The Policy Authoring Workflow — From Hubble to Policy

In practice, policies are not designed in your head; they are derived from observation.

1) Observe what the target service actually exchanges

hubble observe --namespace shop --pod shop/order-api --last 1000

2) Summarize which identities it talks to

hubble observe --namespace shop --pod shop/order-api \

--output json | jq -r '.flow.destination.labels | join(",")' | sort | uniq -c

3) Confirm nothing would be denied after applying (in audit mode)

hubble observe --verdict AUDIT --namespace shop

4) Monitor drops after enforcing

hubble observe --verdict DROPPED --namespace shop --since 1h

For drafting, the network policy editor (editor.networkpolicy.io) is useful. Upload Hubble flows and it visually generates a policy draft based on observed traffic. However, a human must always review the generated draft: legitimate traffic that did not occur during the observation window (failover paths, monthly batches) is missing from it.

Host Policy and External Entities

The entities concept

Reserved identifiers for dealing with the world outside the cluster.

| Entity | Meaning |

| --- | --- |

| world | Everything outside the cluster (including the internet) |

| cluster | All endpoints inside the cluster |

| host | The local node itself |

| remote-node | The other nodes |

| kube-apiserver | The API server |

| health | Cilium health-check endpoints |

Egress example allowing only in-cluster traffic plus the API server

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: internal-tools

spec:

endpointSelector: {}

egress:

- toEntities:

- cluster

- kube-apiserver

Host policy (CCNP + nodeSelector)

The traffic of the node itself can also be policed. An example allowing only SSH and the kubelet port:

apiVersion: cilium.io/v2

kind: CiliumClusterwideNetworkPolicy

metadata:

spec:

nodeSelector:

matchLabels:

node-role.kubernetes.io/control-plane: ""

ingress:

- fromEntities:

- cluster

- fromCIDR:

- 10.50.0.0/24 # management network

toPorts:

- ports:

- port: "22"

protocol: TCP

- port: "6443"

protocol: TCP

A badly written host policy can **lock you out of the entire node**, so it must be validated thoroughly in policy-audit-mode before enforcing. The host firewall feature (hostFirewall.enabled) must also be enabled in helm.

Automating Policy Testing and Verification

Policies are code too. Verification steps you can put into CI:

1) Schema/syntax validation (CI stage)

kubectl apply --dry-run=server -f policies/

2) Simulation: pre-verdict whether specific traffic would be allowed

kubectl -n kube-system exec ds/cilium -- \

cilium policy trace --src-k8s-pod shop:frontend-abc --dst-k8s-pod shop:backend-xyz --dport 8080

3) Real cluster integration test (staging)

cilium connectivity test --test pod-to-pod,pod-to-world

4) Regression test: compare drop counts after deployment

hubble observe --verdict DROPPED --since 10m --output json | jq length

In a GitOps setup, enforce steps 1 and 2 in the pipeline for PRs against the policy directory, and automatically run steps 3 and 4 in staging after merge.

Common Pitfalls and Anti-Patterns

1. **toFQDNs without a DNS rule**: as shown above, without the DNS proxy rule, toFQDNs never matches. The symptom is "DNS works but the connection is refused", or the reverse.

2. **DNS TTL and IP churn**: toFQDNs is based on DNS answers, so an application holding a stale DNS cache and connecting to an expired IP can get denied. For targets with fast IP rotation like CDNs, use broader matchPattern entries and review the agent FQDN TTL settings (tofqdns-idle-connection-grace-period and friends).

3. **Locking down system namespaces along with the rest**: hastily applying default deny to kube-system, monitoring, or ingress controller namespaces kills cluster functionality itself. Treat them as a separate track — last, and most carefully.

4. **Blocking health checks/probes**: kubelet liveness/readiness probes come from the node (host identity). Forget the host entity in an ingress policy and pods fall into endless restarts. Cilium auto-allows probe traffic by default, but combinations with host policy can break this.

5. **The mystery of only new connections breaking**: right after applying a policy, existing connections remain in conntrack and keep working while only new connections are blocked (or vice versa). "It works right now" is not evidence that "the policy allows it".

6. **Label typos and the empty-selector misunderstanding**: a typo in matchLabels fails silently as "selects nothing". After applying, always check the number of endpoints in enforcing state with `cilium endpoint list`.

7. **Spraying L7 policy on all traffic**: sending everything through Envoy raises latency and CPU together. The standard practice is selective application only at boundaries that truly need L7 control (externally exposed APIs, sensitive data access).

A Real-World Scenario — PCI DSS Style Isolation

A pattern for isolating workloads handling card payment data (the CDE) inside a cluster. The regulatory phrasing is a generalized example; confirm actual assessment requirements with your QSA.

1) CDE namespace: default deny + explicit allows only

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

namespace: cde-payments

spec:

endpointSelector: {}

ingress:

Only payment requests from the API gateway, with L7 path restrictions

- fromEndpoints:

- matchLabels:

k8s:io.kubernetes.pod.namespace: gateway

app: api-gateway

toPorts:

- ports:

- port: "8443"

protocol: TCP

rules:

http:

- method: POST

path: /v1/payments

- method: GET

path: /v1/payments/[0-9a-f-]+

egress:

DNS (observed via the proxy)

- toEndpoints:

- matchLabels:

k8s:io.kubernetes.pod.namespace: kube-system

k8s-app: kube-dns

toPorts:

- ports:

- port: "53"

protocol: UDP

rules:

dns:

- matchPattern: "*.internal.example.com"

- matchName: api.pgprovider.com

The internal ledger DB (dedicated namespace)

- toEndpoints:

- matchLabels:

k8s:io.kubernetes.pod.namespace: cde-db

app: ledger-db

toPorts:

- ports:

- port: "5432"

protocol: TCP

Only the external payment provider API

- toFQDNs:

- matchName: api.pgprovider.com

toPorts:

- ports:

- port: "443"

protocol: TCP

Add long-term retention of Hubble flow logs to submit as audit evidence that isolation was actually maintained (covered in the next article), and you have a complete technical answer to network segmentation requirements.

Adoption Checklist

- [ ] Has the team standardized on either standard NetworkPolicy or CNP

- [ ] Does the whole team understand that one policy flips that direction to default deny

- [ ] Have you secured a Hubble observation window (at least 2-4 weeks, covering batch cycles)

- [ ] Does every toFQDNs policy have a paired DNS proxy rule

- [ ] Is a policy-audit-mode validation stage part of the deployment procedure

- [ ] Are system namespaces split into a separate track

- [ ] Are dry-run and policy trace enforced via CI on policy PRs

- [ ] Are DROPPED verdict alerts wired into monitoring after enforcement

- [ ] Are host policies enforced only after audit validation

- [ ] Is the policy change history traceable via GitOps

Closing

The power of Cilium policy lies not only in its expressiveness (L7, FQDN) but in the fact that **observation and policy come from the same datapath**. The flows Hubble shows you and the flows the policy engine judges are identical, so the loop of observe → codify → audit → enforce closes without guesswork. Zero trust is not a single big bang; it is the practice of running this loop one namespace at a time. In the next article we cover Hubble, the observation axis of this loop, and ClusterMesh, which extends it to multiple clusters.

References

- Cilium network policy documentation: https://docs.cilium.io/en/stable/security/policy/

- Cilium L7 policy (HTTP/Kafka) documentation: https://docs.cilium.io/en/stable/security/policy/language/

- Cilium DNS-based policy documentation: https://docs.cilium.io/en/stable/security/dns/

- Kubernetes NetworkPolicy documentation: https://kubernetes.io/docs/concepts/services-networking/network-policies/

- Network policy editor: https://editor.networkpolicy.io/

- NIST SP 800-207 Zero Trust Architecture: https://csrc.nist.gov/pubs/sp/800/207/final

- PCI Security Standards Council: https://www.pcisecuritystandards.org/

- Envoy proxy documentation: https://www.envoyproxy.io/docs

- Apache Kafka documentation: https://kafka.apache.org/documentation/

- gRPC documentation: https://grpc.io/docs/

- Hubble GitHub repository: https://github.com/cilium/hubble