💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

원문 렌더가 준비되기 전까지 텍스트 가이드로 표시합니다.

Cilium Network Policy Engine: L3/L4/L7 Filtering Implementation

Overview

Cilium's network policy engine extends Kubernetes NetworkPolicy to provide fine-grained traffic control at L3/L4/L7 levels. It combines eBPF with Envoy proxy in a hybrid architecture that achieves both high performance and flexibility.

1. Policy Repository and Rule Compilation

1.1 Policy Sources

Cilium collects network policies from multiple sources:

Policy sources:

1. Kubernetes NetworkPolicy

- Standard L3/L4 policies

- Automatically translated and applied by Cilium

2. CiliumNetworkPolicy (CNP)

- Namespace-scoped Cilium extended policy

- Advanced features: L7, FQDN, Identity-based

3. CiliumClusterwideNetworkPolicy (CCNP)

- Cluster-wide policies

- Default deny policies, common security rules

4. CiliumEnvoyConfig (CEC)

- Envoy proxy L7 traffic management rules

1.2 Policy Repository

The policy repository within the Agent manages all policies in a unified manner:

Policy collection flow:

K8s API Server

[Watcher: NetworkPolicy] -> Policy Repository

[Watcher: CNP] -> Policy Repository

[Watcher: CCNP] -> Policy Repository

[Policy Computation Engine]

[Per-Endpoint Policy Set]

[BPF Program Compilation]

1.3 Translation from Policy to BPF Maps

CiliumNetworkPolicy YAML

Parse and normalize policy

Selector matching: determine affected endpoints

Generate Identity-based allow list

Update BPF policy maps:

Add entries to cilium_policy_<endpoint_id> map

Key: (Identity, port, protocol)

Value: (allow/deny, proxy port)

Regenerate endpoint BPF programs (if needed)

2. Identity-Based vs IP-Based Enforcement

2.1 Identity-Based Enforcement

This is Cilium's default policy enforcement method, using numeric Identities instead of IP addresses:

Traditional IP-based firewall:

Allow: 10.244.1.5 -> 10.244.2.10:8080

Problem: IP changes when Pod restarts

Cilium Identity-based:

Allow: Identity 48291 (app=frontend) -> Identity 52103 (app=backend):8080

Advantage: Policy persists regardless of IP changes

Check policy map contents (by endpoint ID)

cilium bpf policy get 1234

Example output:

POLICY DIRECTION IDENTITY PORT/PROTO PROXY PORT ENTRY TYPE

Allow Ingress 48291 8080/TCP 0 allow

Allow Ingress 52103 443/TCP 0 allow

Allow Egress 0 53/UDP 0 allow (DNS)

2.2 Policy Map Structure

// Policy map key (conceptual)

struct policy_key {

__u32 identity; // Source/destination Identity

__u16 dport; // Destination port (0 = all ports)

__u8 protocol; // TCP, UDP, ICMP

__u8 direction; // ingress(1), egress(2)

};

// Policy map value

struct policy_entry {

__u8 action; // allow(1), deny(2), audit(3)

__u16 proxy_port; // L7 proxy port (0 = no proxy)

__u8 auth_type; // Auth type (mTLS, etc.)

__u64 packets; // Matched packet counter

__u64 bytes; // Matched byte counter

};

2.3 Policy Lookup Algorithm

Policy lookup on packet receive:

1. Look up Identity from source IP (ipcache)

2. Search in policy map for (Identity, Port, Protocol) combination

3. Exact match attempt:

- Check (src_identity, dst_port, protocol)

4. Wildcard match:

- Check (src_identity, ANY_PORT, protocol)

- Check (ANY_IDENTITY, dst_port, protocol)

5. Check Deny rules first:

- If Deny exists, unconditionally deny

6. Result: Allow / Deny / Audit

3. L3/L4 Enforcement: High-Speed Filtering in eBPF

3.1 L3 Policy (Network Layer)

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: backend

ingress:

- fromEndpoints:

- matchLabels:

app: frontend

- fromCIDR:

- 10.0.0.0/8

- fromEntities:

- world

egress:

- toEndpoints:

- matchLabels:

app: database

- toCIDR:

- 0.0.0.0/0

toCIDRSet:

- cidr: 0.0.0.0/0

except:

- 169.254.169.254/32

3.2 L4 Policy (Transport Layer)

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: backend

ingress:

- fromEndpoints:

- matchLabels:

app: frontend

toPorts:

- ports:

- port: '8080'

protocol: TCP

- port: '8443'

protocol: TCP

egress:

- toEndpoints:

- matchLabels:

app: database

toPorts:

- ports:

- port: '5432'

protocol: TCP

3.3 L3/L4 Enforcement Flow in eBPF

Packet: src=10.244.1.5:34567 dst=10.244.2.10:8080 TCP

1. [from-container or to-container BPF program]

2. Source Identity lookup:

ipcache[10.244.1.5] -> Identity: 48291

3. Policy map lookup:

Search in policy_map[endpoint_id]:

key = (identity=48291, port=8080, proto=TCP, dir=ingress)

4. Result:

- Allow: Continue packet delivery

- Deny: Drop packet + monitor event

- L7 Redirect: Redirect to Envoy proxy port

4. L7 Enforcement: Envoy Proxy Integration

4.1 L7 Policy Overview

L7 policies provide application protocol-level filtering:

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: api-server

ingress:

- fromEndpoints:

- matchLabels:

app: frontend

toPorts:

- ports:

- port: '8080'

protocol: TCP

rules:

http:

- method: GET

path: '/api/v1/users'

- method: GET

path: '/api/v1/products'

- method: POST

path: '/api/v1/orders'

headers:

- 'Content-Type: application/json'

4.2 L7 Redirect Mechanism

Traffic flow with L7 policy applied:

Pod A -> [from-container BPF]

v (policy map: L7 proxy redirect needed)

[tc redirect to Envoy proxy port]

[Envoy Proxy (1 per node)]

- HTTP/gRPC/Kafka parsing

- L7 rule matching

- Allow/deny decision

v (if allowed)

[Forward to destination via BPF]

Pod B

4.3 Supported Protocols

| Protocol | Filterable Fields |

| -------- | -------------------------------------------- |

| HTTP | method, path, headers, host |

| gRPC | service, method |

| Kafka | topic, clientID, apiKey, apiVersion |

| DNS | query name pattern (matchName, matchPattern) |

4.4 Kafka L7 Policy Example

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: kafka-consumer

egress:

- toEndpoints:

- matchLabels:

app: kafka-broker

toPorts:

- ports:

- port: '9092'

protocol: TCP

rules:

kafka:

- role: consume

topic: 'orders'

- role: consume

topic: 'events'

5. FQDN Policies: DNS Proxy Integration

5.1 How FQDN Policies Work

FQDN policy flow:

1. Policy definition:

toFQDNs:

- matchName: "api.example.com"

2. Cilium DNS proxy activation:

- Transparently intercept Pod DNS traffic

- Learn IP addresses from DNS responses

3. IP cache update:

DNS response: api.example.com -> 203.0.113.10, 203.0.113.11

Add IP -> FQDN mapping to ipcache

Add allow rules for those IPs to BPF policy map

4. Traffic allow/block:

- Only allow traffic to learned IPs

- Mapping expires and refreshes based on DNS TTL

5.2 FQDN Policy Example

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: backend

egress:

- toFQDNs:

- matchName: 'api.external-service.com'

toPorts:

- ports:

- port: '443'

protocol: TCP

- toFQDNs:

- matchPattern: '*.storage.googleapis.com'

toPorts:

- ports:

- port: '443'

protocol: TCP

- toEndpoints:

- matchLabels:

io.kubernetes.pod.namespace: kube-system

k8s-app: kube-dns

toPorts:

- ports:

- port: '53'

protocol: UDP

rules:

dns:

- matchPattern: '*'

6. CiliumNetworkPolicy vs CiliumClusterwideNetworkPolicy

6.1 Scope Differences

CiliumNetworkPolicy (CNP):

- Namespace scope

- endpointSelector selects Pods in the same namespace

- Cross-namespace via namespaceSelector in fromEndpoints/toEndpoints

CiliumClusterwideNetworkPolicy (CCNP):

- Cluster scope

- Can select specific nodes with nodeSelector

- Applies across all namespaces

- Suitable for cluster-level default policies

6.2 CCNP Example

apiVersion: cilium.io/v2

kind: CiliumClusterwideNetworkPolicy

metadata:

spec:

endpointSelector: {}

ingress:

- fromEntities:

- cluster

6.3 Host Policy

apiVersion: cilium.io/v2

kind: CiliumClusterwideNetworkPolicy

metadata:

spec:

nodeSelector:

matchLabels:

node-role.kubernetes.io/worker: ''

ingress:

- fromEntities:

- remote-node

- health

toPorts:

- ports:

- port: '10250'

protocol: TCP

- fromCIDR:

- 10.0.0.0/8

toPorts:

- ports:

- port: '22'

protocol: TCP

7. Policy Audit Mode

7.1 Audit Mode Operation

In audit mode, policy-violating traffic is not blocked but only logged:

Set endpoint to audit mode

cilium endpoint config 1234 PolicyAuditMode=Enabled

Global audit mode

cilium config PolicyAuditMode=true

Check audit logs

cilium monitor --type policy-verdict

Example output:

Policy verdict log: flow ... action audit

-> Would be DROPPED by policy (ingress)

Identity: 48291 -> 52103, port 8080/TCP

7.2 Transitioning from Audit to Enforcement

Audit mode workflow:

Step 1: Enable audit mode

-> All traffic allowed, violations logged

Step 2: Analyze traffic patterns with Hubble

-> Identify actual required communication paths

Step 3: Write policies and compare with audit results

-> Verify no unintended blocks

Step 4: Disable audit mode

-> Actual policy enforcement begins

8. Endpoint Regeneration and Policy Changes

8.1 Regeneration Process on Policy Change

Policy change detected:

Determine affected endpoints:

- All endpoints matching the selector

- Including indirect impact via Identity references

Recompute policy maps:

- Generate new allow/deny rule set

- Compare with existing policy map (apply diff only)

Update BPF maps:

- Add/delete entries in cilium_policy map

- Atomic updates to minimize packet loss

Recompile BPF programs (if needed):

- When L7 redirect is added/removed

- When program logic changes are required

Update endpoint state: Ready

8.2 Regeneration Performance Optimization

Optimization techniques:

1. Incremental updates:

- Update only BPF maps instead of full recompilation

- Most policy changes only need map updates

2. Batch regeneration:

- Batch multiple policy changes in a short timeframe

- Prevent unnecessary duplicate regenerations

3. Selective regeneration:

- Only regenerate affected endpoints

- Precisely calculate impact scope based on Identity

9. Deny Policies

9.1 Deny Rule Behavior

Cilium supports explicit Deny rules, which always take precedence over Allow:

apiVersion: cilium.io/v2

kind: CiliumNetworkPolicy

metadata:

spec:

endpointSelector:

matchLabels:

app: backend

egressDeny:

- toCIDR:

- 169.254.169.254/32

toPorts:

- ports:

- port: '80'

protocol: TCP

egress:

- toCIDR:

- 0.0.0.0/0

toPorts:

- ports:

- port: '443'

protocol: TCP

9.2 Policy Priority

Policy evaluation order:

1. Check Deny rules

- If matching Deny rule exists, immediately deny

- Ignore Allow rules

2. Check Allow rules

- If matching Allow rule exists, allow

3. Default behavior

- If policies exist for that direction (ingress/egress): default deny

- If no policies at all: default allow

10. Policy Troubleshooting

10.1 Debugging Commands

Check applied policies

cilium policy get

Per-endpoint policy state

cilium endpoint get 1234

Policy map contents

cilium bpf policy get 1234

Look up labels from Identity

cilium identity get 48291

Policy verdict monitoring

cilium monitor --type policy-verdict

Check dropped packets

cilium monitor --type drop

10.2 Common Issues and Solutions

Issue: Pod-to-Pod communication blocked

Possible causes:

1. Identity not correctly assigned

-> Verify with cilium identity list

2. Policy selector is incorrect

-> Check applied policies with cilium policy get

3. DNS egress policy missing

-> DNS allow is required when using FQDN policies

Issue: L7 policy not applied

Possible causes:

1. Envoy proxy not functioning properly

-> Check Proxy status in cilium status

2. Syntax error in rules under toPorts

-> Check parsed result with cilium policy get

Summary

Cilium's network policy engine provides robust traffic control through a multi-layered security model:

- **Identity-Based Enforcement**: Applies policies based on labels, not IP addresses, ensuring stability in dynamic environments

- **eBPF High-Speed Filtering**: L3/L4 policies applied directly in the kernel for minimal latency

- **Envoy L7 Integration**: Application-level filtering for HTTP, gRPC, Kafka

- **FQDN Policies**: Domain-based egress control through DNS proxy

- **Deny Precedence**: Explicit deny rules always take priority for stronger security

- **Audit Mode**: Assess impact before enforcing policies

- **Incremental Updates**: Apply policy changes with minimal regeneration