Chaos and Order

💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.

원문 렌더가 준비되기 전까지 텍스트 가이드로 표시합니다.

Introduction

Exposing a web application externally on Kubernetes is relatively intuitive. You create an Ingress resource, write down the host and path, and you are done. Then one day the operations team comes with a request: "Open PostgreSQL so external analytics tools can connect directly," "Expose the game server's UDP port," "Serve our internal DNS from the cluster." At that moment you run straight into the fundamental limits of Ingress.

The `Ingress` API is by design an L7 abstraction, specifically for HTTP and HTTPS. All it does is route on L7 information like hostnames, URL paths, and HTTP headers. The PostgreSQL wire protocol, a game's UDP packets, and DNS queries, by contrast, are pure TCP/UDP traffic rather than HTTP, so there is simply no way to express them with an Ingress resource.

Even so, in practice there is strong demand to expose TCP/UDP through the same Ingress controller, because nobody wants to spin up a pile of separate LoadBalancers. Fortunately, the major controllers support L4 exposure outside the Ingress API, through controller-specific mechanisms like ConfigMaps, EntryPoints, and CRDs. This article walks through those methods and offers criteria for when to use such workarounds versus when a dedicated L4 solution (MetalLB, a cloud NLB, and so on) is the better answer. We also look at how the 2026 successor standard, the Gateway API, treats this problem as first-class with `TCPRoute` and `UDPRoute`.

Ingress is fundamentally L7

Let us be clear up front: the `Ingress` resource has no fields for TCP/UDP. No matter how hard you stare at the spec, `rules` only contains an `http` block, routing HTTP traffic by port, host, and path. So "exposing TCP with Ingress" is strictly speaking a misnomer; what actually happens is "layering additional L4 proxy configuration onto the Ingress controller (that is, the nginx/Traefik/HAProxy process behind it)."

L7 (HTTP) L4 (TCP/UDP)

--------- ------------

Expressible as Ingress Not expressible as Ingress

host/path routing port-level passthrough

TLS termination TLS passthrough (SNI)

| |

+------ same controller process -+

(L4 added via ConfigMap / EntryPoint / CRD)

Making this distinction clear up front saves you from later confusion like "why doesn't host routing work?" In L4 exposure, hostname routing is impossible (unless you use TLS SNI).

The ingress-nginx TCP/UDP services ConfigMap

ingress-nginx uses two separate ConfigMaps for L4 exposure: `tcp-services` and `udp-services`. The key is the port number to expose externally, and the value is a backend reference in the form `namespace/service:port`.

apiVersion: v1

kind: ConfigMap

metadata:

namespace: ingress-nginx

data:

"5432": "databases/postgres:5432"

"6379": "cache/redis:6379"

UDP works the same way.

apiVersion: v1

kind: ConfigMap

metadata:

namespace: ingress-nginx

data:

"53": "dns/coredns:53"

That is not the end of it. You must give the ingress-nginx controller flags so it reads these ConfigMaps, and you must actually open those ports on the controller Pod and LoadBalancer Service. Add the following to the Deployment args.

args:

- /nginx-ingress-controller

- --tcp-services-configmap=ingress-nginx/tcp-services

- --udp-services-configmap=ingress-nginx/udp-services

Then add the ports to the Service that exposes the controller.

apiVersion: v1

kind: Service

metadata:

namespace: ingress-nginx

spec:

type: LoadBalancer

ports:

- name: http

port: 80

targetPort: 80

- name: https

port: 443

targetPort: 443

- name: postgres

port: 5432

targetPort: 5432

- name: dns-udp

port: 53

protocol: UDP

targetPort: 53

If you installed via Helm, declaring the `tcp` and `udp` mappings in values lets the chart handle all three steps above for you.

tcp:

5432: "databases/postgres:5432"

udp:

53: "dns/coredns:53"

Traefik EntryPoints and IngressRouteTCP/UDP

Traefik treats L4 more as a first-class concept. First, define a TCP/UDP EntryPoint in static config.

entryPoints:

postgres:

address: ":5432"

gamedns:

address: ":53/udp"

Then declare TCP routing with the `IngressRouteTCP` CRD. To pass TLS through without terminating it, combine SNI matching with passthrough.

apiVersion: traefik.io/v1alpha1

kind: IngressRouteTCP

metadata:

namespace: databases

spec:

entryPoints:

- postgres

routes:

- match: HostSNI(`*`)

services:

- name: postgres

port: 5432

UDP is handled with `IngressRouteUDP`. Since UDP has no concept of SNI or hosts, traffic arriving on the EntryPoint is forwarded to the backend with no match condition.

apiVersion: traefik.io/v1alpha1

kind: IngressRouteUDP

metadata:

namespace: dns

spec:

entryPoints:

- gamedns

routes:

- services:

- name: coredns

port: 53

TLS passthrough — SNI-based routing

When you want to mimic hostname routing at L4, the only clue is the SNI (Server Name Indication) of the TLS handshake. SNI is carried in plaintext in the ClientHello before encryption, so a proxy can tell which domain the connection is heading to without terminating TLS.

ClientHello (SNI: db1.example.com)

-> proxy reads only SNI and decides routing

-> TLS is not terminated, passed through to the backend

-> the backend terminates TLS itself

The advantage of passthrough is that end-to-end encryption is preserved all the way to the backend, and the proxy never needs to hold a certificate. In ingress-nginx you enable it with the `nginx.ingress.kubernetes.io/ssl-passthrough` annotation and the controller's `--enable-ssl-passthrough` flag. In Traefik you set `tls.passthrough: true` on the IngressRouteTCP above. The limitation is that it cannot apply to pure TCP protocols that do not send SNI (for example, plaintext PostgreSQL).

Gateway API TCPRoute/UDPRoute

With the `Ingress` API now frozen, the right answer for L4 exposure is the Gateway API. Designed from the start with multiple protocols in mind, it provides first-class resources: `TCPRoute` for TCP, `UDPRoute` for UDP, and `TLSRoute` for TLS passthrough.

First, declare the L4 protocol on the Gateway's Listener.

apiVersion: gateway.networking.k8s.io/v1

kind: Gateway

metadata:

namespace: infra

spec:

gatewayClassName: example

listeners:

- name: postgres

protocol: TCP

port: 5432

allowedRoutes:

kinds:

- kind: TCPRoute

Then connect the backend with a TCPRoute.

apiVersion: gateway.networking.k8s.io/v1alpha2

kind: TCPRoute

metadata:

namespace: databases

spec:

parentRefs:

- name: l4-gateway

namespace: infra

sectionName: postgres

rules:

- backendRefs:

- name: postgres

port: 5432

UDPRoute follows the same structure and attaches to a `protocol: UDP` Listener. The key advantage is being expressed without annotation or ConfigMap workarounds and independently of the controller. Note that TCPRoute/UDPRoute reach GA relatively late in the Gateway API, so before adopting them, check the support level of the implementation you use (for example, Cilium, Istio, or Envoy Gateway).

Choosing against dedicated L4 solutions

Here it pays to step back. Layering TCP/UDP onto the Ingress controller is not always the best choice. Often the simpler and more robust answer is a dedicated L4 path.

| Approach | Suitable when | Drawback |

| --- | --- | --- |

| Ingress controller L4 (ConfigMap/CRD) | Controller already in operation, a few extra ports | Controller lock-in, host routing limits |

| Service type LoadBalancer | Independent IP per service, simplest | Cloud LB cost, IP consumption |

| MetalLB (on-prem) | Implements LoadBalancer on bare metal | Requires L2/BGP network understanding |

| Cloud NLB (L4) | High-performance pure L4, high connection volume | No L7 features |

| Gateway API TCPRoute | Standardization and portability, multi-protocol | Variable implementation maturity |

The decision rule is simple. If you have one or two ports and already run ingress-nginx or Traefik, controller L4 exposure saves operational cost. Conversely, if throughput and stability matter (as with databases), if you have many L4 services to expose, or if you need no L7 features at all, an NLB or a type LoadBalancer Service is clearer and easier to debug.

Real-world cases

Exposing a database

Opening a production database directly to the outside warrants caution. If exposure is unavoidable, preserve end-to-end encryption with TLS passthrough, restrict source IPs with a NetworkPolicy, and expose only read replicas where possible.

Game server UDP

Real-time games often use UDP and are latency-sensitive. UDP cannot do SNI routing, so it can only branch per port. When a session must stick to the same backend, use a cloud NLB's session affinity or client-IP-based hashing.

Serving DNS

DNS must handle both UDP 53 and TCP 53 (large responses or zone transfers use TCP). So register the same port 53 in both udp-services and tcp-services, and declare both protocols on the Service as well.

Pitfalls and checklist

Common pitfalls in L4 exposure.

- **Expecting host routing**: L4 has no host concept. Expecting per-host branching without SNI fails.

- **Missing Service port**: even if registered in the ConfigMap, the port is unreachable from outside if not opened on the controller Service.

- **UDP firewall**: cloud security groups and node firewalls often block UDP by default.

- **Controller flag not set**: creating only the ConfigMap without the `--tcp-services-configmap` flag means it is ignored.

- **Source IP preservation**: passing through an L4 proxy can mask the client IP. Consider `externalTrafficPolicy: Local` or the PROXY protocol.

Checklist:

[ ] Is the target truly pure TCP/UDP rather than L7

[ ] Did you register port and backend correctly in the ConfigMap/CRD

[ ] Did you open the port on the controller Service (LoadBalancer)

[ ] Did you set the required controller flag/EntryPoint

[ ] Did you allow the TCP/UDP port in firewall/security groups

[ ] Did you restrict sources with a NetworkPolicy (especially DB)

[ ] Did you compare whether a dedicated L4 (NLB/MetalLB) is simpler

Closing thoughts

The crux of exposing TCP/UDP with Ingress is understanding that Ingress is fundamentally an L7 abstraction, and that L4 traffic is layered on "as a side feature" through controller-specific mechanisms. The ingress-nginx tcp/udp-services ConfigMap, Traefik's EntryPoints and IngressRouteTCP/UDP, and SNI routing via TLS passthrough were those tools.

But the genuinely clean future is the Gateway API, since TCPRoute, UDPRoute, and TLSRoute elevate L4 to standard resources. At the same time, do not forget that a dedicated L4 solution is often better than complicating the controller for a port or two. Choose by weighing the nature of the target, the infrastructure you already run, and your future Gateway API migration plan together.

References

- Kubernetes Ingress official docs: https://kubernetes.io/docs/concepts/services-networking/ingress/

- ingress-nginx exposing TCP/UDP guide: https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/

- ingress-nginx SSL passthrough: https://kubernetes.github.io/ingress-nginx/user-guide/tls/

- Traefik IngressRouteTCP/UDP: https://doc.traefik.io/traefik/routing/providers/kubernetes-crd/

- Gateway API TCPRoute: https://gateway-api.sigs.k8s.io/api-types/tcproute/

- Gateway API UDPRoute: https://gateway-api.sigs.k8s.io/api-types/udproute/

- MetalLB official docs: https://metallb.universe.tf/

- Kubernetes Service official docs: https://kubernetes.io/docs/concepts/services-networking/service/