Skip to content
Published on

Exposing TCP/UDP Services with Ingress — The Limits of L4 and How to Work Around Them

🇰🇷KO🇺🇸EN🇯🇵JA
Split View필사 모드
Authors

Introduction

Exposing a web application externally on Kubernetes is relatively intuitive. You create an Ingress resource, write down the host and path, and you are done. Then one day the operations team comes with a request: "Open PostgreSQL so external analytics tools can connect directly," "Expose the game server's UDP port," "Serve our internal DNS from the cluster." At that moment you run straight into the fundamental limits of Ingress.

The Ingress API is by design an L7 abstraction, specifically for HTTP and HTTPS. All it does is route on L7 information like hostnames, URL paths, and HTTP headers. The PostgreSQL wire protocol, a game's UDP packets, and DNS queries, by contrast, are pure TCP/UDP traffic rather than HTTP, so there is simply no way to express them with an Ingress resource.

Even so, in practice there is strong demand to expose TCP/UDP through the same Ingress controller, because nobody wants to spin up a pile of separate LoadBalancers. Fortunately, the major controllers support L4 exposure outside the Ingress API, through controller-specific mechanisms like ConfigMaps, EntryPoints, and CRDs. This article walks through those methods and offers criteria for when to use such workarounds versus when a dedicated L4 solution (MetalLB, a cloud NLB, and so on) is the better answer. We also look at how the 2026 successor standard, the Gateway API, treats this problem as first-class with TCPRoute and UDPRoute.

Ingress is fundamentally L7

Let us be clear up front: the Ingress resource has no fields for TCP/UDP. No matter how hard you stare at the spec, rules only contains an http block, routing HTTP traffic by port, host, and path. So "exposing TCP with Ingress" is strictly speaking a misnomer; what actually happens is "layering additional L4 proxy configuration onto the Ingress controller (that is, the nginx/Traefik/HAProxy process behind it)."

   L7 (HTTP)                       L4 (TCP/UDP)
   ---------                       ------------
   Expressible as Ingress          Not expressible as Ingress
   host/path routing               port-level passthrough
   TLS termination                 TLS passthrough (SNI)
        |                                |
        +------ same controller process -+
              (L4 added via ConfigMap / EntryPoint / CRD)

Making this distinction clear up front saves you from later confusion like "why doesn't host routing work?" In L4 exposure, hostname routing is impossible (unless you use TLS SNI).

The ingress-nginx TCP/UDP services ConfigMap

ingress-nginx uses two separate ConfigMaps for L4 exposure: tcp-services and udp-services. The key is the port number to expose externally, and the value is a backend reference in the form namespace/service:port.

apiVersion: v1
kind: ConfigMap
metadata:
  name: tcp-services
  namespace: ingress-nginx
data:
  "5432": "databases/postgres:5432"
  "6379": "cache/redis:6379"

UDP works the same way.

apiVersion: v1
kind: ConfigMap
metadata:
  name: udp-services
  namespace: ingress-nginx
data:
  "53": "dns/coredns:53"

That is not the end of it. You must give the ingress-nginx controller flags so it reads these ConfigMaps, and you must actually open those ports on the controller Pod and LoadBalancer Service. Add the following to the Deployment args.

args:
  - /nginx-ingress-controller
  - --tcp-services-configmap=ingress-nginx/tcp-services
  - --udp-services-configmap=ingress-nginx/udp-services

Then add the ports to the Service that exposes the controller.

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 80
    - name: https
      port: 443
      targetPort: 443
    - name: postgres
      port: 5432
      targetPort: 5432
    - name: dns-udp
      port: 53
      protocol: UDP
      targetPort: 53

If you installed via Helm, declaring the tcp and udp mappings in values lets the chart handle all three steps above for you.

tcp:
  5432: "databases/postgres:5432"
udp:
  53: "dns/coredns:53"

Traefik EntryPoints and IngressRouteTCP/UDP

Traefik treats L4 more as a first-class concept. First, define a TCP/UDP EntryPoint in static config.

entryPoints:
  postgres:
    address: ":5432"
  gamedns:
    address: ":53/udp"

Then declare TCP routing with the IngressRouteTCP CRD. To pass TLS through without terminating it, combine SNI matching with passthrough.

apiVersion: traefik.io/v1alpha1
kind: IngressRouteTCP
metadata:
  name: postgres-route
  namespace: databases
spec:
  entryPoints:
    - postgres
  routes:
    - match: HostSNI(`*`)
      services:
        - name: postgres
          port: 5432

UDP is handled with IngressRouteUDP. Since UDP has no concept of SNI or hosts, traffic arriving on the EntryPoint is forwarded to the backend with no match condition.

apiVersion: traefik.io/v1alpha1
kind: IngressRouteUDP
metadata:
  name: dns-route
  namespace: dns
spec:
  entryPoints:
    - gamedns
  routes:
    - services:
        - name: coredns
          port: 53

TLS passthrough — SNI-based routing

When you want to mimic hostname routing at L4, the only clue is the SNI (Server Name Indication) of the TLS handshake. SNI is carried in plaintext in the ClientHello before encryption, so a proxy can tell which domain the connection is heading to without terminating TLS.

ClientHello (SNI: db1.example.com)
        -> proxy reads only SNI and decides routing
        -> TLS is not terminated, passed through to the backend
        -> the backend terminates TLS itself

The advantage of passthrough is that end-to-end encryption is preserved all the way to the backend, and the proxy never needs to hold a certificate. In ingress-nginx you enable it with the nginx.ingress.kubernetes.io/ssl-passthrough annotation and the controller's --enable-ssl-passthrough flag. In Traefik you set tls.passthrough: true on the IngressRouteTCP above. The limitation is that it cannot apply to pure TCP protocols that do not send SNI (for example, plaintext PostgreSQL).

Gateway API TCPRoute/UDPRoute

With the Ingress API now frozen, the right answer for L4 exposure is the Gateway API. Designed from the start with multiple protocols in mind, it provides first-class resources: TCPRoute for TCP, UDPRoute for UDP, and TLSRoute for TLS passthrough.

First, declare the L4 protocol on the Gateway's Listener.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: l4-gateway
  namespace: infra
spec:
  gatewayClassName: example
  listeners:
    - name: postgres
      protocol: TCP
      port: 5432
      allowedRoutes:
        kinds:
          - kind: TCPRoute

Then connect the backend with a TCPRoute.

apiVersion: gateway.networking.k8s.io/v1alpha2
kind: TCPRoute
metadata:
  name: postgres
  namespace: databases
spec:
  parentRefs:
    - name: l4-gateway
      namespace: infra
      sectionName: postgres
  rules:
    - backendRefs:
        - name: postgres
          port: 5432

UDPRoute follows the same structure and attaches to a protocol: UDP Listener. The key advantage is being expressed without annotation or ConfigMap workarounds and independently of the controller. Note that TCPRoute/UDPRoute reach GA relatively late in the Gateway API, so before adopting them, check the support level of the implementation you use (for example, Cilium, Istio, or Envoy Gateway).

Choosing against dedicated L4 solutions

Here it pays to step back. Layering TCP/UDP onto the Ingress controller is not always the best choice. Often the simpler and more robust answer is a dedicated L4 path.

ApproachSuitable whenDrawback
Ingress controller L4 (ConfigMap/CRD)Controller already in operation, a few extra portsController lock-in, host routing limits
Service type LoadBalancerIndependent IP per service, simplestCloud LB cost, IP consumption
MetalLB (on-prem)Implements LoadBalancer on bare metalRequires L2/BGP network understanding
Cloud NLB (L4)High-performance pure L4, high connection volumeNo L7 features
Gateway API TCPRouteStandardization and portability, multi-protocolVariable implementation maturity

The decision rule is simple. If you have one or two ports and already run ingress-nginx or Traefik, controller L4 exposure saves operational cost. Conversely, if throughput and stability matter (as with databases), if you have many L4 services to expose, or if you need no L7 features at all, an NLB or a type LoadBalancer Service is clearer and easier to debug.

Real-world cases

Exposing a database

Opening a production database directly to the outside warrants caution. If exposure is unavoidable, preserve end-to-end encryption with TLS passthrough, restrict source IPs with a NetworkPolicy, and expose only read replicas where possible.

Game server UDP

Real-time games often use UDP and are latency-sensitive. UDP cannot do SNI routing, so it can only branch per port. When a session must stick to the same backend, use a cloud NLB's session affinity or client-IP-based hashing.

Serving DNS

DNS must handle both UDP 53 and TCP 53 (large responses or zone transfers use TCP). So register the same port 53 in both udp-services and tcp-services, and declare both protocols on the Service as well.

Pitfalls and checklist

Common pitfalls in L4 exposure.

  • Expecting host routing: L4 has no host concept. Expecting per-host branching without SNI fails.
  • Missing Service port: even if registered in the ConfigMap, the port is unreachable from outside if not opened on the controller Service.
  • UDP firewall: cloud security groups and node firewalls often block UDP by default.
  • Controller flag not set: creating only the ConfigMap without the --tcp-services-configmap flag means it is ignored.
  • Source IP preservation: passing through an L4 proxy can mask the client IP. Consider externalTrafficPolicy: Local or the PROXY protocol.

Checklist:

[ ] Is the target truly pure TCP/UDP rather than L7
[ ] Did you register port and backend correctly in the ConfigMap/CRD
[ ] Did you open the port on the controller Service (LoadBalancer)
[ ] Did you set the required controller flag/EntryPoint
[ ] Did you allow the TCP/UDP port in firewall/security groups
[ ] Did you restrict sources with a NetworkPolicy (especially DB)
[ ] Did you compare whether a dedicated L4 (NLB/MetalLB) is simpler

Closing thoughts

The crux of exposing TCP/UDP with Ingress is understanding that Ingress is fundamentally an L7 abstraction, and that L4 traffic is layered on "as a side feature" through controller-specific mechanisms. The ingress-nginx tcp/udp-services ConfigMap, Traefik's EntryPoints and IngressRouteTCP/UDP, and SNI routing via TLS passthrough were those tools.

But the genuinely clean future is the Gateway API, since TCPRoute, UDPRoute, and TLSRoute elevate L4 to standard resources. At the same time, do not forget that a dedicated L4 solution is often better than complicating the controller for a port or two. Choose by weighing the nature of the target, the infrastructure you already run, and your future Gateway API migration plan together.

References

Discuss on Twitter • View on GitHub