Skip to content

필사 모드: Envoy Gateway Deep Dive: Inside the Gateway API Reference Implementation

English
0%
정확도 0%
💡 왼쪽 원문을 읽으면서 오른쪽에 따라 써보세요. Tab 키로 힌트를 받을 수 있습니다.
원문 렌더가 준비되기 전까지 텍스트 가이드로 표시합니다.

Introduction

Getting external traffic into a Kubernetes cluster is the starting point of almost every service deployment. For a long time, the Ingress resource played that role. But the moment you use Ingress in any depth in production, you hit its limits quickly. Path-based and host-based routing are standardized, but practical features like weighted traffic splitting, header manipulation, timeouts, retries, mTLS, and gRPC routing all have to be crammed in through annotations.

The problem is that these annotations differ from controller to controller. Annotations you used in ingress-nginx do not work in Traefik, and the AWS ALB Ingress Controller demands yet another syntax. As a result, your Ingress manifests become tightly coupled to a specific controller, and the moment you switch controllers you have to rewrite the entire set of routing rules.

As of 2026, this trend has clearly settled. The Ingress API is effectively frozen. The Kubernetes project has decided not to add any new features to Ingress, and all new functionality goes into Gateway API. On top of that, the most widely used ingress-nginx has drifted into something close to maintenance mode, increasing the burden of handling security issues. For a team starting fresh, there is no longer a reason to choose Ingress.

This post takes a deep look at Envoy Gateway, the reference-grade implementation of Gateway API. From a practical standpoint, we will cover where Envoy Gateway sits, what its internal architecture looks like, how to write the actual YAML, what you can do with the policy extension CRDs, and how to migrate from Ingress.

Where Gateway API and Envoy Gateway Sit

Let us start with terminology. Gateway API is the next-generation traffic routing API specification standardized by the Kubernetes SIG-Network. The specification itself is not an implementation but a set of CRDs, and multiple vendors implement this spec independently. Istio, Cilium, Kong, Traefik, and NGINX Gateway Fabric are all Gateway API implementations.

Envoy Gateway is the official implementation built directly by the Envoy proxy community. It started as part of the CNCF Envoy project, uses the Envoy proxy as its data plane, and maps Gateway API directly on top of it. In other words, its core goal is to expose Envoy's powerful features through the standard interface that is Gateway API.

The table below summarizes the differences between the legacy Ingress and Gateway API.

| Item | Ingress API | Gateway API |

| --- | --- | --- |

| Standardization status | Frozen, no new features | Actively evolving, standard successor |

| Role separation | All responsibility in one resource | Infra and app roles clearly separated |

| Routing expressiveness | Path/host focused, rest via annotations | Headers, weights, methods as first-class |

| Protocols | Mainly HTTP/HTTPS | HTTP, gRPC, TCP, UDP, TLS |

| Portability | Tied to per-controller annotations | Portability via the spec |

| Policy extension | No standard | Provides policy attachment model |

The biggest design philosophy of Gateway API is role-based separation. The cluster operator (infrastructure provider) defines the GatewayClass, the namespace operator brings up a Gateway, and the application developer defines only the routing for their own service via HTTPRoute. Because each role touches resources only within its own scope, permission separation and multi-tenancy follow naturally.

A Look Inside the Architecture

Envoy Gateway is broadly divided into a control plane and a data plane. The control plane watches Gateway API resources, translates them into Envoy configuration, and pushes it down to the data plane. The data plane consists of the Envoy proxy pods that actually handle traffic.

[ Users / Clients ]

|

v

+----------------------------------+

| Envoy Proxy (data plane) |

| - listeners / routes / clusters |

| - filter chains / TLS termination|

+----------------------------------+

^

| xDS (gRPC)

|

+----------------------------------+

| Envoy Gateway controller |

| (control plane) |

| - watch Gateway API resources |

| - translate to IR |

| - generate Envoy xDS config |

+----------------------------------+

^

| watch

|

+----------------------------------+

| Kubernetes API Server |

| GatewayClass / Gateway |

| HTTPRoute / GRPCRoute / TLSRoute |

| policy CRDs |

+----------------------------------+

If we unpack the processing flow inside the controller a bit more, it proceeds through the following stages.

1. The provider layer watches Gateway API resources and related resources (Service, Secret, EndpointSlice, etc.) from the Kubernetes API.

2. The translation layer converts these resources into an internal Intermediate Representation (IR). The IR is an abstract model that is not tied to any specific data plane.

3. The IR is then translated into Envoy's xDS resources (Listener, Route, Cluster, Endpoint, Secret).

4. The xDS server delivers this configuration to the Envoy proxy pods over a gRPC stream.

Thanks to this structure, Envoy Gateway seamlessly bridges the declarative model of Gateway API and the dynamic configuration model of Envoy. Users do not need to write Envoy configuration syntax directly; they only write Gateway API resources.

Installation

Installing with Helm is the most common approach. Here is a basic installation example.

helm install eg oci://docker.io/envoyproxy/gateway-helm \

--version v1.3.0 \

-n envoy-gateway-system \

--create-namespace

Wait until the controller is ready

kubectl wait --timeout=5m \

-n envoy-gateway-system \

deployment/envoy-gateway \

--for=condition=Available

Once the installation finishes, a GatewayClass may be created automatically, or you may need to create it yourself. After installing, confirm the CRDs came in correctly with the following commands.

kubectl get crd | grep -E 'gateway|envoyproxy'

kubectl get gatewayclass

kubectl get pods -n envoy-gateway-system

Hands-on: GatewayClass and Gateway

First, define a GatewayClass. The GatewayClass is a cluster-scoped resource that specifies which controller handles this class.

apiVersion: gateway.networking.k8s.io/v1

kind: GatewayClass

metadata:

name: eg

spec:

controllerName: gateway.envoyproxy.io/gatewayclass-controller

Next, create the Gateway that becomes the actual entry point. A Gateway defines listeners, and a listener specifies the port, protocol, hostname, and which routes can attach to it.

apiVersion: gateway.networking.k8s.io/v1

kind: Gateway

metadata:

name: eg

namespace: default

spec:

gatewayClassName: eg

listeners:

- name: http

protocol: HTTP

port: 80

allowedRoutes:

namespaces:

from: Same

- name: https

protocol: HTTPS

port: 443

hostname: "*.example.com"

tls:

mode: Terminate

certificateRefs:

- kind: Secret

name: example-com-tls

allowedRoutes:

namespaces:

from: All

Here the namespaces setting under allowedRoutes is the heart of multi-tenancy. Setting from to Same allows only routes in the same namespace to attach, All allows all namespaces, and Selector lets you narrow the allowed scope with a label selector.

To check whether the Gateway has been programmed correctly, inspect its status as follows.

kubectl get gateway eg -o yaml | yq '.status.conditions'

kubectl get gateway eg \

-o jsonpath='{.status.addresses[0].value}'

Hands-on: HTTPRoute

HTTPRoute is the resource application developers handle most often. It declares which Gateway to attach to, which hostname and path to match, and which backend to send to.

apiVersion: gateway.networking.k8s.io/v1

kind: HTTPRoute

metadata:

name: backend

namespace: default

spec:

parentRefs:

- name: eg

hostnames:

- "www.example.com"

rules:

- matches:

- path:

type: PathPrefix

value: /api

backendRefs:

- name: backend-svc

port: 8080

Weighted traffic splitting (canary deployment) was only possible through annotations in Ingress, but it is a first-class feature in Gateway API. You simply assign weights to the two backends.

apiVersion: gateway.networking.k8s.io/v1

kind: HTTPRoute

metadata:

name: canary

namespace: default

spec:

parentRefs:

- name: eg

hostnames:

- "www.example.com"

rules:

- backendRefs:

- name: backend-v1

port: 8080

weight: 90

- name: backend-v2

port: 8080

weight: 10

Header matching and request header manipulation are also expressed as standards. The following example matches only when a specific header is present, and applies a filter that adds a request header before forwarding.

apiVersion: gateway.networking.k8s.io/v1

kind: HTTPRoute

metadata:

name: header-route

namespace: default

spec:

parentRefs:

- name: eg

rules:

- matches:

- headers:

- name: x-canary

value: "true"

filters:

- type: RequestHeaderModifier

requestHeaderModifier:

add:

- name: x-routed-by

value: envoy-gateway

backendRefs:

- name: backend-v2

port: 8080

URL rewrite and redirect filters are also frequently used. Here is a rewrite example that swaps out a path prefix.

filters:

- type: URLRewrite

urlRewrite:

path:

type: ReplacePrefixMatch

replacePrefixMatch: /v2

Hands-on: GRPCRoute

gRPC was a representative protocol that was awkward to handle in Ingress. Gateway API provides a dedicated GRPCRoute resource that lets you route at the service and method level.

apiVersion: gateway.networking.k8s.io/v1

kind: GRPCRoute

metadata:

name: grpc-route

namespace: default

spec:

parentRefs:

- name: eg

hostnames:

- "grpc.example.com"

rules:

- matches:

- method:

service: com.example.UserService

method: GetUser

backendRefs:

- name: user-grpc-svc

port: 9000

HTTP/2-based gRPC traffic is an area Envoy handles extremely well. As Envoy Gateway translates a GRPCRoute into Envoy's route configuration, it automatically takes care of gRPC-specific trailer handling and status code mapping.

Hands-on: TLSRoute

TLSRoute is used when you want to pass TLS through based on SNI without terminating it. It is useful when you want to keep encryption all the way to the backend.

apiVersion: gateway.networking.k8s.io/v1alpha2

kind: TLSRoute

metadata:

name: tls-passthrough

namespace: default

spec:

parentRefs:

- name: eg

sectionName: tls-passthrough

hostnames:

- "secure.example.com"

rules:

- backendRefs:

- name: secure-backend

port: 8443

In this case the Gateway listener must set its TLS mode to Passthrough.

listeners:

- name: tls-passthrough

protocol: TLS

port: 8443

hostname: "secure.example.com"

tls:

mode: Passthrough

allowedRoutes:

kinds:

- kind: TLSRoute

Policy Extensions: Envoy Gateway's Real Strength

The core routing resources of Gateway API alone cannot express operational features like timeouts, retries, rate limiting, and authentication. For this, Gateway API defines a policy attachment model, and Envoy Gateway provides powerful policy CRDs that follow it. This is exactly where Envoy Gateway becomes more than a simple Ingress replacement.

| Policy CRD | Attach target | Key features |

| --- | --- | --- |

| ClientTrafficPolicy | Gateway | Client-side TLS, HTTP/2, connection buffers, header handling |

| BackendTrafficPolicy | Gateway/Route | Retries, timeouts, circuit breaking, rate limiting, load balancing |

| SecurityPolicy | Gateway/Route | JWT auth, OIDC, CORS, basic auth, external authorization |

| EnvoyProxy | GatewayClass/Gateway | Proxy deployment shape, resources, logging, telemetry |

ClientTrafficPolicy

This controls behavior between the client and the gateway. For example, you can preserve the client IP accurately or adjust TLS parameters.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: ClientTrafficPolicy

metadata:

name: client-policy

namespace: default

spec:

targetRefs:

- group: gateway.networking.k8s.io

kind: Gateway

name: eg

clientIPDetection:

xForwardedFor:

numTrustedHops: 1

tls:

minVersion: "1.3"

BackendTrafficPolicy

This handles resilience behavior between the gateway and the backend. You can declare retries, timeouts, and circuit breaking in one place.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: BackendTrafficPolicy

metadata:

name: backend-policy

namespace: default

spec:

targetRefs:

- group: gateway.networking.k8s.io

kind: HTTPRoute

name: backend

retry:

numRetries: 3

perRetry:

timeout: 2s

retryOn:

httpStatusCodes:

- 503

- 502

timeout:

http:

requestTimeout: 10s

circuitBreaker:

maxConnections: 1024

maxPendingRequests: 256

Rate limiting is also declared in the same CRD. The following example limits each client IP to 100 requests per minute. It uses only numbers to express the exact limit, so it is safe.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: BackendTrafficPolicy

metadata:

name: ratelimit-policy

namespace: default

spec:

targetRefs:

- group: gateway.networking.k8s.io

kind: HTTPRoute

name: backend

rateLimit:

type: Global

global:

rules:

- clientSelectors:

- sourceCIDR:

value: 0.0.0.0/0

type: Distinct

limit:

requests: 100

unit: Minute

SecurityPolicy

This handles authentication and authorization at the gateway layer. Taking JWT validation as an example, it looks like this.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: SecurityPolicy

metadata:

name: jwt-policy

namespace: default

spec:

targetRefs:

- group: gateway.networking.k8s.io

kind: HTTPRoute

name: backend

jwt:

providers:

- name: example-issuer

remoteJWKS:

uri: https://issuer.example.com/.well-known/jwks.json

issuer: https://issuer.example.com

CORS and external authorization can also be declared via SecurityPolicy, which greatly helps in stripping common authentication logic out of application code.

EnvoyProxy

This tunes the data plane itself. The number of proxy pod replicas, resource requests and limits, the service type, access log format, tracing, and more are all configured here.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: EnvoyProxy

metadata:

name: custom-proxy

namespace: envoy-gateway-system

spec:

provider:

type: Kubernetes

kubernetes:

envoyDeployment:

replicas: 3

container:

resources:

requests:

cpu: 200m

memory: 256Mi

limits:

memory: 512Mi

telemetry:

accessLog:

settings:

- format:

type: JSON

If you connect this EnvoyProxy resource to a GatewayClass via its parametersRef, you can control the data plane shape of every gateway brought up by that class in one place.

The Relationship Between Envoy Proxy and xDS

To operate Envoy Gateway properly, you need to understand how the data plane, the Envoy proxy, works. Envoy's configuration model is built around four core concepts.

- Listener: the entry point that receives connections on a specific port. The Gateway's listener corresponds to this.

- Route: the rule that decides which cluster an incoming request goes to. HTTPRoute corresponds to this.

- Cluster: a group of endpoints for the same upstream service. The backend Service corresponds to this.

- Endpoint: the actual pod IP and port. It is pulled from EndpointSlice.

All of this configuration is delivered dynamically through a bundle of protocols called xDS. xDS consists of discovery services such as LDS (listener), RDS (route), CDS (cluster), EDS (endpoint), and SDS (secret). The Envoy Gateway controller acts as the xDS server, and the Envoy proxy, as a client, receives configuration over a gRPC stream.

Gateway API resource Envoy concept xDS service

-------------------- ------------- -----------

Gateway listener -> Listener -> LDS

HTTPRoute -> Route -> RDS

Service -> Cluster -> CDS

EndpointSlice -> Endpoint -> EDS

TLS Secret -> Secret -> SDS

Thanks to this dynamic configuration model, even when routing rules change, you can swap configuration in without restarting the Envoy proxy. Zero-downtime configuration updates are one of Envoy's greatest strengths, a big step forward compared to the era when Ingress controllers triggered a reload on every configuration change.

When you want to debug how configuration was actually applied to Envoy, you can look at Envoy's config dump directly with the following commands.

Find the Envoy proxy pod name

kubectl get pods -n envoy-gateway-system \

-l gateway.envoyproxy.io/owning-gateway-name=eg

Query the config dump via the admin port

kubectl exec -n envoy-gateway-system <proxy-pod> -- \

curl -s localhost:19000/config_dump | head -c 2000

Migrating from Ingress to Gateway API

Moving a cluster that runs on Ingress over to Gateway API is safest when approached in stages. Rather than changing everything at once, we recommend a strategy of running both systems in parallel and gradually shifting traffic.

First, organize the mapping of which Gateway API resource each existing Ingress rule corresponds to.

| Ingress concept | Gateway API equivalent |

| --- | --- |

| Ingress resource | Gateway plus HTTPRoute |

| ingressClassName | gatewayClassName |

| host rule | hostnames in HTTPRoute |

| path rule | matches in HTTPRoute |

| backend service | backendRefs in HTTPRoute |

| TLS secret | certificateRefs in Gateway listener |

| canary annotation | weight in backendRefs |

| rewrite annotation | URLRewrite filter |

The migration sequence below is a reasonable one to follow.

1. Install Envoy Gateway in a separate namespace and bring up a new Gateway. Leave the existing Ingress controller in place.

2. Get a separate LoadBalancer address assigned to the new Gateway. There is no impact on existing traffic.

3. Write HTTPRoutes per service to replicate routing onto the new Gateway.

4. Adjust weights at DNS or an upstream load balancer to gradually move traffic to the new Gateway.

5. Once all traffic has stably moved over, remove the old Ingress resources and controller.

To help with this transition, the Kubernetes project also provides a conversion tool called ingress2gateway. Given existing Ingress manifests, it generates draft Gateway API resources equivalent to them.

ingress2gateway print \

--input-file=legacy-ingress.yaml \

--providers=ingress-nginx > gateway-resources.yaml

That said, automatic conversion is only a starting point. Some controller-specific annotations have to be moved by hand into policy CRDs, so the conversion result must always be reviewed by a human.

Operations and Tuning

The first thing to take care of at the operational stage is observability. Envoy Gateway configures access logs, metrics, and distributed tracing all in the telemetry section of the EnvoyProxy resource. Structuring access logs as JSON makes them easier to handle in a log collection pipeline.

For metrics, you use the Prometheus format that Envoy exposes as is. You set up the scrape target as follows.

apiVersion: gateway.envoyproxy.io/v1alpha1

kind: EnvoyProxy

metadata:

name: observable-proxy

namespace: envoy-gateway-system

spec:

telemetry:

metrics:

prometheus:

disable: false

tracing:

provider:

type: OpenTelemetry

host: otel-collector.monitoring.svc

port: 4317

On the availability side, the basics are to bring up the data plane proxy with at least two replicas and spread them across nodes with pod anti-affinity. In addition, setting a PodDisruptionBudget guarantees that at least one proxy is always alive even during a node drain.

Performance tuning varies with traffic characteristics, but there are a few general principles.

- Connection reuse: leverage keep-alive and HTTP/2 multiplexing to the backend to reduce connection establishment cost.

- Appropriate timeouts: too long a timeout invites failure propagation, and too short a timeout cuts off legitimate requests. Set them by looking at the backend's actual response distribution.

- Circuit breaking: when the backend is overloaded, quickly block additional requests to prevent cascading failures.

- Right-sizing resources: Envoy is efficient, but at high throughput CPU becomes the bottleneck. Use load testing to settle on the right replica count.

Integrating with cert-manager lets you automate TLS certificate issuance and renewal. Attaching cert-manager annotations to the Gateway resource causes certificates to be issued automatically as secrets. This is a combination that greatly reduces operational burden.

Common Pitfalls and Troubleshooting

Here we summarize the problems frequently encountered in production and how to diagnose them.

First, the case where a route does not attach to the Gateway. The most common causes are a namespace mismatch in the allowedRoutes setting, or parentRefs pointing to the wrong Gateway or listener. Checking the status conditions of the HTTPRoute clearly shows the Accepted and ResolvedRefs states.

kubectl get httproute backend -o yaml | yq '.status.parents'

Here, if Accepted is False, the parent Gateway has rejected the route, and if ResolvedRefs is False, it could not find the backend Service or Secret.

Second, the case of referencing a Service or Secret in another namespace. Gateway API blocks cross-namespace references by default and requires explicit permission via a separate resource called ReferenceGrant. If cross-namespace references do not work, first check whether the ReferenceGrant is missing.

apiVersion: gateway.networking.k8s.io/v1beta1

kind: ReferenceGrant

metadata:

name: allow-route-to-svc

namespace: backend-ns

spec:

from:

- group: gateway.networking.k8s.io

kind: HTTPRoute

namespace: route-ns

to:

- group: ""

kind: Service

Third, the case where a policy is not applied. Check that the policy CRD's targetRefs points exactly to an existing resource, and that multiple policies do not conflict on the same target. Policy CRDs also have status conditions that tell you whether they were applied.

Fourth, TLS certificate problems. Check that the secret referenced by certificateRefs is in the same namespace as the Gateway, and that the secret type is kubernetes.io/tls. If the certificate is not delivered via SDS, the listener will not move to the Programmed state.

Finally, when configuration seems applied but behavior is strange, the fastest path is to check directly with the config_dump shown earlier what configuration actually reached Envoy. Look at the controller logs together with it.

kubectl logs -n envoy-gateway-system \

deployment/envoy-gateway --tail=100

Conclusion

Now that Ingress is frozen and Gateway API has established itself as the standard successor, if you are designing a new traffic entry layer, Gateway API is no longer a choice but closer to the default. Among the implementations, Envoy Gateway is a reference-grade implementation built directly by the Envoy community, cleanly layering a standard interface on top of Envoy's proven data plane.

The core ideas are role separation and policy extension. Declare routing with the standard resources of Gateway API, and cleanly separate operational features like resilience and security into policy CRDs such as ClientTrafficPolicy, BackendTrafficPolicy, and SecurityPolicy. Control the data plane shape with EnvoyProxy. Once you understand this structure, you newly realize just how fragile the old Ingress operations that relied on piles of annotations were.

For a fresh project, design with Gateway API from the start, and for an existing Ingress environment, use ingress2gateway to draft an initial version and migrate in stages. Standing on top of a standard means that even if you switch controllers, your routing rules survive intact.

References

- Envoy Gateway official docs: https://gateway.envoyproxy.io/docs/

- Envoy Gateway GitHub repository: https://github.com/envoyproxy/gateway

- Gateway API official docs: https://gateway-api.sigs.k8s.io/

- Gateway API policy attachment design: https://gateway-api.sigs.k8s.io/reference/policy-attachment/

- Envoy proxy official docs: https://www.envoyproxy.io/docs

- Envoy xDS protocol docs: https://www.envoyproxy.io/docs/envoy/latest/api-docs/xds_protocol

- Kubernetes Ingress concept docs: https://kubernetes.io/docs/concepts/services-networking/ingress/

- ingress2gateway conversion tool: https://github.com/kubernetes-sigs/ingress2gateway

- cert-manager official docs: https://cert-manager.io/docs/

현재 단락 (1/431)

Getting external traffic into a Kubernetes cluster is the starting point of almost every service dep...

작성 글자: 0원문 글자: 20,483작성 단락: 0/431