Introduction
For years, the gateway that let external traffic into a Kubernetes cluster was effectively defined by the Ingress resource and the ingress-nginx controller. By 2026, however, the landscape looks quite different. The standard Ingress API is frozen, meaning no new features will be added; the successor standard, Gateway API, has moved past GA and is establishing itself quickly; and ingress-nginx has effectively entered maintenance mode while a steady stream of security issues has been reported.
In this transitional period, ingress controllers that double as API gateways are drawing renewed attention. The reason is that teams increasingly want more than just deciding which host and path map to which service. They want to handle gateway concerns such as authentication, rate limiting, canary rollouts, and observability directly in the data plane. Apache APISIX is a project that stands out here, using etcd as its configuration store and putting zero-downtime dynamic routing front and center.
This article takes a code- and diagram-driven look at how the APISIX Ingress Controller works internally, why etcd-based dynamic routing enables reload-free changes, how to design custom resources such as ApisixRoute and attach plugins, and how it supports standard Ingress and Gateway API side by side. It closes with a hands-on Helm deployment and the pitfalls you are most likely to encounter in production. We assume the APISIX 3.x line and the APISIX Ingress Controller 1.8 and later line as of the first half of 2026.
The Shifting Ingress Landscape — Why APISIX Now
Let us first frame the 2026 ingress and gateway landscape, because that context reveals exactly which gap APISIX targets.
The standard Ingress API was deliberately frozen once it stabilized at v1. In other words, the standard keeps only the minimum of host and path routing plus TLS termination, and pushes anything beyond that into controller-specific annotations. This proliferation of annotations effectively destroyed portability between controllers, and the enormous annotation surface of ingress-nginx became a breeding ground for security vulnerabilities. The series of ingress-nginx security issues and the maintenance burden reported around 2025 forced many organizations to evaluate alternatives.
The standard direction for that alternative is the Gateway API. Gateway API is the successor standard designed for role separation (infrastructure operators versus application developers), expressive routing (header, method, and query matching plus traffic splitting), and a portable core specification. The APISIX Ingress Controller supports all three of these modes.
| Aspect | Standard Ingress | APISIX Native CRDs | Gateway API |
| --- | --- | --- | --- |
| Standardization | CNCF standard (frozen) | APISIX only | CNCF standard (active) |
| Routing expressiveness | Host and path centric | Very high (method/header/priority) | High (filters/matches/weights) |
| Portability | Low due to annotations | None (APISIX coupled) | High |
| Gateway features | Worked around via annotations | First class via CRDs | Some via extensions |
| Recommended use | Legacy migration | Full use of APISIX features | Adopting the new standard |
In short: if you are starting fresh, treat Gateway API as the standard path, add native CRDs when you need APISIX's rich plugins and fine-grained routing control, and if you have existing Ingress assets you can migrate them comfortably through standard Ingress mode. The core appeal of APISIX is that it handles all three of these tracks within a single data plane.
What APISIX Is — Separating Data Plane from Control Plane
Apache APISIX is a cloud-native API gateway built on top of OpenResty (Nginx plus LuaJIT). Its core design philosophy is the separation of configuration from execution, and that separation is the foundation of zero-downtime dynamic routing.
APISIX is divided into two main parts.
- Data plane (apisix): the OpenResty-based process that actually receives traffic and performs routing, plugin execution, and upstream forwarding.
- Configuration store (etcd): the key-value store that holds all configuration, including routes, upstreams, consumers, and plugin settings.
In Kubernetes, one more layer is added on top.
- APISIX Ingress Controller: a controller that watches the Kubernetes API server, and whenever a custom resource such as ApisixRoute, a standard Ingress, or a Gateway API resource changes, translates it into APISIX configuration and applies it to etcd (or through the APISIX Admin API).
The overall structure looks like this in ASCII.
+---------------------------------------------------------------+
| Kubernetes cluster |
| |
| +-------------------+ watch +----------------+ |
| | kube-apiserver | <------------------ | APISIX Ingress | |
| | - ApisixRoute | | Controller | |
| | - Ingress | translate & sync | (control plane)| |
| | - Gateway API | +-------+--------+ |
| +-------------------+ | |
| | Admin |
| | API |
| v |
| +---------------+ |
| external traffic | etcd | |
| | | (config store)| |
| v +-------+-------+ |
| +---------+ watch routes/upstreams | |
| | apisix | <--------------------------------+ |
| | (data | |
| | plane) | --> backend Service / Pod (upstream) |
| +---------+ |
+---------------------------------------------------------------+
The decisive detail here is that the apisix data plane watches etcd directly. Thanks to etcd's watch mechanism, when configuration changes, apisix workers apply that change to memory almost in real time. Even when a new route is added or an upstream weight changes, there is no need to reload or restart the Nginx process. That is the essence of zero-downtime dynamic routing.
Why etcd-Based Dynamic Routing Is Zero-Downtime
In a traditional Nginx-based gateway, changing routing usually meant rewriting nginx.conf and sending a reload signal. A reload is the process by which the master reads the new configuration, spawns new workers, and gracefully terminates the old ones. Frequent reloads incur costs such as the following.
- During worker turnover, memory and connection counts can momentarily double.
- Long-lived connections (WebSocket, gRPC streams) may be dropped or experience draining delays.
- In environments where routes change dozens of times per second, the reload itself becomes a bottleneck.
APISIX fundamentally changes this model. The route matching table, the upstream list, and plugin configuration live in etcd rather than nginx.conf, and apisix workers watch these and load them into shared memory inside the LuaJIT runtime (a radixtree-based router). When configuration changes, only that memory structure is updated, so no process restart occurs.
The flow, step by step, looks like this.
[developer] kubectl apply -f apisixroute.yaml
|
v
[kube-apiserver] stores the ApisixRoute object + emits a watch event
|
v
[APISIX Ingress Controller]
- translates ApisixRoute into APISIX Route/Upstream schema
- validates (service existence, port, plugin schema)
- PUT via Admin API or etcd
|
v
[etcd] updates /apisix/routes/<id> key -> watch event
|
v
[apisix data plane worker]
- receives the watch event
- updates the radixtree router in memory (no reload)
|
v
[new routing applies from the next request — existing connections unaffected]
The practical benefits of this architecture are clear. Even if you change upstream weights every five seconds during a canary rollout, the data plane does not flinch, and even with thousands of routes coexisting, radixtree-based matching runs in time that is nearly constant relative to path length. Furthermore, even if the control plane (the Ingress Controller) goes down briefly, the apisix data plane keeps serving traffic using the configuration already loaded in etcd. The fact that the control plane and data plane are failure-isolated is extremely important for gateway availability.
That said, because etcd becomes the single source of truth for configuration, etcd's own availability and backup become central operational concerns. We will return to this in the operations section.
Deployment Modes — Two Ways It Works
The APISIX Ingress Controller offers two broad deployment modes. The mode you choose determines whether etcd is required and what the controller synchronizes.
| Mode | Description | etcd required | Characteristics |
| --- | --- | --- | --- |
| Standard (default) | The controller pushes configuration through the APISIX Admin API; APISIX uses etcd as its backend | Yes | Most common, full features |
| Direct etcd (or controller-embedded sync) | The controller syncs translated configuration directly to the store | Depends on mode | An attempt at a simplified topology |
The recommended setup as of 2026 is the standard mode, where the apisix data plane and etcd run together and the Ingress Controller pushes configuration through the Admin API. The hands-on portion of this article uses this setup. Because the coupling between the controller and the data plane has evolved across versions in the APISIX project, when adopting it in practice you must verify the recommended topology in the documentation for the chart version you intend to install.
Anatomy of the Core Custom Resources
To fully use APISIX's native capabilities, you use APISIX's custom resources instead of the standard Ingress. The three most important are ApisixRoute, ApisixUpstream, and ApisixPluginConfig.
ApisixRoute — The Center of Routing
ApisixRoute expresses host, path, and method matching, backend connection, and even per-route plugins all in one place. A basic form looks like this.
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: orders-route
namespace: shop
spec:
http:
- name: orders
match:
hosts:
- api.example.com
paths:
- /orders/*
methods:
- GET
- POST
backends:
- serviceName: orders-svc
servicePort: 8080
plugins:
- name: limit-count
enable: true
config:
count: 100
time_window: 60
rejected_code: 429
key_type: var
key: remote_addr
What stands out here is the expressiveness of the match block. Where standard Ingress deals only with host and path, ApisixRoute supports HTTP methods, priority, and header, cookie, and query based matching as first-class citizens. Here is a header-based routing example.
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: beta-route
namespace: shop
spec:
http:
- name: beta-users
priority: 10
match:
hosts:
- api.example.com
paths:
- /orders/*
exprs:
- subject:
scope: Header
name: X-Channel
op: Equal
value: beta
backends:
- serviceName: orders-beta-svc
servicePort: 8080
Routes with a higher priority are evaluated first, so the beta route above intercepts requests whose X-Channel header is beta, while the rest flow to the default route. Such an expression is hard to build cleanly with standard Ingress.
ApisixUpstream — Fine-Grained Backend Control
ApisixUpstream configures upstream behavior for a specific Kubernetes Service in detail. It covers the load balancing algorithm, health checks, retries, timeouts, and passive health checks.
apiVersion: apisix.apache.org/v2
kind: ApisixUpstream
metadata:
name: orders-svc
namespace: shop
spec:
loadbalancer:
type: ewma
retries: 2
timeout:
connect: 3s
read: 10s
send: 10s
healthCheck:
active:
type: http
httpPath: /healthz
healthy:
interval: 5
successes: 2
unhealthy:
interval: 5
httpFailures: 3
passive:
healthy:
successes: 3
unhealthy:
httpFailures: 3
tcpFailures: 3
The key rule is that metadata.name must match the name of the target Service. Through this match, the APISIX Ingress Controller applies the above configuration to the upstream that points to that Service. For the load balancer type, you can choose round robin, consistent hashing (chash), or ewma, an exponentially weighted moving average of response time, so you can tune things like having ewma automatically steer away from a backend whose latency is erratic.
ApisixPluginConfig — Plugin Reuse
Instead of repeatedly attaching the same bundle of plugins to many routes, you can define a plugin set with ApisixPluginConfig and reference it from routes.
apiVersion: apisix.apache.org/v2
kind: ApisixPluginConfig
metadata:
name: common-gateway-plugins
namespace: shop
spec:
plugins:
- name: cors
enable: true
config:
allow_origins: "https://app.example.com"
allow_methods: "GET,POST,PUT,DELETE"
allow_headers: "Authorization,Content-Type"
- name: prometheus
enable: true
config:
prefer_name: true
A route references it like this.
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: orders-route
namespace: shop
spec:
http:
- name: orders
match:
hosts: ["api.example.com"]
paths: ["/orders/*"]
backends:
- serviceName: orders-svc
servicePort: 8080
plugin_config_name: common-gateway-plugins
This way you manage CORS and prometheus settings in one place while applying them consistently across dozens of routes. Beyond these, resources such as ApisixConsumer (the consumer or authentication subject), ApisixTls (certificates), and ApisixClusterConfig (global settings) let you handle nearly every aspect of the gateway declaratively.
Plugins — The Real Power of the Gateway
What differentiates APISIX is its rich plugin ecosystem. Plugins hook into each stage of the request processing pipeline (rewrite, access, header_filter, body_filter, log) and can be turned on and off dynamically through nothing more than an etcd configuration change. Grouped by category, frequently used plugins look like this.
| Category | Representative plugins | Purpose |
| --- | --- | --- |
| Authentication | key-auth, jwt-auth, basic-auth, openid-connect, hmac-auth | API key, JWT, and OIDC based auth |
| Traffic control | limit-count, limit-req, limit-conn, traffic-split | Rate limiting, concurrency limits, canary splitting |
| Security | ip-restriction, cors, csrf, ua-restriction | Access control, cross-origin policy |
| Observability | prometheus, opentelemetry, zipkin, http-logger | Metrics, distributed tracing, log shipping |
| Transformation | proxy-rewrite, response-rewrite, grpc-transcode | Request/response rewriting, gRPC-JSON transcoding |
Authentication Plugin Example — key-auth
To attach API key authentication, you first define a consumer and then enable key-auth on the route.
apiVersion: apisix.apache.org/v2
kind: ApisixConsumer
metadata:
name: partner-acme
namespace: shop
spec:
authParameter:
keyAuth:
value:
key: acme-secret-key-value
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: partner-api
namespace: shop
spec:
http:
- name: partner
match:
hosts: ["api.example.com"]
paths: ["/partner/*"]
backends:
- serviceName: partner-svc
servicePort: 8080
plugins:
- name: key-auth
enable: true
Now the client must carry the issued key in the apikey header to get through. In practice, the recommended pattern is not to keep the key value in plaintext but to reference a secret or integrate with an external secret manager.
Traffic Splitting Example — Canary Rollout
With the traffic-split plugin you can declaratively configure a weight-based canary.
apiVersion: apisix.apache.org/v2
kind: ApisixRoute
metadata:
name: orders-canary
namespace: shop
spec:
http:
- name: orders
match:
hosts: ["api.example.com"]
paths: ["/orders/*"]
backends:
- serviceName: orders-stable
servicePort: 8080
weight: 90
- serviceName: orders-canary
servicePort: 8080
weight: 10
Assigning weights to the backends makes APISIX distribute traffic at 90 to 10. Thanks to the dynamic routing emphasized earlier, even if you change this weight from 90/10 to 50/50 and apply it, the data plane reflects the new ratio immediately without a reload. It pairs well with an automated pipeline that gradually ramps up a canary.
Observability — prometheus and opentelemetry
Because the gateway sits at the crossroads of traffic, it is a golden spot for observability. Enabling the prometheus plugin exposes metrics such as request counts, latency distributions, and counts by status code.
apiVersion: apisix.apache.org/v2
kind: ApisixClusterConfig
metadata:
name: default
spec:
monitoring:
prometheus:
enable: true
skywalking:
enable: false
When distributed tracing is needed, the opentelemetry plugin can send traces to an OTLP collector. By starting a trace at the gateway and propagating it to the backend, you can view the entire path of a single request as it passes through the gateway and across several microservices as a single trace.
Standard Ingress and Gateway API Support
A major advantage of the APISIX Ingress Controller is that it is not confined to native CRDs. You can drive the same data plane with standard Ingress and Gateway API as well.
Standard Ingress Mode
To bring existing Ingress assets over as is, set ingressClassName to apisix.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: legacy-orders
namespace: shop
annotations:
k8s.apisix.apache.org/enable-cors: "true"
spec:
ingressClassName: apisix
rules:
- host: api.example.com
http:
paths:
- path: /orders
pathType: Prefix
backend:
service:
name: orders-svc
port:
number: 8080
The limits of standard Ingress mode are clear. APISIX's rich features are exposed only through annotations, and those annotation keys are APISIX-specific, so there is no portability. Therefore it is appropriate to view standard Ingress mode as a migration path to "move existing assets over comfortably and gradually transition to native CRDs or Gateway API."
Gateway API Mode
If you are adopting the new standard, use Gateway API. The role separation in which an infrastructure operator defines the Gateway and an application team attaches an HTTPRoute is expressed naturally.
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: apisix-gateway
namespace: infra
spec:
gatewayClassName: apisix
listeners:
- name: http
protocol: HTTP
port: 80
allowedRoutes:
namespaces:
from: All
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: orders-route
namespace: shop
spec:
parentRefs:
- name: apisix-gateway
namespace: infra
hostnames:
- api.example.com
rules:
- matches:
- path:
type: PathPrefix
value: /orders
backendRefs:
- name: orders-svc
port: 8080
weight: 100
Because Gateway API provides expressions such as traffic splitting, header matching, and filters as part of the standard specification, portability between controllers is good. The recommended direction in 2026 is a hybrid strategy that makes Gateway API the default for new workloads and reinforces only the routes that genuinely need APISIX-specific plugins through ApisixPluginConfig and the like.
Deploying with Helm — Hands-On
Now let us actually deploy. The simplest path to install APISIX, etcd, and the Ingress Controller together is the official Helm chart.
First, add the chart repository.
helm repo add apisix https://charts.apiseven.com
helm repo update
kubectl create namespace apisix
Here is a production-oriented values example. The key parts are bringing up etcd in HA, enabling the Ingress Controller alongside it, and configuring the Admin key and Prometheus exposure.
apisix:
replicaCount: 3
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: "1"
memory: 1Gi
etcd:
enabled: true
replicaCount: 3
persistence:
enabled: true
size: 10Gi
ingress-controller:
enabled: true
config:
apisix:
serviceNamespace: apisix
resources:
requests:
cpu: 200m
memory: 256Mi
service:
type: LoadBalancer
metrics:
serviceMonitor:
enabled: true
The install command is as follows.
helm install apisix apisix/apisix \
--namespace apisix \
-f values-prod.yaml
After installation, confirm that all core components are up.
kubectl -n apisix get pods
kubectl -n apisix get svc
kubectl -n apisix get crd | grep apisix
If the CRD list shows apisixroutes, apisixupstreams, apisixpluginconfigs, and so on, the controller is installed correctly. Now you can apply the ApisixRoute discussed earlier and verify routing by sending a request to the external IP of the LoadBalancer service.
EXTERNAL_IP=$(kubectl -n apisix get svc apisix-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
curl -H "Host: api.example.com" "http://$EXTERNAL_IP/orders/123"
Operations and Tuning
A finished deployment does not mean operations are finished. To run APISIX reliably in production, you need to take care of several essentials.
etcd Is a Core Dependency
As emphasized earlier, etcd is the single source of truth for configuration. Therefore etcd's availability governs your ability to change gateway configuration. The following principles are recommended.
- Always configure etcd with an odd number of nodes (3 or 5) to ensure quorum.
- Use fast SSDs for etcd disks and monitor fsync latency. etcd is highly sensitive to disk latency.
- Automate regular snapshot backups and actually rehearse the recovery procedure.
- It is reassuring that the data plane keeps serving traffic with its last received configuration even if it is briefly disconnected from etcd, but no new configuration is applied in this state, so you must detect disconnection quickly.
Resource and Worker Tuning
Because apisix is OpenResty-based, the number of worker processes and the size of shared memory directly affect performance. Set workers to match the node's CPU count, and if you have many routes and plugins, allocate generous shared dictionary sizes. Setting CPU requests and limits too tightly causes latency spikes due to throttling, so it is best to find appropriate values through load testing.
Health Signals from Metrics
Among the metrics that the prometheus plugin exposes, the ones that matter especially in operations are these.
| Metric type | Why you should watch it |
| --- | --- |
| Request latency distribution | Separate gateway-side overhead from backend latency |
| Counts by status code | A surge in 4xx signals auth/rate limit, a surge in 5xx signals upstream failure |
| Upstream health state | Track whether health checks are removing backends |
| etcd connection state | Detect early whether configuration sync has broken |
The Real-World Value of Zero-Downtime Changes
In operations, you end up changing routes frequently. Adding a new service, adjusting a canary ratio, tuning a rate-limit threshold, and emergency IP blocking are all applied immediately with nothing more than an etcd configuration change. With no reload, the change itself becomes a lightweight operation, and that leads to operational agility. In particular, being able to immediately block a specific origin with ip-restriction during security incident response, with zero downtime, makes a big difference in practice.
Common Pitfalls and Troubleshooting
Finally, let us summarize the problems you frequently run into in the field and the direction for resolving them.
You Applied a Route but Get a 404
This is the most common symptom. The order of checks is as follows.
- Confirm that ingressClassName or gatewayClassName is set exactly to apisix. If the class does not match, the controller ignores that resource.
- Look at the controller logs. If schema validation fails during translation, the controller does not apply the change to etcd.
- Confirm that the target Service and port actually exist and that endpoints are populated.
kubectl -n apisix logs deploy/apisix-ingress-controller
kubectl -n shop get endpoints orders-svc
kubectl -n shop describe apisixroute orders-route
Configuration Does Not Reach etcd
This is the case where the controller is alive but the change does not reach the data plane. A mismatched Admin API authentication key, a network policy blocking traffic between the controller and apisix, and loss of etcd quorum are the usual culprits. Start by checking whether there are logs of the controller failing to call the Admin API.
Plugin Configuration Schema Errors
Plugin config has a strict schema per plugin. Entering a wrong field or type can cause the entire route to be rejected. If you suspect this, run an isolated test by attaching just a single plugin to a small route, and follow the validation error messages in the controller logs verbatim to narrow down the cause quickly.
Testing Without a Host Header Causes Routing Misses
Routes with host matching require the Host header to match exactly. If you hit the external IP with curl but omit the Host header, the route will not match and you get a 404. As in the curl example in the hands-on section above, always specify the Host header when verifying.
Delayed Configuration Apply Due to etcd Disk Latency
etcd is sensitive to disk fsync, so on a slow disk or under overload, the commit of a configuration PUT is delayed, and as a result the data plane apply also lags. The preventive measure is to place etcd nodes on separate fast disks and to always monitor etcd's own metrics (disk commit latency, leader change frequency).
Conclusion
The essence of the Apache APISIX Ingress Controller ultimately comes down to two things. First, by keeping configuration in etcd and having the data plane watch it, it implements zero-downtime dynamic routing without reloads. Second, by separating the control plane from the data plane to isolate failures, it drives three modes — standard Ingress, native CRDs, and Gateway API — with a single data plane.
Given that in the 2026 ingress landscape the standard Ingress is frozen, Gateway API has established itself as the successor standard, and ingress-nginx faces a growing maintenance burden, APISIX, which delivers rich plugins and dynamic routing together while also supporting Gateway API, is a sufficiently attractive option for a team designing a gateway from scratch. That said, the prerequisite for a successful adoption is to clearly recognize the additional dependency of etcd and the operational responsibility that comes with it, and to design etcd HA, backups, and observability together from the start.
If you are starting fresh, we recommend a hybrid strategy that uses Gateway API as the standard skeleton and reinforces only the points that need APISIX's plugins with native resources. If you have existing Ingress assets, a realistic path is to move them into standard Ingress mode and then transition gradually.
References
- Apache APISIX official site: https://apisix.apache.org
- Getting started with the APISIX Ingress Controller: https://apisix.apache.org/docs/ingress-controller/getting-started/
- APISIX plugin hub: https://apisix.apache.org/docs/apisix/plugins/limit-count/
- APISIX Ingress Controller GitHub: https://github.com/apache/apisix-ingress-controller
- Kubernetes Ingress concept documentation: https://kubernetes.io/docs/concepts/services-networking/ingress/
- Gateway API official documentation: https://gateway-api.sigs.k8s.io/
- etcd official documentation: https://etcd.io/docs/
- cert-manager official documentation: https://cert-manager.io/docs/
- APISIX Helm chart repository: https://github.com/apache/apisix-helm-chart
현재 단락 (1/413)
For years, the gateway that let external traffic into a Kubernetes cluster was effectively defined b...