API Gateway Pattern in Production 2026: Kong, AWS, Envoy, BFF

Q: What is an API gateway and why does it matter in microservices?

An API gateway is a single entry point that sits between external clients and internal services in a microservices architecture. Every external request passes through it, which makes it the natural place to enforce cross-cutting concerns uniformly: authentication, rate limiting, request and response transformation, routing, and observability. Without a gateway, each service has to implement these concerns independently — which inevitably leads to inconsistent behaviour and security gaps when one service forgets a check that the others enforce. The gateway pattern lets services focus on their domain logic while the gateway handles the cross-cutting concerns that would otherwise pollute every service. For external-facing systems, the gateway is also the primary security boundary, where TLS termination, DDoS protection, and WAF rules apply.

Q: What is the difference between an edge gateway, a mesh gateway, and a Backend for Frontend (BFF)?

An edge gateway sits at the boundary between external clients and the system, handling internet-facing concerns: TLS termination, DDoS protection, public-facing authentication, rate limiting. Implementations include Kong, AWS API Gateway, and Apigee. A mesh gateway (or ingress gateway) operates inside a service mesh, handling traffic entering the mesh from outside and integrating with the mesh's mTLS and policy infrastructure — Istio's gateway and Envoy as a standalone ingress are typical examples. A Backend for Frontend (BFF) is a gateway specialised for a particular client type — mobile app, web app, partner API — that aggregates and transforms responses from internal services to match exactly what that client needs. These patterns are not mutually exclusive: many production systems deploy an edge gateway for security and ingress, a mesh gateway for cluster-internal concerns, and one or more BFFs for client-specific aggregation.

Q: What concerns belong in the gateway and what does not?

Authentication and identity verification, routing and service discovery, rate limiting and throttling, light request and response transformation (header manipulation, format conversion), and observability all belong in the gateway. These are infrastructure concerns that apply uniformly across services. Business logic, schema validation specific to a single service, complex multi-service orchestration, and feature-flag evaluation that affects business behaviour belong in services rather than in the gateway. The test is whether changing the logic requires understanding domain context — if it does, it belongs in a service that owns that domain. The gateway configuration should read as infrastructure rules: route, authenticate, rate-limit, transform headers, observe. When the gateway configuration starts requiring extensive comments to explain business rules, those rules have leaked from where they belong.

Q: Which API gateway should I choose: Kong, AWS API Gateway, Envoy, or Traefik?

The choice depends on the deployment context. Kong is the strongest general-purpose choice for self-hosted or hybrid deployments with a mature plugin ecosystem and both open-source and managed (Kong Konnect) options. AWS API Gateway is the right choice for AWS-native serverless workloads where the deep integration with Lambda, Cognito, and IAM outweighs the AWS lock-in (use HTTP APIs over REST APIs for new workloads — lower cost, simpler). Envoy and Envoy-based gateways (Solo.io Gloo, Istio gateway) are appropriate when raw performance matters or when service mesh integration is the dominant consideration; higher operational complexity. Traefik is the natural choice for Kubernetes-first deployments where the operational simplicity of configuration via Kubernetes resources is the priority. Apigee, Azure API Management, and Tyk are enterprise-focused options with stronger developer-portal, monetisation, and governance features for organisations where external API consumers are a primary concern.

Q: How do I prevent the API gateway from becoming a single point of failure?

Run multiple gateway instances behind a layer-4 load balancer — the minimum production deployment is three or more instances with health checks removing unhealthy instances from rotation. The gateway itself must be stateless: rate-limit counters live in Redis, authentication-token caches live in a shared cache, routing configuration lives in a configuration store. Statelessness is what allows gateway instances to be added, removed, and replaced without coordination. Plan for control plane outages — the data plane (request-handling proxies) must continue serving traffic when the control plane (configuration management) is unavailable, and this should be tested explicitly. Capacity planning must include headroom for the failure of one or more instances or one or more regions; if a region failover doubles traffic to the surviving region, that region's gateway must have enough capacity to handle 2x traffic without latency degradation.

Q: How should rate limiting be implemented in an API gateway?

Rate limits should be enforced at the gateway with counters stored in a shared backend (Redis is the standard choice) so that the limit applies across all gateway instances rather than per-instance. Common dimensions are per API key, per user identity, per IP address, per endpoint, and global. Token bucket is the standard algorithm for most APIs; sliding window provides more accurate counting across time windows; leaky bucket is appropriate for smoother throughput shaping. Avoid the rate-limit thundering herd — when a window resets, throttled clients may retry simultaneously and overwhelm backends — by using sliding windows rather than sharp window boundaries, populating the Retry-After header with a jittered value rather than a constant, and recommending exponential backoff with jitter in client SDKs. For LLM and AI workloads, add per-token rate limiting on top of per-request limits, since requests can vary by orders of magnitude in resource cost.

Q: How does the API gateway integrate with observability?

The gateway is the canonical source for request volume, latency, error rate, and request distribution metrics, since every request and response passes through it. Configure the gateway to emit Prometheus-format metrics (or equivalent for your monitoring stack), structured logs in JSON for log aggregation, and OpenTelemetry traces with proper trace propagation to downstream services. The trace propagation step is critical — many gateway deployments break the distributed trace at the gateway boundary by failing to propagate trace headers, which makes cross-service debugging painful. The gateway should extract incoming trace headers (or generate a new trace if absent), propagate them in the outbound request to the downstream service, and emit its own span for the gateway hop. Per-customer usage analytics for billing, capacity planning, and product analytics are typically derived from gateway logs.

Q: How does an API gateway differ from a service mesh, and do I need both?

An API gateway handles north-south traffic — requests entering the system from external clients — while a service mesh handles east-west traffic — service-to-service communication inside the cluster. The gateway focuses on authentication, rate limiting, edge security, and routing from clients to services. The mesh focuses on mTLS between services, fine-grained retry and circuit-breaking policies, service-to-service authorisation, and detailed observability of internal traffic. The two patterns are complementary, not competitive. Most mature production architectures deploy both: an edge gateway for external traffic, a service mesh for internal traffic, and often a mesh ingress gateway providing the bridge between them. For smaller deployments, a single gateway often handles both roles initially; as the system grows and internal service-to-service complexity increases, adding a service mesh becomes worthwhile to avoid pushing all the per-service concerns into the gateway configuration.

Satyam Kumar

Back to Blog

ai-architecture

API Gateway in Production: The Single Entry Point Pattern (2026)

By Satyam KumarApril 18, 202615 min read

api gateway pattern microservices patterns kong gateway aws api gateway envoy proxy traefik authentication rate limiting edge gateway backend for frontend service mesh distributed systems software architecture system design

API Gateway in Production: The Single Entry Point Pattern (2026)

Frequently Asked Questions

Share this article

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

AI & Cloud Architect. Helping teams build systems that scale to millions.

LinkedIn GitHub

API Gateway in Production: The Single Entry Point Pattern (2026)

Frequently Asked Questions

Share this article

Comments

Leave a comment