Item detail

envoyproxy/ai-gateway

envoyproxy/ai-gateway is an Apache-2.0-licensed, CNCF-affiliated AI Gateway project from the Envoy community that governs Unified Access to Generative AI services on top of Envoy Gateway, so platform teams and SREs can route, observe, and rate-limit LLM traffic with the same Envoy data plane they already trust for non-AI workloads.

Score8.7
Popularity8.5
Risklow
TierGold
Score breakdown
Usefulness9.0
Novelty8.0
Momentum9.0
Maturity6.9
Open-source/build8.4
Evidence7.2
Workflow potential10.0
Setup ease6.4

Popularity is tracked separately. Support, ads, sponsorships, and tips never affect these signals.

Why it matters

Useful for platform engineers, SREs, and Kubernetes operators who already run Envoy Gateway, because Envoy AI Gateway reuses the existing Envoy data plane for LLM routing and governance, which means a single control plane can govern both traditional microservice traffic and LLM-backed traffic without running a second proxy stack.

Who should use it

Platform engineers and SREs who already run Envoy Gateway and want LLM traffic on the same data planeKubernetes operators who need to route, observe, and rate-limit LLM calls alongside microservice trafficTeams evaluating a CNCF-governed LLM gateway instead of a vendor-specific proxy

Who should skip it

Skip if the source link, docs, or setup requirements do not match your workflow.

Risk explanation

It governs LLM traffic on the Envoy data plane, so audit which model providers are routed through the gateway, lock down credentials for upstream LLM APIs, and validate rate-limit and failover behavior before pointing production traffic at it.

Evidence links

Closest alternatives / related signals

ai-gatewayenvoycncfkubernetesllm-routinginferenceapi-gatewayapache-2.0