Kong AI Gateway 2.0 brings unified AI security to GKE, Cloud Run and Vertex AI

A control plane for AI-native traffic

Kong AI Gateway 2.0 rebuilds the gateway around AI-specific constructs: model providers, agents, MCP tool servers and AI consumers become first-class control-plane entities rather than settings attached to generic HTTP plugins. According to a dev.to walkthrough from a Google Cloud community author, the goal is to give teams that already split workloads between Google Kubernetes Engine and Cloud Run — and call Gemini models through Vertex AI — a single layer for authentication, observability and cost governance.

Where traditional gateways fall short

The post's core argument is that classic API gateways assume a straight-line request: a client calls an upstream API and receives a response. Agentic systems do not work that way. LLM pipelines, MCP servers and autonomous agents loop, invoke tools and negotiate with other agents, producing traffic patterns and risk profiles that a deterministic proxy was never designed to handle.

Three capability gaps stand out:

• Quotas need to be measured in input and output tokens rather than request counts, since a single prompt can consume far more resources than a typical REST call.
• Prompts themselves need runtime inspection, alongside model routing and credential abstraction.
• Tool discovery has to be governed dynamically, because agents resolve which tools exist at runtime instead of calling fixed endpoints.

First-class entities instead of plugins

Version 2.0 replaces downstream plugin attachment with dedicated abstractions. Model providers and models act as decoupled backends managing upstream connection pools, fallbacks and credential routing. MCP servers become entities representing tool providers. Agents get secure channels for agent-to-agent traffic. Consumers and policies define identity-scoped limits on spend, prompt guardrails and access rights.

Configuration lives in Kong Konnect, separated from the data plane, so runtime operations can scale across private networks without configuration systems being exposed to data streams.

Running across GKE and Cloud Run

On GKE, the data plane deploys through the Kong Kubernetes Operator, with models and policies configured as custom resources. The gateway authenticates upstream using Workload Identity Federation, inheriting Google Cloud IAM roles and doing away with static service account keys.

Cloud Run services, meanwhile, stay locked to private internal ingress. All inbound client and agent requests route through the gateway for authentication and logging, and both GKE services and Cloud Run endpoints sit under the same consumer-facing namespace — one security stack instead of two.

Vertex AI integration and model fallbacks

Rather than distributing service account keys to each application, model consumption is routed through the gateway, which mints OAuth tokens via Workload Identity to reach Vertex AI's standard and enterprise endpoints. Fallback chains — for example a primary Gemini Ultra model failing over to Gemini Flash — are configured at the gateway without any client-side changes, and routing can direct internal teams to standard tiers while reserving high-throughput, low-latency capacity for mission-critical applications.

MCP bundling and agent-to-agent governance

Instead of an agent opening connections to dozens of separate tool endpoints, the gateway aggregates tool namespaces into a single endpoint. It also intercepts the tool-list negotiation itself: an agent that lacks permission for, say, a billing tool never sees that tool in its discovery response, which the post says prevents both unauthorized access and agents attempting to use capabilities they were never granted.

For agents communicating across microservice boundaries, the gateway applies mutual authentication, traces execution graphs and enforces rate-limiting policies intended to stop unconstrained recursion loops.

Why it matters

Cloud teams rarely standardize on one compute model: stateful orchestration lands on GKE, stateless APIs on Cloud Run, and inference on Vertex AI. The architecture described here separates those runtime decisions from governance — authentication, token-metered cost control, tool permissions and agent identity all live in one control plane regardless of where the workload runs. As MCP servers and multi-agent systems spread, maintaining duplicate security stacks per runtime becomes expensive and error-prone.

One caveat: the details come from a single community walkthrough rather than independent coverage, so teams evaluating the platform should verify behaviour against their own requirements. The direction, however, is clear — AI traffic increasingly needs gateway-level controls that conventional API proxies do not provide.