Migrate from the legacy AI Gateway

The legacy AI Gateway is configured through its operator and the AIGateway resource. The AI Gateway is configured through platform chart values under the ai-gateway-v2 key. Both gateways use the same Enterprise Manager for the directory, budgets, pricing, virtual API keys, and model access, so users, groups, budgets, and keys carry over unchanged and apply the same way on both. What you move is the gateway configuration and the clients' base URLs.

Run both side by side

A platform normally runs one gateway, and the chart refuses to render with both enabled. For a migration window, enable both and acknowledge it:

values.yaml
global:
  stacklok:
    aiGateway:
      enabled: true
    aiGatewayV2:
      enabled: true
    acknowledgeBothDataPlanes: true

Run both only for the migration window. While both run, you're responsible for four things the chart can't check:

Distinct hostnames. Give the AI Gateway its own hostname through httpRoute.hostname, on a listener the AIGateway doesn't serve. If the two routes overlap, nothing decides which gateway serves a request.
Matching screening configuration. Both gateways run the same PII scanning and Bedrock Guardrails code with the same defaults, so equivalent scanning and guardrails values screen requests the same way. The AI Gateway screens only when scanning.enabled and guardrails.enabled are on, scans requests only, and has no AIPolicy authorization. Mirror your screening settings on both.
Two route tables. Each gateway keeps its own model-to-provider mapping. A model added to one is missing from the other until you add it there too.
Two sets of workloads. Both gateways consume cluster resources while installed, including separate Presidio pools when both scan for PII. Both report the same stacklok_ai_gateway_presidio_* metrics, so scope dashboards, alerts, and autoscalers by the pod label app.kubernetes.io/name.

Shared state behaves as one platform:

Spend from both gateways charges the same budgets.
A virtual API key works on both.
Model sets restrict callers on each gateway that enforces them. The legacy gateway enforces them when its model-access webhook is enabled. The AI Gateway enforces them when modelAccess.enabled is on, which is off by default. See Turn on model access.
A user's identity provider token works on both when auth.issuers lists the same issuer and audience as the AIGateway's spec.auth.oidc, with a bindingClaim that matches the claim the directory binds that issuer on.
A PII result cache can share one Redis or Valkey. Each release keeps its entries under its own keys.
Disconnect AI Gateway in Platform Settings stops traffic on both.

Set acknowledgeBothDataPlanes in your own values only, and remove it in the same change that disables the gateway you're retiring.

Map your configuration

The AIGateway fields map to ai-gateway-v2 values as follows. Settings that size the legacy gateway's own components, such as spec.gateway.proxy and each component's replicas and resources, have no direct equivalent. Size the AI Gateway with its own replicaCount and resources values.

`AIGateway` field	AI Gateway value
`spec.providers[]`	`providers[]` with `kind`, `baseUrl`, `models`, `credentialSecretRef`, `region`, `project`, `awsProfile`
`schema: OpenAI`	`kind: openai`
`schema: OpenAI` with the OpenRouter `pathPrefix`	`kind: openrouter`
`schema: Anthropic`	`kind: anthropic`
`schema: AWSAnthropic`	`kind: bedrock-anthropic`
`schema: GCPVertexAI`	`kind: vertex`, for Gemini models only
`schema: GeminiAIStudio`, `schema: GoogleGenerativeLanguage`	`kind: gemini-aistudio`
`schema: AWSBedrock`, `schema: AzureOpenAI`	Not supported
`spec.routes[].match.model`	The model's entry in its provider's `providers[].models`
`spec.routes[].inputSchema`	Not supported. Clients call each model on its provider's API, see Behavior differences
`spec.routes[].backendRefs[].weight` and `.priority`	Not supported
`spec.resilience.retry`	Not configurable
`spec.resilience.passiveHealthCheck`, `circuitBreaker`	Not supported
`spec.resilience.upstreamConnection`, `spec.gateway.bufferLimit`, and `spec.gateway.connectionLimit`	Not configurable
`spec.gateway.streamTimeouts.maxStreamDuration`	`lifecycle.maxStreamDuration`
`spec.gateway.timeouts`, `spec.gateway.streamTimeouts.streamIdleTimeout`, and `spec.routes[].timeouts`	Not configurable
`spec.gateway.listeners` and `tls`	`httpRoute.*` on your existing Gateway
`spec.auth.oidc`	`auth.issuers`, with a `bindingClaim` for each issuer
`spec.auth.oidc.claimToHeaders`	`telemetry.claimLabels`, for token usage labels
`spec.auth.virtualAPIKeys`	`directory.endpoint`, `directory.tokenProjection.enabled`, and `directory.groupsResolutionIssuer`
`spec.auth.authz`, `spec.auth.workloadIssuers`, `spec.policies`, and `AIPolicy`	Not supported
`spec.gateway.stripHeaders`	Not supported
`spec.processor` and `nerProvider`	`scanning.*`
`spec.processor.requestAction`	`scanning.requestAction` and `scanning.entityActions`, with lowercase `block`, `redact`, or `log-only`
`spec.processor.responseAction`	Not supported
`spec.processor.placeholder`	`scanning.placeholder`
`spec.processor.nerProvider.failureAction`	`scanning.ner.failureAction`
`spec.processor.extractionFailureAction`	`scanning.extractionFailureAction`, with lowercase `block` or `continue`
`spec.processor.mutationFailureAction`	`scanning.mutationFailureAction`, with lowercase `block` or `continue`
`spec.processor.nerProvider.resultCache`	`scanning.resultCache`
`spec.guardrails`	`guardrails.*`
`spec.audit.enabled`	`audit.enabled`
`spec.monitoring.otelCollector`	`telemetry.otlp.*`, plus `telemetry.tracing.enabled` for spans
`spec.monitoring.tokenMetrics.prometheusAddr`	`telemetry.prometheus.enabled` and `telemetry.metrics.port`
`spec.monitoring.tokenMetrics.labels`	`telemetry.headerLabels` for headers the caller sends, or `telemetry.claimLabels` for headers set by `claimToHeaders`
`spec.monitoring.spanAttributes`	`telemetry.spanAttributes`
`spec.monitoring.groupLabelAttributes`	`telemetry.groupLabelAttributes`
`spec.monitoring.grafanaDashboards`	Not supported
`spec.networkPolicies`	`networkPolicy.*`
`spec.journaling`	Not supported
`spec.webhooks`	Not supported
Operator `upstream.budgetsWebhook.*` values	`budgets.endpoint` and `budgets.tokenProjection`
Operator `upstream.modelAccessWebhook.*` values	`modelAccess.enabled`

The legacy gateway turns on group label lookup with the platform value global.stacklok.directoryGrpcEndpoint. The AI Gateway uses directory.endpoint with directory.tokenProjection.enabled and directory.groupsResolutionIssuer, and resolves only the labels you list in directory.groupLabels. See Attribute traffic to organizational groups.

Token usage is recorded under a different metric. The legacy gateway's aigw_token_usage_total corresponds to stacklok_ai_gateway_token_usage_total, which adds provider, implementation, and cache_type labels, so update dashboards and billing queries before you move traffic. The counts also differ. Input includes cache reads and writes, which raises input for Anthropic and Bedrock models that use prompt caching, so price cache_read and cache_creation at the provider's cache rates. Gemini output includes thinking tokens. For a virtual API key, a department claim label comes from the key owner's directory group, so add department to directory.groupLabels. See Record token usage.

Provider credentials stay in Kubernetes Secrets. Reference each one with providers[].credentialSecretRef, which names a Secret and the key that holds the credential. The chart refuses inline credentials. You can reference the Secrets your AIGateway uses today when the key holds the credential in the format its provider kind expects. On Amazon EKS, Bedrock and Bedrock Guardrails access needs a Pod Identity association for the AI Gateway's own ServiceAccounts. See Connect model providers.

The legacy gateway's ServiceAccounts don't carry over to the Enterprise Manager allowlists. The platform chart adds the AI Gateway's ServiceAccount by default, but if you set enterprise-manager.grpc.callerAuth.saSubjectAllowlist or httpSaSubjectAllowlist yourself, your list replaces the default. Add system:serviceaccount:<NAMESPACE>:<RELEASE_NAME>-ai-gateway-v2 to both, or the AI Gateway refuses every request. When the release name already contains ai-gateway-v2, the ServiceAccount is named after the release alone.

What the AI Gateway does not support

Check your AIGateway for these features before you migrate. None of them has an AI Gateway equivalent:

Weighted or priority backends, and failover between providers. Each model is served by exactly one provider.
Configurable retries, passive health checks, and circuit breaking. Upstream retries use fixed, built-in bounds.
The AWS Bedrock Converse API and Azure OpenAI providers, and non-Gemini models on Google Vertex AI.
PII scanning of responses. Scanning covers requests. The legacy gateway scans responses whenever spec.processor is configured, applying requestAction when responseAction is unset.
Interaction journaling.
The legacy gateway's management API (/v1/info, /v1/me, and the MCP server catalog). The AI Gateway serves GET /v1/models on its inference listener.
AIPolicy authorization, role-based authorization rules, and workload issuers (spec.auth.workloadIssuers).
Custom webhooks and an operator-defined header strip list.
Hot reload. Configuration and credential changes apply through a rolling restart.

Behavior differences

Each client uses its own API. The AI Gateway forwards requests in the format the client sent and doesn't translate between APIs. OpenAI and OpenRouter models are served on /v1/chat/completions and /v1/responses. Anthropic and Bedrock models are served on /v1/messages and /anthropic/v1/messages. Gemini and Vertex AI models are served on /v1beta. The legacy gateway accepts OpenAI-format requests for every model by default (inputSchema: OpenAI) and translates them for the provider. A client that calls an Anthropic, Bedrock, Gemini, or Vertex AI model through /v1/chat/completions today moves to that provider's API. See Client endpoints.
Unknown models are refused. A model no provider lists returns 404. When no provider serves the Responses API or Gemini streaming at all, those endpoints return 501. Listing the same model twice for the same client API fails startup.
Gemini discovery. Gemini models are listed only on GET /v1beta/models.

See the endpoints reference for the error response format.

Cut over

Install the AI Gateway alongside the legacy gateway with acknowledgeBothDataPlanes, its own hostname, and the budget and directory connections from Configure the AI Gateway.
Recreate your providers and models in providers, using the configuration map. Turn on the same screening you run today, and set modelAccess.enabled if you restrict callers with model sets.
Verify with a test client. Send requests through each client API your users rely on, streaming and non-streaming, with an account that calls the legacy gateway today.
Move clients. Set toolhive-cloud-ui.gatewayProxyUrl to the AI Gateway URL so the console's setup instructions point there, and have users rerun thv llm setup. Point clients that call non-OpenAI models through the OpenAI-compatible API at the provider's own API. Move one group of users at a time.
Delete the AIGateway resource once no traffic reaches it. Do this before step 6, while the AI Gateway operator is still installed, so the operator cleans up the resources it created.
Disable the legacy gateway. Set global.stacklok.aiGateway.enabled: false, remove global.stacklok.acknowledgeBothDataPlanes in the same change, and upgrade. Then delete the provider Secrets that only the AIGateway used.

Next steps

Configure the AI Gateway to set up the gateway.
Connect model providers to recreate your providers.
Control model access to restrict models per group.

Run both side by side​

Map your configuration​

What the AI Gateway does not support​

Behavior differences​

Cut over​

Next steps​