Skip to main content

Using Vercel AI Gateway With Controller

The AI Gateway provides a unified API to access hundreds of models through a single endpoint. It gives you the ability to set budgets, monitor usage, load-balance requests, and manage fallbacks.

Useful links:

Getting an API Key​

An API key is required for authentication.

  1. Sign Up/Sign In: Go to the Vercel Website and sign in.
  2. Get an API Key: Go to the API Key page in the AI Gateway tab. Create a new key.
  3. Copy the Key: Copy the API key.

Supported Models​

The Vercel AI Gateway supports a large and growing number of models. Controller automatically fetches the list of available models from the https://ai-gateway.vercel.sh/v1/models endpoint. Only language models are shown.

The default model is anthropic/claude-sonnet-4 if no model is selected.

Refer to the Vercel AI Gateway Models page for the complete and up-to-date list.

Model Capabilities​

  • Vision Support: Many models support image inputs.
  • Tool/Computer Use: Select models support function calling and computer use.

Check the model description in the dropdown for specific capabilities.

Configuration in Controller​

  1. Open Controller Settings: Click the gear icon () in the Controller panel.
  2. Select Provider: Choose "Vercel AI Gateway" from the "API Provider" dropdown.
  3. Enter API Key: Paste your Vercel AI Gateway API key into the "Vercel AI Gateway API Key" field.
  4. Select Model: Choose your desired model from the "Model" dropdown.

Prompt Caching​

Vercel AI Gateway supports automatic prompt caching for select models including Anthropic Claude and OpenAI GPT models. This reduces costs by caching frequently used prompts.

Tips and Notes​

  • Model Selection: The Vercel AI Gateway offers a wide range of models. Experiment to find the best one for your needs.
  • Pricing: The Vercel AI Gateway charges based on the underlying model's pricing, including costs for cached prompts. See the Vercel AI Gateway Models page for details.
  • Temperature: The default temperature is 0.7 and is configurable per model.
  • Bring Your Own Key (BYOK): The Vercel AI Gateway has no markup if you decide to use your own key for the underlying service.
  • More info: Vercel does not add rate limits. Upstream providers may. New accounts receive $5 credits every 30 days until the first payment.
Last updated on