xAI

Verified : Code examples on this page have been automatically tested and verified.

Route agentgateway LLM traffic to xAI’s Grok models.

Configure xAI (Grok) as an LLM provider in agentgateway.

Configuration

You can add the model in the UI or in your configuration file.

To add the model in the UI, follow these steps. The steps create the same model as the Config file tab.

Set your API key in the XAI_API_KEY environment variable. The UI checks that the variable exists when you save the model.
```
export XAI_API_KEY='<your-api-key>'
```
Create a configuration file with an empty llm section, and start agentgateway in the same terminal. Without an llm section, the UI does not show the LLM > Models page.
```
cat > config.yaml <<'EOF'
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models: []
EOF
agentgateway -f config.yaml
```
Open LLM > Models, such as http://localhost:15000/ui/llm/models, and click Add model.
From the Provider dropdown, select xAI (Grok).
Keep the default xai/* in the Incoming model match field.
For the Provider API key, select Env var, and enter XAI_API_KEY.
For Outgoing model, keep Strip xai/. This option removes the xai/ prefix before the request is forwarded. To send every request to one model instead, select Explicit, and enter the model name. This option sets params.model.
Click Save model.

The form has no field for params.baseUrl. To override the base URL, click the Raw icon (<>) in the drawer header and add the field, or use the Config file tab.

Whether the UI can save depends on your storage mode. For more information, see Save changes from the UI.

Review the following example configuration.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "xai/*"
    provider: xai
    params:
      apiKey: "$XAI_API_KEY"
    transformation:
      model: llmRequest.model.stripPrefix("xai/")

Review the following table to understand this configuration.

Setting	Description
`name`	The model name to match in incoming requests. When a client sends `"model": "<name>"`, the request is routed to this provider. The `xai/*` pattern matches any model name that starts with `xai/`.
`provider`	The LLM provider, set to `xai`.
`params.model`	Optional. If set, this model is used for all requests. If not set, the request must include the model to use.
`params.apiKey`	Your xAI API key. You can reference environment variables using the `$VAR_NAME` syntax.
`params.baseUrl`	Optional. Overrides the provider base URL. Default: `https://api.x.ai/v1`.
`transformation.model`	Removes the `xai/` prefix from the model name before the request is forwarded, so the provider receives its own model name.

Example request

After running agentgateway with the configuration from the previous section, you can send an OpenAI-compatible request to the v1/chat/completions endpoint:

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "xai/grok-2-latest",
    "messages": [{"role": "user", "content": "Hello from Grok!"}]
  }'

TypeSafe Multiple LLM providers

Was this page helpful?

xAI

Configuration

Example request

What could be improved?