For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Configure tracing in agentgateway: enable OTLP export, set up authentication and TLS, control the sampling rate, filter spans, and customize span attributes.

Agentgateway natively exports distributed traces over OTLP (OpenTelemetry Protocol). Traces include HTTP, MCP, and LLM spans with attributes that follow the OpenTelemetry semantic conventions for generative AI.

Enable tracing

To enable tracing in agentgateway, add a tracing block under the frontendPolicies section and point agentgateway to your OTLP-compatible backend. For sample tracing backend setups, see Sample tracing configurations.

You can enable tracing in the UI or in your configuration file.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: localhost:4317
    randomSampling: true

Set up authentication and TLS

When connecting to an OTLP backend that requires authentication, such as a SaaS observability platform, use the policies.requestHeaderModifier section to add required authentication headers and policies.backendTLS to enable TLS for the connection. You can set them in the UI or in your configuration file.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: <your-otlp-endpoint>.com:443
    protocol: http
    randomSampling: true
    policies:
      backendTLS: {}
      requestHeaderModifier:
        set:
          Authorization: "Basic <api-key>"

Control sampling rate

Use the randomSampling setting to control the fraction of requests for which spans are exported. Set randomSampling: true to sample 100% of requests, or provide a decimal between 0 and 1 for a percentage.

Three sampling settings decide whether a request is traced, and which one applies depends on the incoming request.

Only one setting applies to each request. Because clientSampling defaults to true, agentgateway continues sampled client traces even when randomSampling is false. An unsampled parent is respected by default. Set parentNotSampled: true to trace those requests anyway and propagate a sampled flag (01) upstream. Each setting accepts true, false, or a sampling fraction between 0 and 1.

In the following example, you want to sample 10% of requests.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: localhost:4317
    randomSampling: 0.1   # sample 10% of requests

Filter spans

Use the filter field to write a CEL expression that controls which sampled spans are exported. A span is exported only when the expression evaluates to true. The filter runs after sampling, so it evaluates spans selected by any of the three sampling settings.

The following example exports only spans for requests with an HTTP response code of 400 or greater.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: localhost:4317
    randomSampling: true
    filter: 'response.code >= 400'

You can combine conditions to export specific traffic. The following examples show common filter patterns:

For the full list of available CEL variables, see the CEL variables reference.

Customize span attributes

Agentgateway emits standard OpenTelemetry attributes as shown in Default span attributes. You can add custom attributes to your spans or remove default ones. Note that customizing span attributes does not work on policy call child spans. For more information, see Policy call child spans.

Add span and resource attributes

Use the attributes field to add custom tags to individual spans. Attribute values are evaluated as CEL expressions on every request, so they can be dynamic. For example, you can use a CEL expression to add the user ID from a request header or the HTTP response code.

Use the resources field to describe the agentgateway process itself. Resource values are static values that are added to every span. For example, you can add the name of the agentgateway process, the environment it runs in, what version it is. Your tracing backend uses resource attributes to label and group spans in its service list. The most common resource attribute is service.name, which sets the name that is displayed in your tracing backend. It defaults to agentgateway if not set.

The following example sets a custom service name and deployment environment in resources, and adds the user ID and request path to each individual span by using attributes.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: localhost:4317
    randomSampling: true
    resources:
      service.name: '"my-agentgateway"'
      deployment.environment: '"production"'
    attributes:
      user.id: 'request.headers["x-user-id"]'
      request.path: 'request.path'

Remove span attributes

Use the remove field to drop attributes from spans before your attributes expressions are applied. This setting is useful for stripping default attributes that are redundant or that you do not want to export.

The following example removes the HTTP protocol version and client address from the span. The remove list matches the attribute names on the request span, as listed in Default span attributes. A name that is not on the span, such as the earlier src.addr or http.version, removes nothing.

# yaml-language-server: $schema=https://agentgateway.dev/schema/config
frontendPolicies:
  tracing:
    host: localhost:4317
    randomSampling: true
    remove:
      - client.address
      - network.protocol.version