Skip to main content

Provider Trait

All providers implement LlmProvider:

AnthropicProvider

Auth Refresh

GeminiProvider

OpenAIProvider

Uses the OpenAI Responses API.

Environment Variables

Custom Base URL

The base URL is configurable, which makes it easy to point the provider at a local proxy, Azure OpenAI, or any OpenAI-compatible endpoint:
Or in code:

Dynamic Auth (Proxy / JWT)

Notes

  • Thinking / reasoning: ThinkingConfig is fully supported. The budget maps to OpenAI’s reasoning.effort parameter (low, medium, high) and reasoning summaries are streamed as thinking blocks. See Extended Thinking for budget-to-effort mapping.
  • Prompt caching: OpenAI does not use Anthropic-style prompt caching. Disable it in AgentConfig to avoid unnecessary overhead:
  • Tool schema: Tools are translated to OpenAI function definitions automatically. Optional parameters are supported (strict mode is disabled).

SwappableLlmProvider

Change LLM at runtime:

Extended Thinking

Supported by all three providers:

Provider Differences

AnthropicProvider: Uses the budget directly as thinking tokens. GeminiProvider: Automatically converts budget to the appropriate format:
  • Gemini 3 models: Budget maps to thinking levels (minimal, low, medium, high)
    • 0-512minimal
    • 513-2048low
    • 2049-8192medium
    • 8193+high
  • Gemini 2.5 models: Uses numeric budget directly
OpenAIProvider: Budget maps to reasoning.effort and automatically requests reasoning summaries:
  • 0-2048low
  • 2049-8192medium
  • 8193+high
Reasoning summaries are streamed as ContentBlock::Thinking blocks, and reasoning tokens are tracked in Usage.thoughts_token_count. The conversion is automatic based on the provider, so you use the same .with_thinking() API for all three providers.

Message Types

Content Blocks