> ## Documentation Index
> Fetch the complete documentation index at: https://docs.framework.vibeworkapp.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Providers

> Pluggable LLM backends - Anthropic, Gemini, OpenAI, and custom providers

## Provider Trait

All providers implement `LlmProvider`:

```rust theme={null}
use picrust::llm::{LlmProvider, AnthropicProvider, GeminiProvider, OpenAIProvider};

let anthropic: Arc<dyn LlmProvider> = Arc::new(AnthropicProvider::from_env()?);
let gemini: Arc<dyn LlmProvider> = Arc::new(GeminiProvider::from_env()?);
let openai: Arc<dyn LlmProvider> = Arc::new(OpenAIProvider::from_env()?);

let agent = StandardAgent::new(config, openai);
```

## AnthropicProvider

```rust theme={null}
use picrust::llm::AnthropicProvider;

// From env vars (ANTHROPIC_API_KEY, ANTHROPIC_MODEL)
let llm = AnthropicProvider::from_env()?;

// Explicit key and model
let llm = AnthropicProvider::new("sk-ant-...")?
    .with_model("claude-sonnet-4-5@20250929");

// Custom max tokens
let llm = AnthropicProvider::from_env()?
    .with_max_tokens(8192);
```

### Auth Refresh

```rust theme={null}
use picrust::llm::{AuthProvider, AuthConfig};

struct MyAuthProvider;

#[async_trait]
impl AuthProvider for MyAuthProvider {
    async fn get_auth(&self) -> anyhow::Result<AuthConfig> {
        let token = refresh_jwt_token().await?;
        Ok(AuthConfig::new(token))
    }
}

let llm = AnthropicProvider::with_auth_provider_boxed(Arc::new(MyAuthProvider));
```

## GeminiProvider

```rust theme={null}
use picrust::llm::GeminiProvider;

// From env vars (GEMINI_API_KEY, GEMINI_MODEL)
let llm = GeminiProvider::from_env()?;

// Explicit key and model
let llm = GeminiProvider::new("api-key")?
    .with_model("gemini-2.0-flash-exp");
```

## OpenAIProvider

Uses the [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses).

```rust theme={null}
use picrust::llm::OpenAIProvider;

// From env vars (OPENAI_API_KEY, OPENAI_MODEL)
let llm = OpenAIProvider::from_env()?;

// Explicit key and model
let llm = OpenAIProvider::new("sk-...")?
    .with_model("gpt-4o")
    .with_max_tokens(4096);
```

### Environment Variables

| Variable            | Required | Default                               |
| ------------------- | -------- | ------------------------------------- |
| `OPENAI_API_KEY`    | Yes      | —                                     |
| `OPENAI_MODEL`      | Yes      | —                                     |
| `OPENAI_BASE_URL`   | No       | `https://api.openai.com/v1/responses` |
| `OPENAI_MAX_TOKENS` | No       | `32000`                               |

### Custom Base URL

The base URL is configurable, which makes it easy to point the provider at a local proxy, Azure OpenAI, or any OpenAI-compatible endpoint:

```bash theme={null}
export OPENAI_BASE_URL="https://my-proxy.example.com/v1/responses"
```

Or in code:

```rust theme={null}
let llm = OpenAIProvider::new("sk-...")?
    .with_model("gpt-4o")
    .with_base_url("https://my-proxy.example.com/v1/responses");
```

### Dynamic Auth (Proxy / JWT)

```rust theme={null}
use picrust::llm::{OpenAIProvider, AuthConfig};

let llm = OpenAIProvider::with_auth_provider(|| async {
    let token = my_auth_service.get_fresh_token().await?;
    Ok(AuthConfig::with_base_url(
        token,
        "https://my-proxy.example.com/v1/responses",
    ))
});
```

### Notes

* **Thinking / reasoning**: `ThinkingConfig` is fully supported. The budget maps to OpenAI's `reasoning.effort` parameter (`low`, `medium`, `high`) and reasoning summaries are streamed as thinking blocks. See [Extended Thinking](#extended-thinking) for budget-to-effort mapping.
* **Prompt caching**: OpenAI does not use Anthropic-style prompt caching. Disable it in `AgentConfig` to avoid unnecessary overhead:
  ```rust theme={null}
  let config = AgentConfig::new().with_prompt_caching(false);
  ```
* **Tool schema**: Tools are translated to OpenAI function definitions automatically. Optional parameters are supported (strict mode is disabled).

## SwappableLlmProvider

Change LLM at runtime:

```rust theme={null}
use picrust::llm::SwappableLlmProvider;

let swappable = Arc::new(SwappableLlmProvider::new(anthropic_provider));

// Use with agent
let agent = StandardAgent::new(config, swappable.clone());

// Swap provider later
swappable.swap(gemini_provider).await;
```

## Extended Thinking

Supported by all three providers:

```rust theme={null}
use picrust::llm::ThinkingConfig;

let config = AgentConfig::new()
    .with_thinking(16000);  // Budget in tokens

// Or with config object
let config = AgentConfig::new()
    .with_thinking_config(ThinkingConfig::enabled(32000));
```

### Provider Differences

**AnthropicProvider**: Uses the budget directly as thinking tokens.

**GeminiProvider**: Automatically converts budget to the appropriate format:

* **Gemini 3 models**: Budget maps to thinking levels (`minimal`, `low`, `medium`, `high`)
  * `0-512` → `minimal`
  * `513-2048` → `low`
  * `2049-8192` → `medium`
  * `8193+` → `high`
* **Gemini 2.5 models**: Uses numeric budget directly

**OpenAIProvider**: Budget maps to `reasoning.effort` and automatically requests reasoning summaries:

* `0-2048` → `low`
* `2049-8192` → `medium`
* `8193+` → `high`

Reasoning summaries are streamed as `ContentBlock::Thinking` blocks, and reasoning tokens are tracked in `Usage.thoughts_token_count`.

The conversion is automatic based on the provider, so you use the same `.with_thinking()` API for all three providers.

## Message Types

```rust theme={null}
use picrust::llm::{Message, ContentBlock};

let msg = Message::user("Hello");
let msg = Message::assistant("Hi");

// With blocks
let msg = Message::user_with_blocks(vec![
    ContentBlock::text("Analyze this"),
    ContentBlock::image(image_data, "image/png"),
]);
```

## Content Blocks

```rust theme={null}
match &message.content {
    MessageContent::Blocks(blocks) => {
        for block in blocks {
            match block {
                ContentBlock::Text { text } => {},
                ContentBlock::ToolUse { id, name, input } => {},
                ContentBlock::ToolResult { tool_use_id, content, is_error } => {},
                ContentBlock::Thinking { thinking, .. } => {},
                ContentBlock::Image { source } => {},
                ContentBlock::Document { source, .. } => {},
                _ => {}
            }
        }
    }
    _ => {}
}
```