# Configure the Berth AI SRE, providers, budgets & settings

> Set up the built-in AI SRE agent with your own model: local Ollama (Community), or Claude via the Anthropic API or Amazon Bedrock on your own key (Enterprise). Spend guardrails, the usage ledger, the Settings page, and the full environment-variable reference.

The agent runs in-process in the Berth binary and talks to the provider you configure, with your credentials. For the safety model see [How the AI works](https://berth.agrohi.com/how-ai-works/).

## Using the assistant

Open it with **Ask AI** on any page or resource, the **AI SRE** pill, or ⌘J / Ctrl+J. One conversation follows you across pages, grounded on what you are viewing. The header shows the provider picker, today's cloud tokens against budget, and snapshot freshness. It is read-only: it inspects live state with tools, shows each tool call, proposes fixes as manifests, and never applies them.

Ask about: health & triage; high availability & SPOFs (any operator); node drains & PDBs; right-sizing; opt-in logs on demand (previous container for crash loops); any resource incl. CRDs; capacity planning; runbooks & the cluster's incident memory.

## Example prompts

- How healthy is my cluster? Is anything broken right now?
- Are my databases highly available? What breaks if one node fails?
- If I drain a node for maintenance, will any PodDisruptionBudget block it?
- Is <deployment> in <namespace> over- or under-provisioned? Show requests vs usage.
- Why is <pod> crash-looping? Show the previous container's logs.
- Which operators or custom resources are installed here?

## Choose a provider

| Provider id | Where inference runs | Billed by | Tier |
| --- | --- | --- | --- |
| ollama | Your compute | nobody | Community |
| anthropic | api.anthropic.com | Anthropic, your account | Enterprise |
| claude (alias bedrock) | Amazon Bedrock, your region | AWS, your account | Enterprise |

Berth never bills for tokens: no metering server, no credit system.

## Quickstart

Ollama (local):
```
ollama pull sabbir/berth-sre
AI_PROVIDERS=ollama
OLLAMA_HOST=http://localhost:11434
OLLAMA_MODEL=sabbir/berth-sre
OLLAMA_NUM_CTX=16384
```
Keep OLLAMA_NUM_CTX at 16384; raise AI_RUN_TIMEOUT_SECONDS to 240 for thinking models.

Claude via the Anthropic API (Enterprise, needs dashboard auth):
```
AI_PROVIDERS=anthropic,ollama
ANTHROPIC_API_KEY=sk-ant-…
ANTHROPIC_MODEL=claude-opus-5      # or claude-sonnet-5
```

Claude via Amazon Bedrock (Enterprise):
```
AI_PROVIDERS=claude,ollama
AWS_BEARER_TOKEN_BEDROCK=…
AWS_REGION=us-east-1
MODEL=anthropic.claude-opus-4-8
```

Cloud providers require authentication, Enterprise and persistent AI_DATA_DIR. Unauthenticated cloud access is not supported.

## Spend guardrails

```
AI_BUDGET_DAILY_TOKENS=2000000
AI_BUDGET_MONTHLY_TOKENS=20000000
AI_RATE_LIMIT_PER_MINUTE=20
AI_MAX_CONCURRENT_RUNS=4
AI_RESERVE_TOKENS=20000
```
Checked before a run starts with an estimated in-flight reservation within one process. Actual usage can exceed the reservation; this is not a guaranteed billing ceiling. Use provider-side controls too. Zero = unlimited. Budgets are also editable on the Settings page.

## Usage ledger

Every run is recorded in SQLite under AI_DATA_DIR: principal, provider, model, tokens (input/output/cached), tool calls, duration, status, focused resource, estimated cost. Settings → AI → Usage shows today, this month, per-provider breakdown, recent runs. Cost is an estimate (built-in table or AI_PRICE_IN_PER_MTOK / AI_PRICE_OUT_PER_MTOK).

## Settings page & persistence

Settings → AI changes providers, hosts, models, keys (write-only) and budgets live; editing requires dashboard auth. Set `AI_DATA_DIR=/data/ai` on a volume to persist settings (ai-config.json, 0600), the ledger and incident memory (ai.db).

## Config reference

| Variable | Default | Effect |
| --- | --- | --- |
| AI_PROVIDERS | ollama | providers to build: ollama, anthropic, claude/bedrock |
| AI_PROVIDER | first listed | default provider |
| OLLAMA_HOST | http://localhost:11434 | Ollama base URL |
| OLLAMA_MODEL | sabbir/berth-sre | model tag |
| OLLAMA_NUM_CTX | 16384 | context window |
| OLLAMA_MAX_TOKENS | 8192 | max tokens per local reply |
| ANTHROPIC_API_KEY | - | Anthropic API key |
| ANTHROPIC_MODEL | claude-opus-5 | Anthropic model |
| ANTHROPIC_BASE_URL | - | optional gateway |
| AWS_BEARER_TOKEN_BEDROCK | - | Bedrock bearer token (or credential chain) |
| AWS_REGION | - | Bedrock region |
| MODEL | anthropic.claude-opus-4-8 | Bedrock model id |
| EFFORT | high | low/medium/high/xhigh/max |
| AI_THINKING | adaptive | adaptive or off |
| MAX_TOKENS | 8192 | max tokens per Claude reply |
| AI_MAX_TOOL_ITERATIONS | 8 | model round-trips per run |
| AI_BUDGET_DAILY_TOKENS | 0 | cloud tokens per UTC day |
| AI_BUDGET_MONTHLY_TOKENS | 0 | cloud tokens per month |
| AI_RESERVE_TOKENS | 20000 | hold per in-flight cloud run |
| AI_RATE_LIMIT_PER_MINUTE | 20 | per principal |
| AI_MAX_CONCURRENT_RUNS | 4 | concurrent runs |
| AI_PRICE_IN_PER_MTOK / AI_PRICE_OUT_PER_MTOK | - | USD per MTok for estimates |
| AI_DATA_DIR | - | SQLite ledger/memory + settings |
| AI_CONFIG_PATH | $AI_DATA_DIR/ai-config.json | settings file |
| AI_ALLOW_RAW_DIAGNOSTICS | false | opt in to logs/descriptions reaching the model |
| AI_RUN_TIMEOUT_SECONDS | 180 | wall-clock bound per run |
| AI_SNAPSHOT_INTERVAL_SECONDS | 25 | snapshot refresh |

## FAQ

- **Does my cluster data leave the cluster?** With Ollama, inference goes to the server you configured. With Claude, the text the tools returned for that turn goes to Anthropic or AWS under your own account. Secrets are never read.
- **Can the assistant change my cluster?** No. Every cluster tool is read-only; there is no apply path.
- **Does it cost money?** Ollama is free. Claude is billed by Anthropic/AWS to your account; Berth adds nothing.
- **Why can't I pick Claude?** Unauthenticated dashboard, Community license, or no key. The Settings page says which.
- **Can I use OpenAI/Gemini?** Not yet; on the roadmap.

Security defaults: cloud AI requires persistent AI_DATA_DIR and authentication. Raw logs/descriptions require AI_ALLOW_RAW_DIAGNOSTICS=true; keep off until data egress is approved. Use one replica with local RWO storage. Token reservations are estimates, not a provider billing ceiling.