Configure the AI SRE

Example prompts to try

Health & availability

• How healthy is my cluster? Is anything broken right now?
• Are my databases highly available? What breaks if one node fails?
• What are the single points of failure in this cluster?

Operations & maintenance

• If I drain a node for maintenance, will any PodDisruptionBudget block it?
• Which node pools are over-committed, and can I safely remove a node?
• Summarise the warning events in <namespace> and what's causing them.

Right-sizing & cost

• Is <deployment> in <namespace> over- or under-provisioned? Show requests vs usage.
• Which workloads are wasting the most CPU or memory reservations?

Debugging

• Why is <pod> crash-looping? Show the previous container's logs.
• What's the status of gateway <name> / ingress <name> / PVC <name>?

Discovery

• Which operators or custom resources are installed here?
• Read the <custom-resource> and tell me if it's configured for high availability.

Choose a provider

Start with Ollama if everything must stay on your network or you are on the Community tier. Add Claude on your own key when you want the strongest diagnosis and can send cluster-derived text to your provider account.

Ollama, local & private

ollama pull sabbir/berth-sre

AI_PROVIDERS=ollama
OLLAMA_HOST=http://localhost:11434     # in-cluster: http://ollama.ollama.svc:11434
OLLAMA_MODEL=sabbir/berth-sre
OLLAMA_NUM_CTX=16384

Claude via the Anthropic API, your key

Requires an Enterprise license and dashboard auth (see gates).

AI_PROVIDERS=anthropic,ollama
ANTHROPIC_API_KEY='sk-ant-…'
ANTHROPIC_MODEL=claude-opus-5          # or claude-sonnet-5 for lower cost
# ANTHROPIC_BASE_URL=                  # optional gateway / proxy

Or paste the key in Settings → AI; it is write-only and stored with 0600 permissions under AI_DATA_DIR.

Claude via Amazon Bedrock, your account

Enterprise. Uses a Bedrock bearer token, or the standard AWS credential chain (IRSA, profiles) when no token is set.

AI_PROVIDERS=claude,ollama
AWS_BEARER_TOKEN_BEDROCK='<your-bedrock-token>'
AWS_REGION=us-east-1
MODEL=anthropic.claude-opus-4-8         # default

Pick which providers are offered

AI_PROVIDERS=anthropic,ollama   # built and shown in the picker (default: ollama)
AI_PROVIDER=ollama              # selected when a chat names none (default: first listed)

A listed provider that cannot be built (no key, unreachable host) is skipped with the reason shown on the Settings page, not silently.

Spend guardrails

Admission limits for cloud usage, checked before a run starts. Actual usage can exceed the reservation; this is not a guaranteed billing ceiling. Zero means unlimited.

AI_BUDGET_DAILY_TOKENS=2000000     # cloud tokens per UTC day
AI_BUDGET_MONTHLY_TOKENS=20000000  # per calendar month
AI_RATE_LIMIT_PER_MINUTE=20        # per user, all providers
AI_MAX_CONCURRENT_RUNS=4
AI_RESERVE_TOKENS=20000            # estimated hold per in-flight cloud run

Each in-flight cloud run holds an estimated reservation against the budget within one process. Actual usage can exceed it; use provider-side controls too. Budgets are also editable on the Settings page. When a limit is hit the panel says which one and when it resets.

Usage ledger

Every run, local or cloud, successful or not, is recorded in a SQLite file underAI_DATA_DIR: principal, provider, model, input/output/cached tokens, tool calls, duration, status, the focused resource, and an estimated cost. Settings → AI → Usageshows today, this month, a per-provider breakdown and recent runs.

Cost estimates use a built-in list-price table for known Claude models, or the prices you set with AI_PRICE_IN_PER_MTOK / AI_PRICE_OUT_PER_MTOK. Your provider's invoice is the source of truth.

Configure from the Settings page

Settings → AI lets you change, without a restart: the provider list and default; Ollama host and model; the Anthropic key, model and base URL; the Bedrock token, region and model; and the daily/monthly budgets. Keys are write-only and never shown back. Changes rebuild the engines live.

Persist settings, usage and memory

Mount a volume and point the runtime at it. Without it, Settings changes, the ledger and incident memory live in memory and reset on restart.

AI_DATA_DIR=/data/ai        # ai.db (ledger + memory) and ai-config.json (0600)

Config reference

Providers

Claude behaviour (both transports)

Guardrails, persistence & safety

Tuning

• Local model slow, looping, or empty? Confirm OLLAMA_NUM_CTX=16384, use the tuned model, raise AI_RUN_TIMEOUT_SECONDS.
• Answers cut off? Raise OLLAMA_MAX_TOKENS or MAX_TOKENS.
• Provider missing from the picker? The Settings page lists why it was not built: missing key, unreachable host, auth gate, or Community tier.
• “Budget used up”? The panel names the limit; raise it on the Settings page or wait for the UTC reset.

FAQ

Does my cluster data leave the cluster?

With Ollama, inference goes to the server you configured. With Claude, the text the tools returned for that turn (describe output, events, bounded log tails, resource YAML) goes to Anthropic or AWS under your own account. Secret/ConfigMap content is blocked, and generic resource tools expose limited metadata and replica counters. Logs and descriptions require explicit opt-in and may contain sensitive data. Details in How the AI works.

Can the assistant change my cluster?

No. Every cluster tool is read-only and there is no apply path. It proposes manifests or commands for you to review and apply with your own access.

Does it cost money?

Ollama is free and local. Claude is billed by Anthropic or AWS to your account at their rates; Berth adds nothing and shows an estimate in the ledger.

Why can't I pick Claude?

Either the dashboard is unauthenticated (enable AUTH_MODE=token or proxy-header), the license is Community, persistent AI_DATA_DIR is missing, or no key is set. The Settings page tells you which.

Which model should I use?

Locally, sabbir/berth-sre. For the strongest diagnosis, claude-opus-5; claude-sonnet-5 for lower cost. Avoid vision variants and models without structured tool calls.

Can I use OpenAI, Gemini or another provider?

Not yet. Ollama already fronts many open models locally; additional hosted providers are on the roadmap.

Do I have to pick a resource before asking?

No. There is always a cluster-wide snapshot, so “is anything broken?” works immediately. Opening a resource grounds follow-ups on it.