Add OpenAI-compatible AI provider (pi-native)

[georgestephanis]

Adds a first-class openai-compatible AI provider to the CLI so the Studio agent can run against a self- or independently-hosted open-weight model (vLLM, Apfel, LM Studio, Ollama, llama.cpp, …). Configure it with /openai-config; /model lists the endpoint's live models. This is the pi-native successor to #4206 — see the note below.

Proof of Concept. Opened as a draft with the Proof of Concept label. It's directional: it works end to end, but the model-type widening and the pi-runtime hook cross architectural boundaries the team will likely want to review before landing.

Related issues

• Related to Feature Request: Support custom api endpoints #4198
• Supersedes Add OpenAI-compatible CLI AI provider #4206 (closed). That PR predated trunk's move to the pi runtime and used a local Anthropic⇄OpenAI translation gateway plus custom compaction. pi speaks OpenAI-compatible natively and does its own context compaction, so both are now redundant — this PR drops ~500 lines of shim in favor of a thin provider.

How AI was used in this PR

Built with Claude Opus 4.8 operating in VS Code. The author reviewed all changes. It was tested live against two local backends: Apfel (Apple's on-device FoundationModel, ~4k context) and a networked vLLM install running qwen3.6-27b (64k context). The 64k model completes turns cleanly; the 4k model is deliberately included to document the minimum-context limit (see Testing).

For anyone unfamiliar with Apfel: on a Mac it's a one-line brew install that exposes the OS's built-in Foundation Model over an OpenAI-compatible endpoint — no downloads or keys.

Proposed Changes

• Enables self- and independently-hosted open-weight LLMs (as discussed in the recent Code for the People documentary) via any OpenAI-compatible server. Because the agent now runs on the pi runtime — which speaks OpenAI chat/completions natively and manages its own compaction — no wire-translation shim or Studio-side compaction is needed; the provider simply points pi at the endpoint.
• The endpoint's models are discovered from GET /v1/models, so /model offers the server's real models (not Studio's built-in catalog), and each model's advertised context window (context_window / max_model_len) is read and handed to pi, which sizes its native compaction to the local model's real limit — important given the smaller context windows of consumer-GPU-hosted models.
• Trade-off / limitation now documented: compaction trims conversation history but not the fixed system prompt (~4.5k tokens before tools), so models with a context window at/under ~5k (e.g. Apfel at 4096) can't host the agent. Pick a model with ≥16k, more for real work. Details in docs/design-docs/local-ai-providers.md.
• Config is stored as an array (openAiCompatibleEndpoints) so multiple endpoints can be added later without a breaking migration; this PR wires up a single active endpoint.

Testing Instructions

Prerequisites: a local OpenAI-compatible server. Any works (vLLM, LM Studio, Ollama, llama.cpp --server, Apfel). Note the context-window caveat above — use a model with ≥16k context for a working chat; Apfel (4k) is only useful for observing the too-small-window error.

1. Automated (fastest signal):

nvm use            # .nvmrc: Node 24.x / npm 11
npm test -- apps/cli/ai packages/common/ai
npm run typecheck

Covers model-family/label fallbacks for unknown ids (packages/common/ai/tests/models.test.ts), endpoint discovery field-name handling and graceful failure (apps/cli/ai/tests/openai-compatible.test.ts), and the provider list (apps/cli/ai/tests/auth.test.ts).

2. In-app, against a ≥16k-context endpoint:

npm run cli:build && node apps/cli/dist/cli/main.mjs

• Run /openai-config, enter the base URL and optional key. It lists the endpoint's models — pick one.
• Chat normally: replies stream back through the local model. Confirm /model re-lists the endpoint's models and lets you switch.
• Confirmed live: a networked vLLM (qwen3.6-27b, 64k) completes turns cleanly (api=openai-completions, provider=openai, correct model id).

3. Minimum-context behavior (Apfel, 4k):

• Point /openai-config at Apfel (http://localhost:11435/v1, model apple-foundationmodel).
• The first turn errors with context_length_exceeded — expected: Studio's system prompt alone exceeds 4096 tokens. This documents the lower bound, it isn't a regression.

4. Built-in providers unaffected: wpcom and anthropic-api-key still work exactly as before (the model-id type was widened, not changed for them).

Pre-merge Checklist

• Have you checked for TypeScript, React or other console errors?

🤖 Generated with Claude Code