Add OpenAI-compatible CLI AI provider

[georgestephanis]

Introduces a new openai-compatible provider in the CLI, including persisted config fields (base URL, API key, model) and provider selection support. It adds a local Anthropic-to-OpenAI compatibility gateway that maps /v1/messages requests to /chat/completions responses, with support for tool calls and streaming SSE translation. Also adds a /openai-config slash command for updating endpoint settings and updates provider availability tests.

Related issues

• Fixes Feature Request: Support custom api endpoints #4198

How AI was used in this PR

The PR was built with the help of Claude Sonnet 5 operating in VSCode. We also set up and ran local tests against Apfel running locally, as well as Qwen3.6-27b running on a networked vLLM install with a 64k context window.

For anyone unfamiliar with Apfel -- it generally has a 4k context window, and if you're testing on a Mac is trivial to set up with the Foundation Model that MacOS generally already has in place.

macOS Tahoe ships with a 3B parameter LLM. apfel gives you CLI access with one brew install. No model downloads, no API keys, no configuration needed, just works.

Proposed Changes

• To enable self- and independently-hosted open-weight LLMs (as discussed in the recent Code for the People documentary) this enables compatibility with any server running OpenAI-Compatible endpoints (a translation from the default Anthropic formats -- until such time as an acceptable standard such as the [circa 2024] Open Message Format or the like gains adequate traction for better cross-compatibility)
• Due to the typically smaller context windows of various locally hosted models (often constrained by consumer-grade GPU VRAM limitations) this PR is also adding compaction tooling for context windows if an exchange gets overly long where it may cause overflows. It detects the available context window based on the model listing on the host.

Testing Instructions

Prerequisites: a local OpenAI-compatible server. Easiest two, both already on this machine:

• Apfel (http://localhost:11435/v1, model apple-foundationmodel) — 4096-token window, so compaction triggers almost immediately. Best for seeing the behavior.
• My Network vLLM install (http://hostname:8000/v1, model qwen3.6-27b) — 65536-token window. Best for confirming the fast-path (no needless compaction) and real summarization. -- if anyone wants to test against this machine, DM me on .org slack and I'll get you the port and an api key.

Any OpenAI-compatible backend works too (LM Studio, Ollama, llama.cpp --server).

1. Automated tests (fastest signal):

nvm use 24.13.1
npm test -- apps/cli/ai/tests/openai-compat-gateway.test.ts
npm run typecheck

Expect 10 passing tests covering discovery (context_window / max_model_len / fallback), budget-triggered compaction, tool_use↔tool_result pairing safety, and rolling-summary reuse/divergence.

2. In-app black-box test (the meaningful one):

npm run cli:build && node apps/cli/dist/cli/main.mjs

• In the chat, run /openai-config and point it at Apfel (base URL http://localhost:11435/v1, no key, model apple-foundationmodel).
• Have a long back-and-forth — paste large blocks of text, ~15-20 turns, enough to blow past 4096 tokens.
• Pass: the assistant keeps responding coherently and does not error with context_length_exceeded. Before this change, that same long conversation against a 4k backend would fail once history exceeded the window. The recent turns stay accurate; older context is represented by a summary the model folded into its system prompt.

3. Discovery + fast-path (no over-compaction):

• /openai-config → point at vLLM.
• Normal short conversations should behave exactly as before — compaction only engages once the transcript approaches ~64k tokens, so day-to-day chats are untouched.

4. Graceful degradation:

• Point at a backend whose /v1/models omits any context field, or an unreachable URL → gateway falls back to an 8192-token default and still works (verified: discovery never throws).

I also have a standalone tsx harness (in scratchpad) that prints the discovered window, before/after message counts, and the injected summary. Confirmed live this session: Apfel discovered 4096 and compacted 41→10 messages; vLLM discovered 65536, correctly skipped an 11k-token request, and correctly summarized a 251-message / ~71k-token conversation down to ~64k with a real model-generated summary. I can promote that into a committed script under apps/cli/ai/ if you'd like a repeatable manual harness.

Pre-merge Checklist

• Have you checked for TypeScript, React or other console errors?

Verification: 10 new unit tests + all 98 existing ai-module tests pass, lint and typecheck are clean, and manual smoke tests against both real backends confirm the whole pipeline — discovery, budget math, splitting, live summarization, and normal chat flow — works end to end.

[georgestephanis]

ugh, seems my fork I worked on for the pr was slightly out of date, fixing via a merge ...

[georgestephanis]

Oof, much bigger out of date as it's now using pi as the harness. Closing this, I'll rebuild on top of current trunk.