This repository is built for AI infrastructure engineers who want agents to do real work, not recite generic prompts.

It gives an agent the operational memory needed to benchmark SGLang, vLLM, TensorRT-LLM, and TokenSpeed fairly; turn a new SGLang model architecture into an auditable Day-0 support and release plan; explain serving capacity from startup logs; split prefill and decode profiler evidence; inspect traces at layer and kernel level; estimate operator FLOPs and MFU; review SGLang patches against real maintainer discussion patterns; run Humanize-governed SGLang and vLLM SOTA loops; triage SGLang production incidents from a replay; and keep model-family optimization history close to the code that actually changed.

For standalone kernel campaigns and kernel evidence tools, see the sibling project KDA-Pilot.

If this saves you one stale model-support assumption, one misleading profiler trace, or one late-night benchmark loop, a star helps more AI-infra engineers find it.

sglang-sota-humanize-loop always patches SGLang, while the competitor set is caller-controlled. By default the comparison framework set can include vLLM, TensorRT-LLM, and TokenSpeed; a prompt can also narrow it, for example:

Use sglang-sota-humanize-loop for <model>.
comparison_frameworks: [vllm]
Do not consider TensorRT-LLM or TokenSpeed; record them as user-excluded.

sgl-engine-sglang-diffusion is a persistent evidence controller for one serial interactive campaign. It combines a self-contained six-family optimization catalog with commit-bound SGLang Diffusion skills, SGLang runtime source, KDA-Pilot diffusion kernels, KernelWiki, NCU guidance, and FastVideo knowledge. The current root agent chooses hypotheses and edits code; the controller locks the SGLang revision and workload, issues one work order, deterministically verifies real GPU evidence, integrates the latency-positive subset, and emits a clean-room-checked sglang.patch.

Its taxonomy and verification design acknowledge the Sol-Engine code and paper, but the controller does not clone, import, or read Sol-Engine at runtime.

The controller is not another AI agent. It never starts an executor, reviewer, nested Codex process, or Claude process. Its AWAITING_AGENT state means the current conversation should inspect evidence and continue.

The recommended entrypoint is the installable sglang-diffusion-auto-optimize skill. You do not need to write a campaign YAML or install the controller by hand on the GPU machine. The same root-agent conversation remains the only AI owner throughout the campaign.

For Codex, clone this repository and expose the one skill:

git clone https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS.git
cd AI-Infra-Auto-Driven-SKILLS
mkdir -p ~/.codex/skills
ln -s "$PWD/skills/sglang-diffusion-auto-optimize" \
  ~/.codex/skills/sglang-diffusion-auto-optimize

Restart Codex after installing. For Claude Code, install the complete plugin:

/plugin marketplace add BBuf/AI-Infra-Auto-Driven-SKILLS
/plugin install ai-infra-auto-driven-skills@ai-infra-auto-driven-skills
/reload-plugins

The Claude Code skill name is ai-infra-auto-driven-skills:sglang-diffusion-auto-optimize.

Tell the agent only the machine, model, exact native SGLang Diffusion baseline command, and measured end-to-end target:

Use sglang-diffusion-auto-optimize.

Machine: <machine skill or SSH alias>
Model: Wan-AI/Wan2.2-T2V-A14B-Diffusers
Baseline command:
CUDA_VISIBLE_DEVICES=0 python
python/sglang/multimodal_gen/benchmarks/bench_offline_throughput.py
--model-path Wan-AI/Wan2.2-T2V-A14B-Diffusers
--dataset vbench
--dataset-path /persistent/benchmarks/validation-prompts.txt
--num-prompts 5
--output-dir /persistent/benchmarks/wan22-baseline

Target: 2x measured end-to-end speedup.
Own the campaign until the target is verified, the reviewed search space is
exhausted, or a checkable unreachable certificate is produced.

The prompt file must contain exactly five non-empty validation prompts. The root agent discovers the named host instructions, enters its container, locks the latest fetched SGLang main commit, freezes the supplied command, and launches deterministic setup. The watchdog stops at AWAITING_AGENT. The root agent then claims one technique, edits one detached worktree, measures one complete candidate, reviews its own diff and evidence, and submits it for deterministic verification. Rejections return to AWAITING_AGENT; they do not kill the campaign.

The agent reports meaningful state changes in the conversation. The durable campaign also maintains PROGRESS.json and renders a live view like this:

Wan-AI/Wan2.2-T2V-A14B-Diffusers · gpu-host · TARGET 2.00x

performance [██████████████░░░░░░] 1.68x / 2.00x
search      [███████░░░░░░░░░░░░░] 4 / 12 rounds
phase       AWAITING_AGENT · epoch 4 · elapsed 06:14:09
latency     128.4000s baseline -> 76.4286s integrated

technique          state       gate          rounds  isolated e2e
kernel             integrated  passed             2         1.27x
cache              verified    passed             1         1.18x
quantization       unsupported pending            0             -
-------------------------------------------------------------------
integrated stack                                  1.68x

current: awaiting the root agent's next claim or skip

The search counter includes only explicitly submitted full-workload measurements. Technique rows show independently verified results, while integrated stack is measured again after composition. Isolated speedups are never added together. The CLI marks current-conversation token usage unavailable instead of inventing a per-role ledger.

Inspect and continue the same durable campaign with:

sgl-diffusion-engine work --campaign <campaign-dir> --json
sgl-diffusion-engine claim --campaign <campaign-dir> --technique <name>
sgl-diffusion-engine submit --campaign <campaign-dir> \
  --delivery <worktree>/DELIVERY.json
sgl-diffusion-engine progress --campaign <campaign-dir>

The workflow finishes only as TARGET_REACHED, SEARCH_SPACE_EXHAUSTED, or UNREACHABLE_CERTIFIED. Search exhaustion is not presented as a theoretical impossibility. See the engineering README for the evidence contract, low-level controller interface, artifacts, and patch application.

The model optimization layer is now one knowledge base: model-pr-optimization-history. It contains 66 PR-driven history dossiers and a small query helper. These are not per-model runbook skills; they preserve diff-backed model evolution records for SGLang, vLLM, TensorRT-LLM, and TokenSpeed so SOTA loops can read prior source and PR evidence before patching.

Covered families include:

DeepSeek V3/R1/V3.1/V3.2/V4, Qwen3, Qwen3-Coder, Qwen3-Next,
Qwen3.5/Qwen3.6, Qwen VLM/Omni/ASR, GLM 4.5/4.6/4.7/5,
Kimi, MiniMax, Llama 4, Mistral Small 4, Mixtral, Nemotron,
Gemma, Ernie 4.5, Intern-S1, InternVL, Hunyuan, MOSS-VL,
GPT-OSS, Step 3.5, Mimo, and model-specific MoE/quantization paths.

Each model-family history is designed to answer practical questions:

• Which PRs changed this model path?
• Was the PR merged, closed, or still open?
• Which files and symbols moved?
• What optimization or correctness risk should be checked before touching it?
• Which upstream idea should be compared before writing a new kernel or fusion?

Query examples:

cd model-pr-optimization-history
python3 scripts/query.py --list
python3 scripts/query.py --framework sglang --model qwen3-core --paths-only
python3 scripts/query.py --framework vllm "qwen3 fused qk norm"
python3 scripts/query.py --framework tokenspeed --model qwen35 qk rmsnorm

Open PR freshness is tracked separately from merged history cards:

python3 tools/check_open_pr_watch.py --format markdown \
  --output model-pr-optimization-history/open-pr-watch.md

This report uses the GitHub pulls API with an anonymous REST fallback when gh api is rate-limited. If every repo fetch fails, the tool exits non-zero instead of writing a misleading empty report.

The repo is opinionated about evidence because performance work gets noisy fast.

• Benchmark rows should include model, framework, GPU count, workload, request rate or concurrency, SLA status, launch command, benchmark command, and raw artifacts.
• Profiler reports should keep prefill and decode separate, then emit the same three tables: kernel table, overlap-opportunity table, and fuse-opportunity table.
• SOTA claims should be scoped to the exact model, hardware, framework commits, precision, workload, and SLA used in the run.
• SGLang human review should use the full PR episode corpus: inline review threads for line-local findings, PR conversations for design/test/repro negotiation, and review submissions for blocking maintainer summaries.
• Humanize SOTA loops should keep only the fixed fair benchmark outside the patch loop; gap decisions, profiler triage, required layer-pipeline deep dives, kernel evidence, target-framework code changes, and revalidation all stay inside one model-level RLCR loop.
• Kernel-local fixes inside that loop should use ncu-report-skill when Nsight Compute counter evidence is needed, store NCU digests, and still pass the same real-model benchmark/profile gate.
• Incident triage should start from replayable evidence instead of changing code from symptoms alone.
• Model optimization histories should point back to PRs, files, diffs, and risk surfaces rather than vague summary text; they live as one PR-driven knowledge base, not per-model skills.
• Root-level update_prompt.md captures the full refresh and validation workflow for updating this repo again without relying on memory from a previous run.

This repository is not Codex-only. The skills are plain SKILL.md directories and can be installed into Claude Code, Codex, Kimi, or another compatible agent runtime.

The repository ships a .claude-plugin/ manifest so the whole skill set can be installed as a single Claude Code plugin via the built-in marketplace flow:

/plugin marketplace add BBuf/AI-Infra-Auto-Driven-SKILLS
/plugin install ai-infra-auto-driven-skills@ai-infra-auto-driven-skills
/reload-plugins

After reload, the 13 skills appear namespaced as ai-infra-auto-driven-skills:<skill-name> (for example ai-infra-auto-driven-skills:sglang-sota-humanize-loop). Update later with /plugin marketplace update ai-infra-auto-driven-skills.

Prefer this when you only want a subset of the skills, or when developing against a local checkout. Symlink is recommended for local development because updates to this checkout are picked up immediately:

git clone https://github.com/BBuf/AI-Infra-Auto-Driven-SKILLS.git
cd AI-Infra-Auto-Driven-SKILLS

mkdir -p ~/.claude/skills
ln -s "$PWD/skills/llm-serving-auto-benchmark" ~/.claude/skills/llm-serving-auto-benchmark
ln -s "$PWD/skills/llm-serving-capacity-planner" ~/.claude/skills/llm-serving-capacity-planner
ln -s "$PWD/skills/llm-torch-profiler-analysis" ~/.claude/skills/llm-torch-profiler-analysis
ln -s "$PWD/skills/llm-pipeline-analysis" ~/.claude/skills/llm-pipeline-analysis
ln -s "$PWD/skills/model-compute-simulation" ~/.claude/skills/model-compute-simulation
ln -s "$PWD/skills/model-optimization/model-pr-diff-dossier" ~/.claude/skills/model-pr-diff-dossier
ln -s "$PWD/skills/model-optimization/sglang-model-day0-support" ~/.claude/skills/sglang-model-day0-support
ln -s "$PWD/skills/sglang-humanize-review" ~/.claude/skills/sglang-humanize-review
ln -s "$PWD/skills/sglang-diffusion-auto-optimize" ~/.claude/skills/sglang-diffusion-auto-optimize
ln -s "$PWD/skills/sglang-sota-humanize-loop" ~/.claude/skills/sglang-sota-humanize-loop
ln -s "$PWD/skills/vllm-sota-humanize-loop" ~/.claude/skills/vllm-sota-humanize-loop
ln -s "$PWD/skills/sglang-prod-incident-triage" ~/.claude/skills/sglang-prod-incident-triage
ln -s "$PWD/skills/model-architecture-diagram" ~/.claude/skills/model-architecture-diagram
ln -s "$PWD/model-pr-optimization-history" ~/.claude/skills/model-pr-history-knowledge

Restart Claude Code after installing. The skills can then be invoked by name, for example [$llm-serving-auto-benchmark], [$llm-serving-capacity-planner], [$llm-torch-profiler-analysis], [$llm-pipeline-analysis], [$model-compute-simulation], [$model-pr-diff-dossier], [$sglang-model-day0-support], [$sglang-humanize-review], [$sglang-diffusion-auto-optimize], [$sglang-sota-humanize-loop], or [$vllm-sota-humanize-loop].

If you prefer copies instead of symlinks, replace ln -s with cp -R. Copy model-pr-optimization-history only when you want the agent to query the PR-driven model knowledge base locally. It replaces the old per-model runbook skill layout with one shared knowledge root.

For Codex, Kimi, or another compatible runtime, copy or symlink the same directories into that runtime's skill directory:

cp -R skills/llm-serving-auto-benchmark <agent-skill-dir>/llm-serving-auto-benchmark
cp -R skills/llm-serving-capacity-planner <agent-skill-dir>/llm-serving-capacity-planner
cp -R skills/llm-torch-profiler-analysis <agent-skill-dir>/llm-torch-profiler-analysis
cp -R skills/llm-pipeline-analysis <agent-skill-dir>/llm-pipeline-analysis
cp -R skills/model-compute-simulation <agent-skill-dir>/model-compute-simulation
cp -R skills/model-optimization/model-pr-diff-dossier <agent-skill-dir>/model-pr-diff-dossier
cp -R skills/model-optimization/sglang-model-day0-support <agent-skill-dir>/sglang-model-day0-support
cp -R skills/sglang-humanize-review <agent-skill-dir>/sglang-humanize-review
cp -R skills/sglang-diffusion-auto-optimize <agent-skill-dir>/sglang-diffusion-auto-optimize
cp -R skills/sglang-sota-humanize-loop <agent-skill-dir>/sglang-sota-humanize-loop
cp -R skills/vllm-sota-humanize-loop <agent-skill-dir>/vllm-sota-humanize-loop
cp -R skills/sglang-prod-incident-triage <agent-skill-dir>/sglang-prod-incident-triage
cp -R skills/model-architecture-diagram <agent-skill-dir>/model-architecture-diagram
cp -R model-pr-optimization-history <agent-skill-dir>/model-pr-history-knowledge

These skills are exercised with coding agents in full-autonomy mode. For reproducibility, here is exactly how I launch them.

Claude Code — use the current Opus alias with Auto permission mode:

claude --model opus --permission-mode auto

opus tracks Claude Code's current Opus model instead of pinning a model number. Auto mode can approve routine work while retaining the permission system. bypassPermissions (or --dangerously-skip-permissions) should be reserved for an isolated sandbox, ideally without internet access.

Codex — full-access, no approval prompts:

codex --sandbox danger-full-access --ask-for-approval never

The Codex command is intentionally unsandboxed and should likewise be used only inside an isolated environment with a scoped checkout and its own benchmark and correctness gates.

skills/
├── llm-serving-auto-benchmark/      # serving benchmark search and comparison
├── llm-serving-capacity-planner/     # startup memory and request capacity analysis
├── llm-torch-profiler-analysis/     # profiler capture and trace triage
├── llm-pipeline-analysis/           # forward/layer/kernel trace analysis
├── model-compute-simulation/        # operator FLOPs, tensor shapes, and MFU
├── sglang-humanize-review/          # human SGLang PR review corpus and workflow
├── sglang-diffusion-auto-optimize/  # remote autonomous diffusion speedup campaigns
├── sglang-sota-humanize-loop/       # Humanize-governed SGLang SOTA loop
├── vllm-sota-humanize-loop/         # Humanize-governed vLLM SOTA loop
├── sglang-prod-incident-triage/     # replay-first serving incident workflow
├── model-architecture-diagram/      # public architecture diagram resolver
└── model-optimization/
    ├── model-pr-diff-dossier/       # shared PR history quality standard
    └── sglang-model-day0-support/   # model Day-0 PR and release gates

model-pr-optimization-history/
├── SKILL.md                         # knowledge-base usage instructions
├── scripts/query.py                 # local model/keyword query helper
├── sglang/                          # 31 PR-driven SGLang model histories
├── vllm/                            # 31 PR-driven vLLM model histories
├── tensorrt_llm/                    # TensorRT-LLM competitor histories
└── tokenspeed/                      # TokenSpeed competitor histories

prompts/
├── sglang-sota-b200-prompts.md       # B200 SGLang SOTA task prompts
├── sglang-sota-b200-codex-goal-prompts.md
├── sglang-sota-h200-prompts.md       # H200 SGLang SOTA task prompts
└── sglang-sota-h200-codex-goal-prompts.md

• Humanize provides the RLCR workflow that powers the Humanize-governed SGLang and vLLM SOTA loops.
• KDA-Pilot is the sibling home for standalone kernel loops, kernel knowledge, and NCU report workflows.