Skip to content

feat: improve agent-customization skill — Medium → High compliance#3866

Merged
digitarald merged 4 commits intomicrosoft:mainfrom
spboyer:feat/agent-customization-skill-improvements
Feb 24, 2026
Merged

feat: improve agent-customization skill — Medium → High compliance#3866
digitarald merged 4 commits intomicrosoft:mainfrom
spboyer:feat/agent-customization-skill-improvements

Conversation

@spboyer
Copy link
Copy Markdown
Member

@spboyer spboyer commented Feb 19, 2026

Summary

Applies skill-compliance recommendations to the agent-customization built-in skill. This skill teaches users how to create and update good agent customization files — it should be the best example of its own advice.

Before / After

Check Before After
Skill type prefix (**WORKFLOW SKILL**) ❎ Missing ✅ Added
USE FOR: triggers ✅ Present (~7 cases) ✅ Kept
DO NOT USE FOR: anti-triggers ❎ Missing ✅ 4 anti-triggers
INVOKES: clause ❎ Missing ✅ file system, ask-questions, subagents
FOR SINGLE OPERATIONS: ❎ Missing ✅ Quick YAML fixes bypass
Description length ~475 chars ~942 chars (under 1024 limit)
Reference modules 6 files 3 files
Common Pitfalls section ❎ None ✅ 3 compressed gotchas

What Changed

Frontmatter Description (SKILL.md)

  • Added **WORKFLOW SKILL** prefix for routing clarity
  • Added DO NOT USE FOR: — general coding, runtime debugging, MCP config, extension dev
  • Added INVOKES: — file system tools, ask-questions, subagents
  • Added FOR SINGLE OPERATIONS: — quick YAML fixes don't need the skill
  • Kept all existing USE FOR: triggers intact

Reference Consolidation (6 → 3 files)

Fewer reference modules means less context loaded per invocation. Six separate files meant the skill pulled in more tokens than necessary — consolidating by interaction pattern keeps the same content with better organization.

New File Merged From Content
file-customizations.md instructions.md + workspace-instructions.md + prompts.md File-level and workspace-level customization primitives
agent-extensibility.md agents.md + skills.md Agent and skill creation patterns
hooks.md (unchanged) Lifecycle hooks — distinct enough to stay standalone

All templates, anti-patterns, location tables, and core principles preserved. Quick Reference table updated to 3 rows.

Common Pitfalls (new section)

Three highest-impact gotchas in compressed format:

  1. Description as discovery surface — triggers must be in the description or agents won't find it
  2. YAML frontmatter silent failures — unescaped colons, tabs, name mismatch
  3. applyTo: "**" burns context — loads instruction on every request even when irrelevant

Why These Changes

The agent-customization skill already had strong content — good USE FOR triggers and procedural guidance. These changes are the minimal set to improve routing clarity:

  • Anti-triggers tell the routing system when NOT to load this skill, preventing false-positive invocations
  • INVOKES + FOR SINGLE OPERATIONS complete the routing picture so other skills/agents know what this skill does and when to bypass it
  • Reference consolidation reduces context cost — 3 focused modules organized by interaction pattern instead of 6 scattered by topic

No content was removed — only reorganized and augmented. The skill still does exactly what it did before, but now routes better and loads faster.

@spboyer spboyer force-pushed the feat/agent-customization-skill-improvements branch from c75947f to ff1feeb Compare February 19, 2026 21:00
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the agent-customization built-in skill to improve its compliance score from Medium to High by applying Sensei skill-compliance recommendations. The changes focus on better skill routing through enhanced frontmatter metadata and consolidating reference documentation from 6 files to 3 for improved performance based on SkillsBench research.

Changes:

  • Enhanced SKILL.md frontmatter description with **WORKFLOW SKILL** prefix, DO NOT USE FOR anti-triggers, INVOKES clause, and FOR SINGLE OPERATIONS bypass guidance
  • Consolidated 6 reference files into 3: merged workspace-instructions.md, instructions.md, and prompts.md into file-customizations.md; merged agents.md and skills.md into agent-extensibility.md; kept hooks.md standalone
  • Added Common Pitfalls section with three high-impact gotchas about descriptions, YAML frontmatter, and applyTo patterns

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
assets/prompts/skills/agent-customization/SKILL.md Updated description with workflow skill metadata and routing information; updated Quick Reference table to 3 files; added Common Pitfalls section
assets/prompts/skills/agent-customization/references/file-customizations.md New consolidated file merging workspace instructions, file instructions, and prompts documentation with shared principles and anti-patterns
assets/prompts/skills/agent-customization/references/agent-extensibility.md New consolidated file merging custom agents and skills documentation with combined core principles
assets/prompts/skills/agent-customization/references/workspace-instructions.md Deleted (content merged into file-customizations.md)
assets/prompts/skills/agent-customization/references/instructions.md Deleted (content merged into file-customizations.md)
assets/prompts/skills/agent-customization/references/prompts.md Deleted (content merged into file-customizations.md)
assets/prompts/skills/agent-customization/references/agents.md Deleted (content merged into agent-extensibility.md)
assets/prompts/skills/agent-customization/references/skills.md Deleted (content merged into agent-extensibility.md)
digitarald and others added 2 commits February 19, 2026 14:05
Keep original 6 reference files unchanged.
Retain improved SKILL.md frontmatter description and Common Pitfalls section.
@spboyer
Copy link
Copy Markdown
Member Author

spboyer commented Feb 19, 2026

Recommendation: Reference File Consolidation (Future)

Based on the audit, the current 6 reference files could benefit from consolidation into 3 files organized by interaction pattern:

Current (6 files) Proposed (3 files) Rationale
workspace-instructions.md + instructions.md + prompts.md file-customizations.md All file-based customization primitives share similar frontmatter patterns and applyTo scoping
agents.md + skills.md agent-extensibility.md Both define autonomous entities with tool restrictions and subagent patterns
hooks.md hooks.md (unchanged) Already well-scoped as a standalone reference

Why this helps:

  • Fewer modules means less context loaded per interaction — agents resolve references faster with 3 targeted files vs 6
  • Reduces cross-reference overhead (instructions ↔ workspace-instructions share patterns that are currently split)
  • The grouping follows how users actually think about customization: files I edit, agents I define, hooks I configure

This was implemented in an earlier revision of this PR but reverted to keep scope focused on the frontmatter and pitfalls improvements. Flagging here as a follow-up worth considering.

@digitarald digitarald enabled auto-merge February 20, 2026 00:22
@digitarald digitarald requested a review from pwang347 February 20, 2026 00:22
@vs-code-engineering vs-code-engineering bot added this to the February 2026 milestone Feb 20, 2026
@digitarald digitarald added this pull request to the merge queue Feb 24, 2026
Merged via the queue into microsoft:main with commit d543055 Feb 24, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

6 participants