Refine plugin guidance in AI system prompt - #2797
Conversation
|
@nerrad, let me know when this PR is ready for a review, or if it’s no longer necessary. |
|
hey, sorry, I haven't had a chance to circle back to this yet. I was getting some early signals this prompt helped improve the experience of integrating plugins (like WooCommerce) but it needs more testing and validation. Something I can't do at the moment. Much of the prompt was hand crafted (PR description, not so much 🙂). |
|
Perfect, thank you Darren. We'll add evals to measure how |
|
You mean beyond what is already in this PR? |
|
I'm interested in knowing the other side. Some examples of user requests/queries that will be improved like:
|
|
Oh gotcha, ya so the angle I was looking at with this PR is just making sure that if the agent decides to install a particular plugin to satisfy a user request (either explicit or implicit), it has some basic direction on what to use from that plugin (if it exists) when creating the site. This is particularly important for plugins like WooCommerce which is supporting both classic (classic themes/templates/shortcodes/template functions) and new functionality (block themes, block templates, blocks etc). It's also useful for making sure the agent looks for any existing onboarding flows for a plugin to infer questions to ask the user for additional configuration as well. I thought that'd be a good step one. |
|
Thanks for adding more context. That's really helpful. I agree that make a difference between classic and block theme is key. 🤔 |
Related issues
How AI was used in this PR
Proposed Changes
Testing Instructions
apps/cli/ai/system-prompt.ts.Pre-merge Checklist