-
Notifications
You must be signed in to change notification settings - Fork 86
Studio Code: Add data liberation slash command #3940
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
nightnei
merged 28 commits into
trunk
from
stu-1510-data-liberation-add-slash-commands-skills-to-import-content
Jul 6, 2026
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
d82de6d
Revert temporary blocks-engine vendor to registry ^0.2.2
nightnei 35a1f0e
unskip test
nightnei 332c36b
In progress
nightnei 23442a1
In progress
nightnei 7b1e858
Add data-liberating to the bundle (#3949)
nightnei a827a32
Revert
nightnei 1426e50
Adjust to the built-in data-liberation-agent
nightnei c88a7e0
Fix build process
nightnei 1044de0
In progress
nightnei 5f2d839
Switch to use agent dirrectly
nightnei a4876e2
In progress
nightnei 04b4885
In progress
nightnei c764dea
In progress
nightnei d0e8636
In progress
nightnei c2e51a6
In progress
nightnei 9e91860
In progress
nightnei 2cda69c
Add skills
nightnei 4af569a
Ensure chromium
nightnei 19f06ac
Clean up
nightnei 8d74df1
Clean up
nightnei 85096aa
Clean up
nightnei e5feebe
Clean up
nightnei 4c2a9ec
Clean up
nightnei c8b21c2
Fix runtime
nightnei a4ad4a9
Fix args
nightnei 088f461
Merge branch 'trunk' into stu-1510-data-liberation-add-slash-commands…
nightnei fa6d85f
Clean up
nightnei a04daa9
Resolve conflicts
nightnei File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,31 @@ | ||
| --- | ||
| name: liberate | ||
| description: Import and rebuild a website from a closed platform (Wix, Squarespace, Webflow, Shopify, GoDaddy, Hostinger, HubSpot, Weebly) into a Studio WordPress site. Extracts pages/posts/products + media, then reconstructs the design as editable blocks + WooCommerce OR as a high-fidelity replica theme. Invoke when the user wants to migrate, import, liberate, or rebuild a site from one of these platforms. | ||
| --- | ||
|
|
||
| # Liberate a website into Studio | ||
|
|
||
| This skill is only a **redirect**. The real, always-up-to-date pipeline lives in the Data Liberation engine's own skill. Do NOT re-plan or summarize the steps here — defer to the engine skill so its updates take effect automatically. Your job is just to (1) stand the engine up and (2) follow its skill, translating its tool calls into Studio's bridge. | ||
|
|
||
| ## Step 1 — Locate the engine | ||
|
|
||
| Call the `data_liberation` tool with `{ tool: "setup" }`. It returns `{ engineDir, liberateSkill, skillsDir }` — the paths to the engine's skill files. The engine ships prebuilt with Studio, so this is instant — just proceed. | ||
|
|
||
| ## Step 2 — Load the engine's tool catalog | ||
|
|
||
| Call `data_liberation` with `{ tool: "list" }` to load the engine's full tool catalog (every tool name + its argument JSON-Schema). Treat that catalog as the **source of truth** for tool names and arguments — the engine skill names the tools to call, but the catalog tells you their exact arguments. If you are ever unsure of an argument, re-read the catalog rather than guessing. | ||
|
|
||
| ## Step 3 — Follow the engine's real liberate skill | ||
|
|
||
| `Read` the `liberateSkill` path from Step 1 (the engine's own `SKILL.md`) and execute its workflow verbatim, applying these Studio bindings wherever it instructs you: | ||
|
|
||
| - **Engine tool call** — where it says "call the `liberate_X` tool with `{…}`", instead call Studio's `data_liberation` tool with `{ tool: "liberate_X", args: {…} }` and read the returned text/JSON as that tool's result. (`args` is a JSON **object**, never a stringified JSON.) | ||
| - **Sub-skill dispatch** — where it dispatches another skill (e.g. `replicate-with-blocks` / `replicate-theme`), `Read` it under `engineDir/skills/<name>/SKILL.md` and continue inline with these same bindings. | ||
| - **Install & import** — follow the engine's install step (it documents both `liberate_preview` and `liberate_install_theme` + `liberate_import`); `/liberate` is the engine's *standalone* context, so prefer **`liberate_preview`** (one call: creates the Studio site, imports the WXR + media, runs WP-default cleanup, activates the theme). Two Studio specifics the engine can't know: (1) `liberate_import` needs credentials — create an application password with `studio wp user application-password create` and pass it; (2) never point `studio wp import` / `wp eval-file` at the host `_liberations` path — Studio's PHP sandbox only reads inside the site (`/wordpress/...`), so let `liberate_preview` / `liberate_import` do the import rather than hand-rolling it. | ||
| - **Don't skip the final QA** — it's nested a level deeper and easy to stop short of after a long run. The chosen sub-skill ends with a mandatory parity step: **blocks** → `Read` `engineDir/skills/design-qa/SKILL.md` and complete its loop; **theme** → `replicate-theme`'s `liberate_compare` step. Run it fully; never substitute eyeballed screenshots, and honor the engine's honesty rule — no "looks good / matches" without measured parity. | ||
| - **`liberate_blockify_wxr`** — if it reports "no platform recorded", re-call with the platform from `liberate_detect` (e.g. `{ platform: "wix" }`); a no-op on platforms without a block recipe is expected, not a failure. | ||
| - **Timeouts (`MCP error -32001`) mean "still working," not "failed."** Any engine op can return `-32001` on the client side while it keeps running **inside the engine** — never treat it as a failure and never blindly re-invoke (a re-invoke often errors, e.g. `extraction already in progress`). For extraction/reconstruct ops (`liberate_extract`, `liberate_screenshot`, reconstruct), poll `liberate_status` with `{ outputDir }` until `running` is `false`, then continue from the output artifacts. For any other op that times out, wait and re-check its output/state before concluding it failed. | ||
|
|
||
| ## Reporting | ||
|
|
||
| When the engine finishes, summarize its run report: the Studio site built into, the reconstruct path taken, counts (pages/posts/products/media), the parity/verdict, and anything flagged for manual attention. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| import { existsSync } from 'fs'; | ||
| import path from 'path'; | ||
| import { Client } from '@modelcontextprotocol/sdk/client/index.js'; | ||
| import { StdioClientTransport } from '@modelcontextprotocol/sdk/client/stdio.js'; | ||
| import { Type } from 'typebox'; | ||
| import { ensurePlaywrightChromiumInstalled } from '../browser-utils'; | ||
| import { defineTool } from './define-tool'; | ||
| import { textResult } from './utils'; | ||
|
|
||
| const engineDir = path.join( import.meta.dirname, 'data-liberation-agent' ); | ||
|
|
||
| let chromiumPromise: Promise< void > | null = null; | ||
|
|
||
| function ensureEngineChromium(): Promise< void > { | ||
| if ( ! chromiumPromise ) { | ||
| chromiumPromise = ( async () => { | ||
| const { chromium } = await import( 'playwright' ); | ||
| const problem = await ensurePlaywrightChromiumInstalled( chromium ); | ||
| if ( problem ) { | ||
| chromiumPromise = null; // allow a retry on the next tool call | ||
| console.error( | ||
| `[data_liberation] ${ problem } Browser-dependent steps (extract/screenshot/reconstruct) may fail.` | ||
| ); | ||
| } | ||
| } )(); | ||
| } | ||
| return chromiumPromise; | ||
| } | ||
|
|
||
| let clientPromise: Promise< Client > | null = null; | ||
|
|
||
| function getClient(): Promise< Client > { | ||
| if ( ! clientPromise ) { | ||
| clientPromise = connectClient().catch( ( error ) => { | ||
| clientPromise = null; | ||
| throw error; | ||
| } ); | ||
| } | ||
| return clientPromise; | ||
| } | ||
|
|
||
| async function connectClient(): Promise< Client > { | ||
| if ( ! existsSync( path.join( engineDir, 'dist', 'mcp-server.js' ) ) ) { | ||
| throw new Error( | ||
| 'Data Liberation engine is not compiled. Run `npm run cli:build` — it builds the ' + | ||
| '`data-liberation` workspace and bundles it into `dist/cli`.' | ||
| ); | ||
| } | ||
|
|
||
| const transport = new StdioClientTransport( { | ||
| command: process.execPath, | ||
| args: [ path.join( engineDir, 'dist', 'mcp-server.js' ) ], | ||
| cwd: engineDir, | ||
| stderr: 'pipe', | ||
| } ); | ||
| const client = new Client( { name: 'studio-code', version: '1.0.0' }, { capabilities: {} } ); | ||
| await client.connect( transport ); | ||
| return client; | ||
| } | ||
|
|
||
| interface DataLiberationResultContent { | ||
| type: string; | ||
| text?: string; | ||
| [ key: string ]: unknown; | ||
| } | ||
|
|
||
| // The model sometimes sends `args` as a JSON-encoded STRING instead of an object; | ||
| // forwarding that as MCP `arguments` fails the SDK schema ("expected record"). | ||
| // Coerce nullish → {}, a JSON string → its parsed object, and reject anything else. | ||
| function normalizeArgs( raw: unknown ): Record< string, unknown > { | ||
| let value = raw; | ||
| if ( typeof value === 'string' ) { | ||
| const trimmed = value.trim(); | ||
| if ( ! trimmed ) { | ||
| return {}; | ||
| } | ||
| try { | ||
| value = JSON.parse( trimmed ); | ||
| } catch { | ||
| throw new Error( | ||
| `data_liberation: \`args\` must be a JSON object, not a stringified JSON. Received: ${ trimmed.slice( | ||
| 0, | ||
| 200 | ||
| ) }` | ||
| ); | ||
| } | ||
| } | ||
| if ( value === undefined || value === null ) { | ||
| return {}; | ||
| } | ||
| if ( typeof value !== 'object' || Array.isArray( value ) ) { | ||
| throw new Error( 'data_liberation: `args` must be a JSON object.' ); | ||
| } | ||
| return value as Record< string, unknown >; | ||
| } | ||
|
|
||
| export const dataLiberationTool = defineTool( | ||
| 'data_liberation', | ||
| 'Bridge to the Data Liberation engine, which extracts content from closed web platforms ' + | ||
| '(GoDaddy, Hostinger, HubSpot, Shopify, Squarespace, Webflow, Weebly, Wix) ' + | ||
| 'and reconstructs it into a WordPress site. This tool forwards a single call to the engine; ' + | ||
| "the `/liberate` skill orchestrates the full sequence. Pass `tool: 'setup'` to get the paths " + | ||
| 'to its skill files (the engine ships prebuilt with Studio).', | ||
| { | ||
| tool: Type.Optional( | ||
| Type.String( { | ||
| description: | ||
| "Engine MCP tool name, e.g. 'liberate_detect', 'liberate_discover', 'liberate_extract', " + | ||
| "'liberate_reconstruct_pages', 'liberate_install_theme'. Pass 'setup' to get the engine's " + | ||
| "skill-file paths, or 'list' to fetch the full catalog of engine tools with " + | ||
| 'their argument schemas — consult it whenever you are unsure of a tool name or its arguments.', | ||
| } ) | ||
| ), | ||
| args: Type.Optional( | ||
| Type.Unknown( { | ||
| description: | ||
| 'Arguments forwarded to the engine tool. MUST be a JSON OBJECT, not a stringified ' + | ||
| 'JSON — e.g. { "url": "https://example.com" }, NOT "{\\"url\\":\\"…\\"}". ' + | ||
| "Site-targeting tools expect a Studio target, e.g. { kind: 'studio', sitePath: '/Users/you/Studio/my-site' }.", | ||
| } ) | ||
| ), | ||
| }, | ||
| async ( args ) => { | ||
| if ( ! args.tool ) { | ||
| throw new Error( | ||
| 'data_liberation: a `tool` is required. Pass "setup" to get the engine skill-file paths, ' + | ||
| '"list" for the tool catalog, or an engine tool name (e.g. "liberate_detect").' | ||
| ); | ||
| } | ||
|
|
||
| if ( args.tool === 'setup' ) { | ||
| return textResult( | ||
| JSON.stringify( { | ||
| engineDir, | ||
| skillsDir: path.join( engineDir, 'skills' ), | ||
| liberateSkill: path.join( engineDir, 'skills', 'liberate', 'SKILL.md' ), | ||
| } ) | ||
| ); | ||
| } | ||
|
|
||
| const client = await getClient(); | ||
|
|
||
| if ( args.tool === 'list' ) { | ||
| const listed = await client.listTools(); | ||
| return textResult( JSON.stringify( listed.tools, null, 2 ) ); | ||
| } | ||
|
|
||
| await ensureEngineChromium(); | ||
|
|
||
| const result = await client.callTool( { | ||
| name: args.tool, | ||
| arguments: normalizeArgs( args.args ), | ||
| } ); | ||
|
|
||
| const rawContent = Array.isArray( result.content ) | ||
| ? ( result.content as DataLiberationResultContent[] ) | ||
| : []; | ||
| const text = rawContent | ||
| .map( ( part ) => ( part.type === 'text' ? part.text ?? '' : `[${ part.type }]` ) ) | ||
| .join( '\n' ); | ||
|
|
||
| if ( result.isError ) { | ||
| throw new Error( `Engine tool ${ args.tool } failed: ${ text }` ); | ||
| } | ||
|
|
||
| return textResult( text ); | ||
| } | ||
| ); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a very weird way to use the data_liberation. If the data liberation packages bundles skills, can't we actually "import" them directly where we define the skills of Studio Code instead of having a "wrapper" skill?