[codex] restore search console indexing cleanup

[basit3407]

What changed

• restore the intended Search Console cleanup from [codex] fix Search Console indexing and redirects #131 on top of current main
• make versioned trailing-slash API URLs the canonical navigation targets in Docusaurus config
• add the post-generation cleanup that prunes generated auth-*.api.mdx alias docs
• add the post-build SEO normalization step that rewrites canonicals and sitemap entries onto the preferred versioned URLs
• fix generated sidebar cleanup so removed docs do not leave stale sidebar references behind
• carry forward the stale internal docs link fixes that were part of the original scope

Why

#131 had the right functional scope but never landed because the branch accumulated Cloudflare debugging churn and was later replaced by #136, which only carried a small subset of the original work. This PR recreates the actual intended #131 end state cleanly from current main.

Not included

• no .node-version pin
• no extra OAuth discovery redirect work

Impact

• Search Console should see one preferred canonical form for the versioned API docs
• legacy generated alias docs are pruned from crawl targets
• sitemap output is normalized around the preferred versioned URLs and excludes noisy utility paths
• sidebar generation no longer keeps references to docs removed by the cleanup step

Validation

• rebuilt the recreated branch locally on April 22, 2026 with CI=true
• verified the full Docusaurus generation/build plus postbuild-seo.js completed successfully

Replaces

• closes [codex] fix Search Console indexing and redirects #136
• supersedes the closed [codex] fix Search Console indexing and redirects #131

[cloudflare-workers-and-pages]

Deploying qf-api-docs with    Cloudflare Pages

Latest commit:	94371a0
Status:	✅  Deploy successful!
Preview URL:	https://010967ac.qf-api-docs.pages.dev
Branch Preview URL:	https://codex-search-console-indexin.qf-api-docs.pages.dev

View logs

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Restores and extends the Search Console indexing cleanup workflow by pruning generated alias docs, normalizing API sidebar output, and post-processing the built site so canonicals/sitemap consistently prefer versioned API URLs.

Changes:

• Adds generated sidebar cleanup that removes references to pruned/missing API docs, plus a unit test for the behavior.
• Adds post-generation pruning for generated auth-*.api.mdx alias docs and a post-build SEO normalization step (canonicals + sitemap rewriting).
• Updates Docusaurus config and internal docs links to prefer versioned, trailing-slash canonical API URLs.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/api-sidebar-cleanup.test.cjs	Adds unit coverage for filtering/cleanup of generated sidebar items.
static/_redirects	Normalizes commented redirect examples (formatting).
src/pages/legal/developer-terms.mdx	Fixes legal doc link target to an absolute /legal/.../ route.
scripts/set-api-displayed-sidebars.js	Adds missing-doc sidebar filtering and exposes functions for testing.
scripts/prune-generated-api-aliases.js	Deletes generated auth-*.api.mdx alias docs after generation.
scripts/postbuild-seo.js	Rewrites built HTML canonicals/og:url and normalizes sitemap URLs/entries.
package.json	Wires new cleanup + SEO steps into gen-all and build.
docusaurus.config.js	Enables trailingSlash and updates navbar links + sitemap ignore patterns.
docs/user-related-apis/reading-sessions-vs-activity-days.mdx	Updates related links to preferred versioned API doc URLs.
docs/user-related-apis/1.0.0/reading-sessions-vs-activity-days.mdx	Updates versioned doc’s related links to preferred versioned API doc URLs.
docs/tutorials/sync/handling-conflicts.mdx	Fixes API reference + tutorial next-step links to new canonical targets.
docs/tutorials/sync/getting-started.mdx	Fixes API reference + next-step links to new canonical targets.
docs/tutorials/oidc/example-integration.mdx	Updates user API links to preferred versioned endpoints.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.