Skip to content

[codex] fix Search Console indexing and redirects#131

Closed
basit3407 wants to merge 7 commits intomainfrom
codex/search-console-indexing-cleanup
Closed

[codex] fix Search Console indexing and redirects#131
basit3407 wants to merge 7 commits intomainfrom
codex/search-console-indexing-cleanup

Conversation

@basit3407
Copy link
Copy Markdown
Collaborator

What changed

  • make the versioned API URLs the canonical/indexable URLs in Docusaurus navigation and generated output
  • add Cloudflare redirects for legacy docs URLs reported in Search Console, including old SDK routes, old API intro aliases, and renamed user/content API slugs
  • add a post-generation cleanup step that removes generated auth-* alias docs to avoid duplicate crawl targets
  • add a post-build SEO normalization step that rewrites canonicals and sitemap entries onto the versioned trailing-slash URLs
  • fix stale internal docs links that were breaking production builds

Why

Search Console was reporting redirect, duplicate, and 404 indexing issues caused by mixed canonical forms, duplicate versioned/unversioned API docs, and legacy URLs that no longer resolved.

Impact

  • old Search Console 404s should resolve through redirects instead of failing
  • Google gets a single canonical versioned URL per API doc page
  • duplicate crawl paths are reduced by pruning generated alias docs and sitemap entries
  • the build now enforces the SEO normalization step automatically

Validation

  • yarn build passed on this exact change set before it was cherry-picked onto this clean branch
  • verified the built sitemap excludes /search/, /request-scopes/, and unversioned API-family URLs
  • verified generated auth-*.api.mdx alias docs are removed after generation
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages Bot commented Apr 20, 2026

Deploying qf-api-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: 4143594
Status:🚫  Build failed.

View logs

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b74a63db7d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread static/_redirects Outdated
/docs/search_apis_versioned/1.0.0/:slug /docs/search_apis_versioned/1.0.0/:slug/ 301
/docs/user_related_apis_prelive/:slug /docs/user_related_apis_prelive/:slug/ 301
/docs/user_related_apis_versioned/:slug /docs/user_related_apis_versioned/1.0.0/:slug/ 301
/docs/user_related_apis_versioned/:slug/ /docs/user_related_apis_versioned/1.0.0/:slug/ 301
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Exclude scopes from generic user API version redirect

This catch-all rule also matches /docs/user_related_apis_versioned/scopes/, which then redirects to /docs/user_related_apis_versioned/1.0.0/scopes/; that URL is explicitly redirected back to /docs/user_related_apis_versioned/scopes/ earlier in the same file, creating a 301 loop. In production this makes the scopes page unreachable (including links like /docs/user_related_apis_versioned/scopes#search).

Useful? React with 👍 / 👎.

@basit3407
Copy link
Copy Markdown
Collaborator Author

Not needed after #136

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant