[codex] add discovery headers

[basit3407]

What changed

• add homepage Link discovery headers for machine-readable API descriptions and human-facing documentation entry points
• add a scoped /robots.txt header rule in static/_headers so the robots asset gets its intended cache policy without inheriting the default HTML cache header
• keep the change scoped to static/_headers only

Why

The homepage did not advertise useful discovery relations for agents and API consumers. In addition, the existing robots.txt asset on main should be handled through headers only, not re-added in this branch.

Impact

• GET / will return additional Link relations: describedby, service-desc, and service-doc
• GET /robots.txt will keep using the existing file from main, with the intended cache behavior from _headers
• agents and tooling have a clearer machine-readable and human-readable discovery path from the homepage

Root cause

The Cloudflare Pages header configuration did not include discovery Link relations on the homepage, and the robots asset needed a path-specific header override rather than a duplicate file addition.

Validation

• verified the homepage discovery header payload is 1079 characters, below the Cloudflare Pages 2000-character per-header limit
• ran local Cloudflare Pages dev and confirmed GET / returns the planned discovery Link header
• ran local Cloudflare Pages dev and confirmed GET /robots.txt returns 200, Content-Type: text/plain; charset=utf-8, and Cache-Control: public, max-age=86400
• verified all linked production targets currently return 200

[cloudflare-workers-and-pages]

Deploying qf-api-docs with    Cloudflare Pages

Latest commit:	d950fb7
Status:	✅  Deploy successful!
Preview URL:	https://d369a843.qf-api-docs.pages.dev
Branch Preview URL:	https://codex-add-discovery-headers.qf-api-docs.pages.dev

View logs

[Copilot]

Pull request overview

Adds homepage discovery Link headers to advertise machine-readable API descriptions and human-facing documentation, and scopes /robots.txt caching behavior via Cloudflare Pages _headers.

Changes:

• Add homepage (/) Link discovery relations (describedby, service-desc, service-doc).
• Add a path-specific /robots.txt header rule to ensure the intended cache policy is applied.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.