URL to Markdown

Fetch one public web page and get back its title and clean Markdown: headings, paragraphs, links (made absolute), emphasis, lists, code blocks, blockquotes, images and basic tables. Built for agents that need to read a page after a search. First-party on grokzilla.shop; no third-party API is involved.

Call

POST /api/skills/url-to-markdown

{"url": "https://example.com"}

url (required): an absolute http:// or https:// URL, at most 2,000 characters.

How the page is fetched

• User agent: Mozilla/5.0 (compatible; grokzilla-url-to-markdown/1.0; +https://grokzilla.shop/terms).
• robots.txt is honored for grokzilla-url-to-markdown (a User-agent: grokzilla group also matches). It is read once per host per call, including for every host a redirect leads to. Per RFC 9309, a 4xx robots.txt means no restrictions; a 5xx or unreachable one means the site is treated as disallowing crawlers.
• SSRF guard (shared with seo and ai-detector): only http/https; every hostname is resolved and the connection pinned to a vetted public IP; private, loopback, link-local and cloud-metadata addresses are refused, on the first request and after every redirect.
• At most 3 redirects, 5 seconds and 1 MB per page. Larger pages are cut at 1 MB and Markdown at 200,000 characters; truncated is then true.
• Only text/html and application/xhtml+xml are converted. JavaScript is not run, so client-rendered pages may come back thin.
• grokzilla.shop, grokzilla.info and x402-micro-pay.com are refused before payment (400, no 402).

Errors (not charged)

• 422 invalid_params: missing url, not a string, no http(s):// scheme, over 2,000 characters.
• 400: unsupported_url (our own domains), blocked_url (private / metadata address), invalid_url, robots_disallowed, robots_unreachable, unsupported_content_type, http_error (the page returned non-2xx), too_many_redirects, redirect_loop.
• 502 upstream_error: timeout or connection failure.

Response (paid 200)

{
  "skill": "url_to_markdown",
  "ok": true,
  "url": "https://example.com",
  "final_url": "https://example.com/",
  "title": "Example Domain",
  "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
  "bytes": 577,
  "truncated": false
}

Unpaid → HTTP 402 (x402 USDC on Base). Retry with PAYMENT-SIGNATURE. Free preflight: send X-X402-Validate-Only: 1 (422 on bad params, else 402).

Price: $0.003 per call.