Tools

OKF Bundle Generator

Paste any website's URL below and get back an Open Knowledge Format bundle for it: a zip of clean, linked markdown files that AI agents can read instead of scraping HTML.

How it works
01 Paste a URL

Give it any site’s homepage or sitemap URL.

02 It reads the sitemap

Discovers every page (up to 30) and skips anything robots.txt disallows.

03 Each page becomes markdown

Main content is extracted and converted to clean markdown with YAML frontmatter, links intact.

04 Download the bundle

A zip of linked concept files plus an index.md manifest, ready to use.

What to do with your bundle
  • Open index.md first: it lists every page with a one-line description, so an agent (or you) can see the shape of the site before opening every file.
  • Drop the unzipped folder into a coding agent or chat tool’s project files so it can read the whole site as clean markdown instead of scraping HTML.
  • Use it for RAG: the files are already split one-per-page and cross-linked, so you can chunk and embed them directly instead of scraping raw HTML yourself.
  • Host it at /okf/ on your own site if you want AI agents to discover it. That’s what this site itself does with its own bundle.
Limitations
  • No JavaScript rendering: only the initial server-rendered HTML is read, so pages that need client-side JS to show their main content won’t extract well.
  • Capped at 30 pages and about 75 seconds of crawl time per site. Larger sites get a partial bundle, and index.md notes when that happens.
  • Respects robots.txt: disallowed pages are skipped, and sites publishing a Content-Signal: ai-input=no header are refused entirely.
  • Rate-limited to a few generations per hour per visitor, since this is a shared tool running on a small server.
  • Best-effort content extraction: the main-content heuristic can occasionally grab too much or too little on unusually structured pages.