Turn a website into an AI-readable knowledge bundle.
Paste a URL below and get back an Open Knowledge Format bundle: a zip of clean, linked markdown files that AI agents can read instead of scraping HTML.
What you get
index.md article.md product.md + linked markdown files
01 Paste a URL
Give it any site's homepage or sitemap URL.
02 It reads the sitemap
Discovers every page (up to 30) and skips anything robots.txt disallows.
03 Each page becomes markdown
It pulls out the main content and converts it to clean markdown with YAML frontmatter, links intact.
04 Download the bundle
A zip of linked concept files plus an index.md manifest, ready to use.
- Open index.md first. It lists every page with a one-line description, so an agent (or you) can see how the site is laid out before opening every file.
- Drop the unzipped folder into a coding agent or chat tool's project files so it can read the whole site as clean markdown instead of scraping HTML.
- Use it for RAG. The files are already split one-per-page and cross-linked, so you can chunk and embed them directly instead of scraping raw HTML yourself.
- Host it at /okf/ on your own site if you want AI agents to discover it. That's what this site itself does with its own bundle.
- No JavaScript rendering. The crawler reads only the initial server-rendered HTML, so pages that need client-side JS to show their main content won't extract well.
- Capped at 30 pages and about 75 seconds of crawl time per site. Larger sites get a partial bundle, and index.md notes when that happens.
- Respects robots.txt. The crawler skips disallowed pages, and refuses any site publishing a Content-Signal: ai-input=no header.
- Rate-limited to a few generations per hour per visitor, since this is a shared tool running on a small server.
- Content extraction is best-effort. The main-content heuristic can grab too much or too little on unusually structured pages.