See what AI assistants read on your site
Enter an address. In about ten seconds you see the words an AI crawler actually gets, which crawlers you block, and what fails.
What it checks, and why
What an AI crawler reads
The words in your page's HTML before any script runs. GPTBot, ClaudeBot and PerplexityBot do not run JavaScript, so a page drawn by a script is close to empty to them. Source: Vercel and MERJ, December 2024.
An address that does not exist
Many sites answer every address with the same page and status 200. Checkers then report robots.txt, the sitemap and other files as present when they are not. We ask for a made-up address to find out.
AI crawlers in robots.txt
Ten crawlers by name, split into the ones that train models and the ones that read a page when a person asks about you. Blocking the second kind keeps you out of answers.
Structured data, title, description and sitemap
What lets an assistant say what the page is about and find the rest of the site.
Markdown and llms.txt, reported but not scored
Ahrefs found that almost every llms.txt file it studied got no requests at all in May 2026, and only some coding agents ask for markdown. We tell you whether you have them and do not pretend they matter more than they do. Sources: Ahrefs, 15 June 2026; Checkly, February 2026.
Whether the site looks AI-built
The palette, faces and layout habits that make a site look generated, read from the page and its stylesheets.
We fetch the page, robots.txt, the sitemap, llms.txt, one made-up address and up to three stylesheets, as a crawler would. Results are kept for 30 days at their own address, and are not listed or indexed.