Product

A complete diagnosis of the site, the Google Business Profile and Bing, with proof for every number.

Who it's for

Agencies, consultants and companies that need to show what is wrong with a site and follow the fix.

Help

Straight answers, a real report to explore and real people on the other side.

Try it free
Free test

Can AI crawlers reach your site?

The AI crawler test is a free Path Ranker tool that requests your home page with the identity of 18 AI and search crawlers, reads robots.txt, and compares what each one receives with what a browser receives. Type the address: it takes 10 to 30 seconds.

How the test works

01

Reads the page like a browser

The home page is requested first as an ordinary browser. The size and title of that answer become the reference.

02

Repeats it as each crawler

The same page is requested with the user-agent of 18 crawlers, and robots.txt is read with the longest-match rule Google uses.

03

Compares and explains the effect

Each crawler ends up as gets in, blocked, different page or error. The result is then translated into what matters: can ChatGPT, Perplexity, Google, Copilot and Claude read your site?

What this test does not show

  • It uses each crawler's name (user-agent), not its real network addresses. A firewall that checks the address of the genuine crawler may treat it differently.
  • Only the home page is tested. Rules that apply to a folder of the site do not show here.
  • The request leaves from a server in Brazil. A country rule may give another result for crawlers that come from the United States.
  • Getting in is not being cited: the test shows whether the door is open, not whether AI recommends your brand.

That was only crawler access

The full Path Ranker diagnostic also measures speed, structured data, citability and whether AI assistants cite your brand, with the proof for every number.

Try it freeSee how Path Ranker tests crawlers

Frequently asked questions

  • Not necessarily. GPTBot collects pages to train models. What feeds search-based answers is OAI-SearchBot, and ChatGPT-User opens pages when a person asks. The test separates training crawlers from search and on-request ones, and only counts an assistant as blocked when the crawlers it needs are shut out.
  • The server answered the crawler with a refusal a browser did not get (for example 403), or with a challenge page such as "Just a moment". It is usually a firewall or CDN rule, and the fix is to allow the user-agent there, not in robots.txt.
  • It is optional. A study of 300,000 domains measured no effect on citations, so the test only reports whether the file exists and does not count it as a problem.
  • We ask for no sign-up and do not keep the result. It stays in the server's memory for 10 minutes so the same site is not tested back to back, then it is discarded. To prevent abuse we count how many tests each network address ran in the last hour.
  • The test requests public pages only: the home page, robots.txt and llms.txt. Even so there is a limit per visitor and one test per site every 10 minutes.

See the rest of your site, not just the door

Path Ranker runs the full SEO and AI-search diagnostic, with the evidence for every finding and the command to repeat the measurement.

Try it free