Field guides

Make your website easier for AI systems to use

Start with the access decision you need to make. Each guide separates established crawler controls from optional or emerging conventions, then gives you a way to verify the result.

Not sure where to begin?

Follow the request from discovery to action.

  1. DiscoverCheck robots.txt, crawler identities, and the sitemap.
  2. UnderstandExpose clear HTML, page facts, links, and structured data.
  3. UseMake important forms and actions identifiable and reliable.

Check the production evidence

See which signals your website actually serves.

Scan my website