Skip to content
Cloudflare Docs

Detect AI crawlers

AI Audit metrics provides you with insight on how AI crawlers are interacting with your website.

View AI Audit metrics

AI Audit provides you with the following metrics to help you understand how AI crawlers are interacting with your website.

MetricDescription
Request by AI crawlersA graph which displays the number of crawl requests from each AI crawler
SummaryA list of AI crawlers with the most number of crawl requests
Most popular paths by AI crawlersThe most popular pages crawled by AI crawlers, for each AI crawler

The Summary table also enables you to Enforce your robots.txt.

Filter AI crawler data

You can use filters to narrow the scope of your result.

  • Provider: Filter by the AI crawler owners.
  • Bot type: Filter by the type of the AI bot (for example, AI crawler, AI assistant, or archiver).
  • Date range: Filter the date range of your results. You can choose from three predetermined date ranges:
    • Past 7 days
    • Past 14 days
    • Past month

The values of the AI Audit metrics will update according to your filter.

Filter subdomains

You can use the subdomain filter to narrow the scope of your result.

From the dropdown, select either All subdomains, or the specific subdomain you wish to view.

Selecting a specific subdomain allows you to access:

  • Violations only toggle: Toggles the AI Audit page to only display bots which are violating your configured rules.
  • Enforce robots.txt policy: Ensure bots cannot access webpages which are off-limits, as specified in your robots.txt file.

The values of the AI Audit metrics will update according to your filter.