HTTP response header

Header usage statistics

x-crawler response header information and usage statistics.

Websites using header x-crawler7,648
Percentage of websites that use x-crawler header<0.1%
Total discovered header values1
Header uses directivesNo
Header values are unique or randomNo
Most popular in the country US

x-crawler Directives

x-crawler directives value information and usage statistics

DirectiveShareWebsites countUnique Values

Distribution by websites popularity

x-crawler detection in the top websites by popularity

Top 10k sites0 websites
Top 100k sites2 websites
Top 1m sites8 websites

Websites utilizing x-crawler

List of websites that use x-crawler header

See full domain list
Flat price per the report, subscription is not required.

Geographical Distribution

Header usage distribution by websites across the globe.

Common header values

List of top common x-crawler header values

Header valueValue prevalence