x-robots-tag

HTTP response header

The robots meta tag lets you utilize a granular page-specific approach to controlling how an individual page should be indexed and served to users in Google Search results. The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in a robots meta tag can also be specified as an X-Robots-Tag

Header usage statistics

x-robots-tag response header information and usage statistics.
Websites using header x-robots-tag638,161
Percentage of websites that use x-robots-tag header0.79%
Total discovered header values1,554
Header uses directivesNo
Header values are unique or randomNo
Most popular in the country United States

x-robots-tag Directives

x-robots-tag directives value information and usage statistics
DirectiveShareWebsites countUnique Values

x-robots-tag header usage distribution by website rank



Geographical Distribution

Header usage distribution by websites across the globe.



Websites utilizing x-robots-tag

List of websites that use x-robots-tag header
DomainCountryRankContacts
support.google.com United States43
cloud.google.com United States71
www.adobe.com United States88
data.jsdelivr.com United States152
support.mozilla.org United States155
yandex.ru Russia163
See full domain list

Common header values

List of top common x-robots-tag header values
Header valueValue prevalence
noindex, nofollow39.60%
noindex22.57%
none7.27%
noindex, nofollow, noarchive6.71%
all4.77%
nofollow3.55%
all,noodp1.57%
noai, noimageai1.55%
noarchive1.35%
noindex,nofollow1.27%
noindex, follow1.17%
noindex, nofollow, nosnippet, noarchive1.04%
index,follow1.02%
index, follow1.01%
noindex, nofollow, noimageindex0.51%
noindex,follow0.45%
none, noarchive0.35%
noarchive, noindex, nofollow0.29%
noimageai, noai0.28%
noai0.27%