X-Robots-Tag

HTTP response header

The robots meta tag lets you utilize a granular page-specific approach to controlling how an individual page should be indexed and served to users in Google Search results. The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in a robots meta tag can also be specified as an X-Robots-Tag

Header usage statistics

X-Robots-Tag response header information and usage statistics.
Websites using header X-Robots-Tag698,322
Percentage of websites that use X-Robots-Tag header1.05%
Total discovered header values1,685
Header uses directivesNo
Header values are unique or randomNo
Most popular in the country United States

X-Robots-Tag Directives

X-Robots-Tag directives value information and usage statistics
DirectiveShareWebsites countUnique Values

Websites utilizing X-Robots-Tag

List of websites that use X-Robots-Tag header
DomainCountryRankContacts
support.google.com United States43
cloud.google.com United States71
data.jsdelivr.com United States152
support.mozilla.org United States155
yandex.ru Russia163
apple.com United States173
See full domain list

Common header values

List of top common X-Robots-Tag header values
Header valueValue prevalence
noindex, nofollow40.07%
noindex26.01%
noindex, nofollow, noarchive6.17%
none5.03%
all4.57%
nofollow3.26%
noai, noimageai2.21%
all,noodp1.46%
noindex,nofollow1.22%
noindex, follow1.14%
noindex, nofollow, nosnippet, noarchive1.10%
noarchive1.07%
index,follow0.86%
index, follow0.79%
noindex, nofollow, noimageindex0.47%
noindex,follow0.43%
noindex, nofollow, noarchive, nosnippet0.32%
none, noarchive0.31%
noai0.21%
noindex, noarchive, nosnippet0.17%