The robots meta tag lets you utilize a granular page-specific approach to controlling how an individual page should be indexed and served to users in Google Search results. The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in a robots meta tag can also be specified as an X-Robots-Tag
Websites using header X-Robots-Tag | 39,798 |
Percentage of websites that use X-Robots-Tag header | 0.37% |
Total discovered header values | 303 |
Header uses directives | No |
Header values are unique or random | No |
Most popular in the country | ![]() |
Top 10k sites | 109 websites |
Top 100k sites | 673 websites |
Top 1m sites | 5,242 websites |
Domain | Country | Rank | Contacts |
---|---|---|---|
|
![]() |
12 | |
|
![]() |
162 | |
|
![]() |
228 | |
|
![]() |
229 | |
|
![]() |
332 | |
|
![]() |
369 |
Header value | Value prevalence |
---|---|
noarchive | 16.04% |
nofollow | 14.41% |
noindex, nofollow | 13.52% |
all | 11.02% |
noindex | 8.76% |
noindex,nofollow | 8.24% |
noindex, nofollow, noarchive | 8.15% |
none | 7.24% |
all,noodp | 4.62% |
index, follow | 1.14% |
index,follow | 1.14% |
noindex, nofollow, nosnippet, noarchive | 0.62% |
noimageindex | 0.62% |
noindex, nofollow, nosnippet | 0.43% |
noarchive, notranslate | 0.27% |
noarchive, noindex, nofollow | 0.20% |
index | 0.19% |
noindex, noarchive, nosnippet | 0.18% |
noindex, follow | 0.17% |
noindex, nofollow, noimageindex | 0.15% |