Robots

HTTP response header

Header usage statistics

Robots response header information and usage statistics.
Websites using header Robots2,335
Percentage of websites that use Robots headerless than 0.1%
Total discovered header values37
Header uses directivesNo
Header values are unique or randomNo
Most popular in the country United States

Robots Directives

Robots directives value information and usage statistics
DirectiveShareWebsites countUnique Values

Websites utilizing Robots

List of websites that use Robots header
DomainCountryRankContacts
global.gotomeeting.com United States9,253
gotomeet.me United States36,751
secp.gov.pk Pakistan91,458
iqconnect.lmhostediq.com United States102,778
ngp.zdf.de United States147,397
getbeans.io United States151,936
See full domain list

Common header values

List of top common Robots header values
Header valueValue prevalence
none70.32%
noindex10.28%
noindex, nofollow5.22%
noarchive2.53%
noindex, nofollow, noai, noimageai2.27%
noindex,nofollow2.23%
no-cache1.07%
index, follow0.86%
all0.77%
noindex, nofollow, noarchive, nosnippet0.51%
noindex, nofollow, noimageindex0.51%
Follow0.47%
index,follow0.47%
,0.39%
NOINDEX,FOLLOW0.30%
noindex, follow0.21%
NoIndex,NoFollow,NoArchive,NoCache,NoSnippet,NoImageIndex0.17%
noindex,nofollow,noarchive0.17%
index,follow,noarchive0.13%
no-index0.13%