HTTP response header

you say to your clients you will crawl their sites as googlebot crawls. And then you should be sure of requesting the same HTTP headers as googlebot from their servers. The response information and later data you collect from a server depend on what you request in your crawlers HTTP headers.

Header usage statistics

X-GoogleNews-Bot response header information and usage statistics.

Websites using header X-GoogleNews-Bot 2,410
Percentage of websites that use X-GoogleNews-Bot header <0.1%
Total discovered header values 1
Header uses directives No
Header values are unique or random No
Most popular in the country United States of America

Distribution by websites popularity

X-GoogleNews-Bot detection in the top websites by popularity

Top 10k sites 5 websites
Top 100k sites 14 websites
Top 1m sites 77 websites

Websites utilizing X-GoogleNews-Bot

List of websites that use X-GoogleNews-Bot header

Domain Country Rank Contacts
lifehacker.com United States of America 167
gizmodo.com United States of America 380
kotaku.com United States of America 1,616
theonion.com United States of America 1,919
www.theonion.com United States of America 1,919
mylesqgms263.kinja.com United States of America 38,811
See full domain list
Flat price per the report, subscription is not required.

Geographical Distribution

Header usage distribution by websites across the globe.

Common header values

List of top common X-GoogleNews-Bot header values

Header value Value prevalence
false 100.00%