Blocking domestic Google's in Robots.txt

[Bush_JSM]

Hey,

I want to block Google.co.uk from crawling a site but want Google.de to crawl it.

I know how to configure the Robots.txt to block Google and other engines - is there a fix to block certain domestic crawlers?

any ideas?

Thanks

B

[Bush_JSM]

Thanks Guys for all of the help.

I think we will just implement cross domain GeoIP redirects to ensure users get the right location and currency.

Cheers

[CHarkins]

Are you having the issue of your .de pages ranking in .co.uk instead of your .co.uk pages?

If that's the case then I'd look towards usage of HREFLANG both on-page and in the xml sitemaps. That is going to provided Googlebot with a better view of the country-language targeting for the site.

[Devanur-Rafi]

Hi, country specific search engine spiders cannot be blocked using robots.txt file or any other method. However, you can block certain IP ranges pertaining to certain countries.

Best regards,

Devanur Rafi

[Martijn_Scheijbeler]

Hi Gareth,

I don't think this is going to work as every crawler by Google is run by the same useragent: Googlebot. What you could do but what I really wouldn't recommend to do is generating the robots.txt automatically. Check if the IP address of the user is in another country and then Disallow them. It probably won't work as the crawler from let's say Germany could also be used for the UK.

Also the specific data for a countries search engine gets collected first and then gets looked at to see what they need users to serve, not the other way around that content gets acquired for a specific country.

Hope this helps!