User Agent -teracent-feed-processing

[propertyshark]

Does anyone knows some info about  "teracent-feed-processing" user agent?

IP's from which user agent reside:  74.125.113.145, 74.125.113.148, 74.125.187.84 ....

In our logs, 2 out of 3 requests are made by it,   causing server crash.

[propertyshark]

It seems that the Sudden drop in indexed pages reported in  WMT might relate  to some reporting issues from Google - https://productforums.google.com/forum/#!topic/webmasters/qkvudy6VqnM;context-place=topicsearchin/webmasters/sitemap|sort:date

[propertyshark]

Since "teracent-feed-processing" didn't followed the rules in robots.txt,  we had to hard-block it. If server detects the user agent beeing "teracent-feed-processing" it  will drop the connection: _ (104) Connection reset by peer_

[Andy.Drinkwater]

Well it isn't Googlebot and it isn't one I have come across before. Don't forget that any user agent can be spoofed very easily so I wouldn't worry about blocking it.

**Should I  assume that the drop in reported indexed pages is a result of blocking the teracent-feed-processing user agent? **

I really don't think that this is Google. The only one they have is Googlebot, and tell you this is the one to add if you wish to block them.

Just a thought, can you share your robots.txt file just to make sure pages aren't being unintentionally blocked?

-Andy

[propertyshark]

It seems that "teracent-feed-processing" user agent is somehow linked to Google. If you analyse the Ip's , you'll noticed that are Google owned. Teracent company has been bought by Google in 2009.

btw - we've already blocked it,  but I'm trying to figure it out what's the key role played by this user agent. We've also noticed a drastic decline in number of pages being reported in Google Webmaster Tools (half of what we used to have).  Should I  assume that the drop in reported indexed pages is a result of blocking the teracent-feed-processing user agent?

[Andy.Drinkwater]

It sounds like your typical spammy site so I would suggest just blocking them. Add the following to the top of your robots.txt file:

**User-agent: teracent-feed-processing**
**Disallow: /**

However, before you go live with this, use the webmaster tools Robots.txt tester to make sure everything else still gets crawled.

-Andy