Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Index process multi language website for different countries
-
We are in charge of a website with 7 languages for 16 countries. There are only slight content differences by countries (google.de | google.co.uk). The website is set-up with the correct language & country annotation e.g. de/DE/ | de/CH/ | en/GB/ | en/IE. All unwanted annotations are blocked by robots.txt. The «hreflang alternate» are also set.
The objective is, to make the website visible in local search engines. Therefore we have submitted a overview sitemap connected with a sitemap per country. The sitemap has been submitted now for quite a while, but Google has indexed only 10 % of the content.
We are looking for suggestion to boost the index process.
-
Thank you.
-
Just a couple thoughts off the top of my head:
1. Double-check all technical international SEO issues and ensure that the robots.txt file is not mistakenly blocking any desired pages.
2. Make sure that you have a separate Google Webmaster Tools setup for each root domain / subdomain / subdirectory (however you have set up the international sites) and have submitted an individual XML sitemap for each one. Also make sure that the geographical targeting in each GWT setup is set to the desired country.
3. If Google is only indexing a small percentage of a site's pages, it is often because Google is thinking (accurately or not) that a site has duplicate content. "Duplicate content" is not a penalty per se -- it is when Google, for example, sees two pages that are very similar and then indexes only one of them so as to not provide redundant pages in search results.
Example: Say that you have an e-commerce product that has ten variations (such as color). The content of each variation page would often be very similar except, for, the listed color. In the case, you would want to use a rel=canonical tag on all variation pages that points to the main page for that product. (In other words, you don't want all of those pages to be indexed, and Google often would not index them anyway.)
Most likely, I would use a tool such as Moz or any other SEO software to crawl the site and see if any duplicate-content issues are present. Once these are addressed (if the problem exists), then Google will likely crawl and index your sites more thoroughly and accurately.
I hope this helps -- good luck!
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Hreflang : mixing with/without country code for same language
Hello, I would like to display 3 different english versions of my website : 1 for UK, 1 for CA and 1 for other english users. It would look like this for a page: . (english content with £ prices) <link rel="alternate" href="https: xxx.com="" en-ca" hreflang="en-CA">(english content with $CA prices)</link rel="alternate" href="https:> <link rel="alternate" href="https: xxx.com="" en="" " hreflang="en">(english content without currency)</link rel="alternate" href="https:> I wonder if I can mix this hreflang without country code with hreflangs with country code for the 2 other specific versions... or if the hreflang without country code version will appear whatever the country, even if i specified it . In other terms, is hreflang="en" > hreflang="en-CA" + hreflang="en-GB" if tagged together on a same page? Thank you
Intermediate & Advanced SEO | | AlexisH0 -
Check website update frequency?
Is the tools out there that can check our frequently website is updated with new content products? I'm trying to do an SEO analysis between two websites. Thanks in advance Richard
Intermediate & Advanced SEO | | seoman100 -
Why does Moz recommend subdomains for language-specific websites?
In Moz's domain recommendations, they recommend subdirectories instead of subdomains (which agrees with my experience), but make an exception for language-specific websites: Since search engines keep different metrics for domains than they do subdomains, it is recommended that webmasters place link-worthy content like blogs in subfolders rather than subdomains. (i.e. www.example.com/blog/ rather than blog.example.com) The notable exceptions to this are language-specific websites. (i.e., en.example.com for the English version of the website). Why are language-specific websites excepted from this advice? Why are subdomains preferable for language-specific websites? Google's advice says subdirectories are fine for language-specific websites, and GSC allows geographic settings at the subdirectory level (which may or may not even be needed, since language-specific sites may not be geographic-specific), so I'm unsure why Moz would suggest using subdirectories in this case.
Intermediate & Advanced SEO | | AdamThompson0 -
How to fully index big ecommerce websites (that have deep catalog hierarchy)?
When building very large ecommerce sites, the catalog data can have millions of product SKUs and a massive quantity of hierarchical navigation layers (say 7-10) to get to those SKUs. On such sites, it can be difficult to get them to index substantially. The issue doesn’t appear to be product page content issues. The concern is around the ‘intermediate’ pages -- the many navigation layers between the home page and the product pages that are necessary for a user to funnel down and find the desired product. There are a lot of these intermediate pages and they commonly contain just a few menu links and thin/no content. (It's tough to put fresh-unique-quality content on all the intermediate pages that serve the purpose of helping the user navigate a big catalog.) We've played with NO INDEX, FOLLOW on these pages. But structurally it seems like a site with a lot of intermediate pages containing thin content can result in issues such as shallow site indexing, weak page rank, crawl budget issues, etc. Any creative suggestions on how to tackle this?
Intermediate & Advanced SEO | | AltosDigital-10 -
Google indexed wrong pages of my website.
When I google site:www.ayurjeewan.com, after 8 pages, google shows Slider and shop pages. Which I don't want to be indexed. How can I get rid of these pages?
Intermediate & Advanced SEO | | bondhoward0 -
Links from non-indexed pages
Whilst looking for link opportunities, I have noticed that the website has a few profiles from suppliers or accredited organisations. However, a search form is required to access these pages and when I type cache:"webpage.com" the page is showing up as non-indexed. These are good websites, not spammy directory sites, but is it worth trying to get Google to index the pages? If so, what is the best method to use?
Intermediate & Advanced SEO | | maxweb0 -
How do you de-index and prevent indexation of a whole domain?
I have parts of an online portal displaying in SERPs which it definitely shouldn't be. It's due to thoughtless developers but I need to have the whole portal's domain de-indexed and prevented from future indexing. I'm not too tech savvy but how is this achieved? No index? Robots? thanks
Intermediate & Advanced SEO | | Martin_S0 -
How important is the number of indexed pages?
I'm considering making a change to using AJAX filtered navigation on my e-commerce site. If I do this, the user experience will be significantly improved but the number of pages that Google finds on my site will go down significantly (in the 10,000's). It feels to me like our filtered navigation has grown out of control and we spend too much time worrying about the url structure of it - in some ways it's paralyzing us. I'd like to be able to focus on pages that matter (explicit Category and Sub-Category) pages and then just let ajax take care of filtering products below these levels. For customer usability this is smart. From the perspective of manageable code and long term design this also seems very smart -we can't continue to worry so much about filtered navigation. My concern is that losing so many indexed pages will have a large negative effect (however, we will reduce duplicate content and be able provide much better category and sub-category pages). We probably should have thought about this a year ago before Google indexed everything :-). Does anybody have any experience with this or insight on what to do? Thanks, -Jason
Intermediate & Advanced SEO | | cre80