Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
What referrer is shown in http request when google crawler visit a page?
-
Is it legit to show different content to http request having different referrer?
case a: user view one page of the site with plenty of information about one brand, and click on a link on that page to see a product detail page of that brand, here I don't want to repeat information about the brand itself
case b: a user view directly the product detail page clicking on a SERP result, in this case I would like to show him few paragraph about the brand
Is it bad? Anyone have experience in doing it?
My main concern is google crawler. Should not be considered cloaking because I am not differentiating on user-agent bot-no-bot.
But when google is crawling the site which referrer will use? I have no idea, does anyone know?
When going from one link to another on the website, is google crawler leaving the referrer empty?
-
After googling about it and thinking better I choose not too. I think you are absolutely right, too inconsistent.
-
You won't get a referrer. Googlebot is not like a real user, surfing from site to site. What Googlebot does is this (more or less)
- Googlebot requests a standalone page
- Googlebot parses the page out. During this process it notes the links on that page and, depending on various mechanisms (nofollow, internal page rank, the mood of Matt Cutts, etc) it will note those links for the system to parse later
- Googlebot is done so it grabs another page off the page list (likely without know how it got on said list) and goes back to #1
Now, to your question. Since Googlebot has no referrer it won't get your alternate content. This means that your alternate content page won't get indexed.
I would suggest here that a best practice is NOT to filter on referrer data, which can be inconsistent and potentially fake. Instead, I would make a separate page that contains your extra data and allow users to decide if they want more information Thus Googlebot finds all your content and your users get a better experience.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How many pages for a new site for Google to sink it's teeth into
Hello, I've got a high traffic site I'm building. Is 20 articles enough to start or should I start with 40 or 60 to not flop? I've got good internal linking. I've got: 3 10X articles
Intermediate & Advanced SEO | | BobGW
17 cornerstones so far. Worked hard so far. Do I launch? or wait for 40? 60? I know thousands is best. Competitors have much more. Thank you, Bob0 -
Rel canonical tag from shopify page to wordpress site page
We have pages on our shopify site example - https://shop.example.com/collections/cast-aluminum-plaques/products/cast-aluminum-address-plaque That we want to put a rel canonical tag on to direct to our wordpress site page - https://www.example.com/aluminum-plaques/ We have links form the wordpress page to the shop page, and over time ahve found that google has ranked the shop pages over the wp pages, which we do not want. So we want to put rel canonical tags on the shop pages to say the wp page is the authority. I hope that makes sense, and I would appreciate your feeback and best solution. Thanks! Is that possible?
Intermediate & Advanced SEO | | shabbirmoosa0 -
Home Page Title - Google Overriding my Title Tag
Hi, We have noticed our home page title tag has now been replaced by our brand name (by Google I'm assuming). We have also noted that the page is dropping off SERPS for our main keywords, I suspect they are related but ofc I cant be sure. I know the recent Google update has impacted titles but I wasn't sure if it would apply here. Has anyone any advice on this and/or having the same issue? We normally rank well for grass seed (UK search) https://thegrasspeople.com/ I also noticed some strange mark up in our source code which seems to have been left behind by Sketch - we are getting this removed. <title>Combined Shape</title> <desc>Created with Sketch.</desc> Chris
Intermediate & Advanced SEO | | Chris_Mc0 -
Should I apply Canonical Links from my Landing Pages to Core Website Pages?
I am working on an SEO project for the website: https://wave.com.au/ There are some core website pages, which we want to target for organic traffic, like this one: https://wave.com.au/doctors/medical-specialties/anaesthetist-jobs/ Then we have basically have another version that is set up as a landing page and used for CPC campaigns. https://wave.com.au/anaesthetists/ Essentially, my question is should I apply canonical links from the landing page versions to the core website pages (especially if I know they are only utilising them for CPC campaigns) so as to push link equity/juice across? Here is the GA data from January 1 - April 30, 2019 (Behavior > Site Content > All Pages😞
Intermediate & Advanced SEO | | Wavelength_International0 -
Google Adsbot crawling order confirmation pages?
Hi, We have had roughly 1000+ requests per 24 hours from Google-adsbot to our confirmation pages. This generates an error as the confirmation page cannot be viewed after closing or by anyone who didn't complete the order. How is google-adsbot finding pages to crawl that are not linked to anywhere on the site, in the sitemap or linked to anywhere else? Is there any harm in a google crawler receiving a higher percentage of errors - even though the pages are not supposed to be requested. Is there anything we can do to prevent the errors for the benefit of our network team and what are the possible risks of any measures we can take? This bot seems to be for evaluating the quality of landing pages used in for Adwords so why is it trying to access confirmation pages when they have not been set for any of our adverts? We included "Disallow: /confirmation" in the robots.txt but it has continued to request these pages, generating a 403 page and an error in the log files so it seems Adsbot doesn't follow robots.txt. Thanks in advance for any help, Sam
Intermediate & Advanced SEO | | seoeuroflorist0 -
Date of page first indexed or age of a page?
Hi does anyone know any ways, tools to find when a page was first indexed/cached by Google? I remember a while back, around 2009 i had a firefox plugin which could check this, and gave you a exact date. Maybe this has changed since. I don't remember the plugin. Or any recommendations on finding the age of a page (not domain) for a website? This is for competitor research not my own website. Cheers, Paul
Intermediate & Advanced SEO | | MBASydney0 -
Will Google View Using Google Translate As Duplicate?
If I have a page in English, which exist on 100 other websites, we have a case where my website has duplicate content. What if I use Google Translate to translate the page from English to Japanese, as the only website doing this translation will my page get credit for producing original content? Or, will Google view my page as duplicate content, because Google can tell it is translated from an original English page, which runs on 100+ different websites, since Google Translate is Google's own software?
Intermediate & Advanced SEO | | khi50 -
Does Google crawl the pages which are generated via the site's search box queries?
For example, if I search for an 'x' item in a site's search box and if the site displays a list of results based on the query, would that page be crawled? I am asking this question because this would be a URL that is non existent on the site and hence am confused as to whether Google bots would be able to find it.
Intermediate & Advanced SEO | | pulseseo0