Googlebot being redirected but not users?
-
Hi,
We seem to have a slightly odd issue.
We noticed that a number of our location category pages were slipping off 1 page, and onto page 2 in our niche. On inspection, we noticed that our Arizona page had started ranking in place of a number of other location pages - Cali, Idaho, NJ etc. Weirdly, the pages they had replaced were no longer indexed, and would remain so, despite being fetched, tweeted etc.
One test was to see when the dropped out pages had been last crawled, or at least cached. When conducting the 'cache:domain.com/category/location' on these pages, we were getting 301 redirected to, you guessed it, the Arizona page. Very odd. However, the dropped out pages were serving 200 OK when run through header checker tools, screaming frog etc.
On the face of it, it would seem Googlebot is getting redirected when it is hitting a number of our key location pages, but users are not. Has anyone experienced anything like this? The theming of the pages are quite different in terms of content, meta etc.
Thanks.
-
Can anyone offer any insight on this? The issue shows no sign of improving, and our organic is tanking.
If this really is an issue on Google's side, and it appears based on our technical set up, how can I force Google to take note and reindex the pages?
Here are some more examples, from across a number of categories..
Category A:
Page that has dropped out of index: https://goo.gl/KEQ8Yh
Cached version of that page: https://goo.gl/DPzFWMPage that has dropped out of index: https://goo.gl/5KiQ4s Cached version of that page: https://goo.gl/myRWNg Category B: Page that has dropped out of index: https://goo.gl/pr3YQs
Cached version of that page: https://goo.gl/8SEYi5 Page that has dropped out of index: https://goo.gl/LqzDrg
Cached version of that page: https://goo.gl/iwPs45 Category Page that has dropped out of index: https://goo.gl/YBZS7c Cached version of that page:https://goo.gl/n33QzG Page that has dropped out of index: https://goo.gl/Ht4gfO
Cached version of that page: https://goo.gl/u81vbA -
Weirdly, at the time this seemed to correct itself overnight without any fixes our end. However 6 months on and it has happened again... Only this time, all location pages have dropped out of index, but latest cache versions all redirect to the Kansas page.
Have checked log files for these relevant URLs, and all show 200 (and being crawled regularly). When you fetch them in SC the show as complete, and they are all still indexed in Bing.
Very confused!
-
Hi Nicola,
Have you checked your canonical tags to be sure that they are pointing to the correct URL? Sometimes with storefront pages people just copy/paste the same HTML, swapping out the contact information. But maybe they forgot to update the canonical tags?
Keep us updated!
-
Thanks for advice Chris. Have checked, and can't see any issues with htaccess or robots seem. A very strange one indeed!
-
Hi Anthony,
Desktop returns 200, whilst mobile returns a 302 to the mDOT site.
Thanks.
-
Hi Nicola,
Sounds like a very strange one indeed. I can't say I've ever heard of Google giving a redirect when other 3rd party crawlers give you a 200.
My only suggestion to begin the troubleshooting would be to have a look at your htaccess and maybe Robots to see if there are any directives in there addressing Google specifically.
Anthony's suggestion is also a great idea. At least you'll see what Google is picking up on those pages right now rather what they saw last time it was cached.
-
You mention these pages are returning 200 in SEO tools, but would you please share what are these URLs are returning when you Fetch as Googlebot in Google Search Console? Test it in both desktop and mobile versions.
Got a burning SEO question?
Subscribe to Moz Pro to gain full access to Q&A, answer questions, and ask your own.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How to manage user profiles in your website?
We have a real estate website in which agents and builders can create their profiles. My question is shall we use h1 or h2 tags in business profile pages or make them according to web 2.0 standards? In case header tags are used, if two agents have the same name and we have used h2 tag for them, then search result page will end up having two same h2's. Can someone please tell me the right way to manage business profiles in a website? Thanks
Intermediate & Advanced SEO | | dailynaukri1 -
After 301 redirect
hello i do after 301 redirect from old domain to new since 3 month ago my qa : should i replace the backlinks links to new doamin Or the he backlinks in the old link will works
Intermediate & Advanced SEO | | cristophare790 -
Redirection: Load balancer or CNAME?
We had a bunch of domains which no longer have sites tied to them but many have decent links pointing to them. In most cases we have other relevant content on live sites we can redirect these URL's to. We have been given the choice of redirection through the load balancer or direct as a cname on our CDN. I only have experience of 301's - What would be the preferred choice from an SEO perspective? Thanks, Sam
Intermediate & Advanced SEO | | Samsam00000 -
Redirecting Old Websites to New Websites
Hi Everyone, We are about to take down a number of websites in favour of a new singular B2B hub and would be looking to redirect all of these sites to the new home. For SEO purposes, what would be the best way to do this? Due to the difference in setups and scale of the site, it would be difficult to correctly match up each page to page between the sites for individual 301 redirects. Could someone advise on the best plan of action? Thanks.
Intermediate & Advanced SEO | | chbiz0 -
Fetch As Google Redirect from HTTPS to HTTP
Hi Moz Community, We just launched a redesigned site and moved our blog from http://sparkline.motifinvesting.com to https://www.motifinvesting.com/blog. When I went to check the site today I noticed something strange. When I Fetch a Blog URL as Google it is redirecting from https to http AND dropping the / <post-id>from the end of the URL.</post-id> So for the URL https://www.motifinvesting.com/blog/cleantech-obama-romney/1494 I am getting the attached results when I Fetch as Google. When I access the same URL in my browser, I see a 200 status code and no redirect. Why would Google Bot show that the URL is being redirected? Thanks,
Intermediate & Advanced SEO | | motif_marketing
Breanna https://www.motifinvesting.com/blog/cleantech-obama-romney/1494 I7yQPza.png0 -
301 redirecting staff Domain to Company Domain
My colleague owns a domain (A) for about 10 years that he does not use. The domain's content is the same as my company's website (B) content.
Intermediate & Advanced SEO | | khi5
Question: Can I 301 redirect domain A to domain B's homepage or is it better he just closes down his website since this would not be SEO best practices? thank you0 -
How long do 301 redirects have to stay in place?
For a large retail site we have plenty of "old" pages that are 2-3 years old and still have 301 redirects to a new page. After a search engine has recognized a 301 redirect and dropped the "Old" URL from the index and started displaying the "New" URL, is it safe to delete that old page and thus remove the 301 redirect?
Intermediate & Advanced SEO | | SEOmoxy0 -
Redirects/Forwarding
I have two niche e-commerce sites. One is a PR3 with 3K pages indexed, the other is PR0 with 5K pages indexed. Each site has a blog that has been updated regularly. They both rank well for some fairly competitive keywords and some good links pointing to them. I also have a main site that is PR3. I am thinking of closing down the sites because they are not generating enough revenue, here are my questions: What is the best way to get the most SEO value from these sites? Do I just do a redirect to the main site? Should I keep the sites and use canonical URLs to the main site? Should I keep the domain as a wordpress blog and point links to the main site? What should I do with the blogs? They are on sub-domains, neither has pagerank. Thanks
Intermediate & Advanced SEO | | inhouseseo0