Crawler issues
-
Can anyone please suggest why our site is not being crawled by Google at the moment?
Thanks,
-
No problem, happy to help
-
Thats a good point Adam. Thanks for your assistance.
-
Thanks Doug. That is really helpful.
-
Thanks for your help Chris
-
Yep. We did that. Thanks for the suggestion.
-
whoop there it is!
-
Double check your redirects again. When I tried to check your backlinks, I was showing redirects between tasteinc.com, www.tasteinc.com, and www.tasteinc.com/ Every time I tried to find the profile using one of those links, it would tell me that it redirects to another one of those.
I didn't have a problem going to your site, but the way that software was trying to crawl things, I would double check that again. That's the only issue that I saw. Has the site been up long?
-
Tried to crawl your site with Screaming Frog and noticed that your http response header contains a x-robots-tag which is being set to noindex, nofollow.
You can check your http headers on this site: http://web-sniffer.net/
-
I would agree with Mike you can see how here - https://support.google.com/sites/answer/100283?hl=en-GB
How long have you had the site up for? Remember it can take time for it to get crawled. You can ask google to crawl a page but this can still take time.
Good luck.
-
Hi Craig,
Did you submit your sitemap in Google Webmaster Tools?
Mike
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Does a similar CMS fabric/theme used by 2 colleagues, cause SEO issues?
Does using the same CMS fabric/theme with substantially different content for each website but the same business address (on google places and the websites) cause SEO issues? For example: 2 colleagues with somewhat similar services located at the same business addresses desire to use the same CMS fabric /theme but will have different content on the sites. Will this hurt their SEO / should they use a different website skin/theme?
Web Design | | toti5880 -
Canonical and Sitemap issue
Hi all, I was told that I could change my homepage Canonical tag to match that of my XML sitemap, this sitemap is being generated for me automatically and shows the homepage as e.g. https://www.mysite.com/index.html, yet my Canonical tag has been set to https://www.mysite.com. Google currently shows as https://www.mysite.com/ being indexed, but https://www.mysite.com/index.html is not currently displayed in search results. Can someone please tell me if I should change the Canonical to the index.html version, or if I should do nothing, or remove the Canonical tag altogether? Thank you for looking.
Web Design | | scarebearz0 -
Is there an issue if we show our old mobile site to Google & new site to users
Hi, We have our existing mobile site that contains interlinking in footer & content and new mobile site that does not have interlinking. We will show existing mobile site to google crawler & new mobile site to users. Will this be taken as black hat by Google. The mobile site & desktop site will have same url across devices & browsers. Regards
Web Design | | vivekrathore0 -
Confirm responsive design is accessible to crawlers?
Trying to ensure crawlers can access all important info on a new responsive website, but it's javascript-heavy. Following Google's advice/best practices, but can't get the Fetch and Render tool to confirm it's working. Anyone dealt with this before? Also posted in Product Forums with more detail: https://productforums.google.com/forum/#!msg/webmasters/PjxtBYClupw/56id80FUQdMJ
Web Design | | BoxerPropertyHouston0 -
W3C My site has 157 Errors, 146 warning(s) Is it an issue?
Is having this number of W3C errors & warnings an issue and will be impacting my site's performance? When the site was built 6 months ago my developers told me that it "was nothing to worry about", but I have read that any errors aren't good, let alone the huge number my site has? Your advice please Thanks Ash
Web Design | | AshShep10 -
Panda and Penquin Fall - Could HTML Design an Issue?
Hi, We were hit hard by Panda 3.4 on March 23rd 2012. Then Penguin came along and slapped us down a little farther on April 24th. White hat SEO for 13 years on the site. I have been trying to discover the reason we got hit so hard, to date 90% down. We ae wiped. I have a couple of keywords still #2 and #3 and we see up and down changes in Google webmaster tools, i.e. a keyword is supposedly up 50 points then another down 50. All other 150 keywords that we used to rank on the first page for are not even showing up. I have a person that is about to do a full link analysis but since we never went after links I just never had the feeling that is where our problem is at, but definitely going to explore it. The reason for my post is that last night I spoke with an SEO person that has some pretty good credentials (9 years experience and works currently at large online marketing company with seo with clients like Honda) and he was nice enough to just take a quick look at the site. He said he saw nothing really wrong and did not think that we were hit for any of the normal issues people are listing, i.e. duplicate content, backlinks. His first impression was that we were knocked down because the site is "hard to index". He said the site still uses tables and a lot of our Doc Statements were for HTML 4.01 from 1999. As we all know, there are 'many' experts in this industry. So I wanted a little feedback from the community. Our main site was built in Dreamweaver using tables. We do have a Wordpress blog that is very small and just now posting to add fresh content. (posts seem to rank pretty good, this is why I thought, you know he may be right) Would an older site be penalized like this for using tables? What would you do at this stage if you had a site that is not recovering? I have now reached panic mode and have to do something, just not sure of the next step. I will be happy to post the URL if anyone wants to help with advice. Thanks,
Web Design | | Force7
Force70 -
Wordpress drop down menu issue...
While I am tweaking my current site with getting rid of my footer links, my site will not allow a drop down menu for my service area parent page. It will do a drop down menu for any other tab. Could not find anything in a google search besides just link to parent for drop down... Any ideas?
Web Design | | greenjoe0 -
Are my duplicate meta titles and descriptions an issue ?
HelloMy website http://www.gardenbeet.com has been rebuilt using prestacart and there are 158 duplicate title and meta descriptions being reported by google.My developer advised the following Almost all the duplicates are due to the same page being accessible at the root and following the category heading. e.g; /75-vegetable-patio-planter-turquoise.html
Web Design | | GardenBeet
/patio-planters/75-vegetable-patio-planter-turquoise.html This is hard-wired into PrestaShop. Was the Canonical module (now disabled) responsible for the confusion by not including the category name? The Googlebot shouldn't be scanning the root versions now. I don't believe this to be a serious issue but I'd recommend a second opinion from someone more SEO savvy just to be sure.Opinions??0