Website Crawl problems
-
I have a feeling that Google doesn't crawl my website.
E.g. this blogpost - I copy a sentence from it and paste it to Google. The page that shows up in search results is www.silvamethodlife.com/page/9/ - which is just a blog page with all the articles listed, not the link to the article itself!
Did anyone ever have this problem? It's definitely some technical issue.
Any advice will be deeply appreciated
Thanks
-
When i use the same tactic as you defined in your question’s explanation i was able to see your page in the search results...
Use this common in search bar to see how many pages are indexed in Google ‘site:example.com’.
May be not all the pages are indexed, for that try and check the following two things:
- Make sure all pages of the website are follow,index for Google and other search engines
- Update your sitemap and submit it to Google webmaster tool and you will see the efficiency in website page index.
Hope this helps!
-
I noticed that your category pages specify noindex,follow and index,follow. Might want to make a decision there.
Google has indexed 676 pages of the 817 on your site and, since their are quite a few URLs that specify nofollow correctly without the additional index like mentioned above, I'd say that is about right.
Did not see the issue you spoke of when I checked some random sentences.
Wouldn't be concerned unless you notice this is coupled with a decrease in organic traffic. If so start looking at off page factors.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
About DA of the Website
Hi There, I would like to know why one of my website - https://patnalocalguide.com Domain Authority is not improving. It is not going up and down but it is maintaining its score between 10 since last 5 months. Not going above 10 or below. Why is it so? I'm conitnuously working on creating backlinks for the website and the DR score on Ahref improving but the DA is not. What steps to follow to increase DA? How to find out whats going wrong with our present strategy?
Intermediate & Advanced SEO | | MozByAnuj0 -
Website removed from Bing and Yahoo
Hello, Our website howtoremove.guide was recently removed from the Bing and Yahoo index. The first thing we did was contact Bing Webmaster support to ask what the issue was since we did not get any notifications or messages in our webmaster dashboard. The email that we got back said “I have escalated the issue to our engineers and will get back to you once I receive an update.” Since then, we haven't received any word back from them, but we did not find any technical problems and we strongly believe we were manually penalized. We've never had issues with a search engine before, so we are at a loss what to do. Could you please give us advice as to what technical issue our website might have or what could incur a deindex penalty in our case? We want to do everything that is possible to get back into Bing and Yahoo search results ASAP. The website has primarily affiliate content, so we are doing anything we can to clean everything up, but any recommendations will be incredibly useful to us. We are also open to contacting an expert on this, but we have no idea where to look.
Intermediate & Advanced SEO | | ThreatAnalyzer0 -
Conundrum with brand new website keywords...
I'm working with on a website for an app called BetterRX. There's a prescription card called BetterRX Card. Our domain is Better RX.com and the card is BetterRXCard.com. "Better RX" as a brand search is dominated by prescription discount cards, with Good RX being the most dominant. Any suggestions on how to go about mixing optimization for brand as well as the app?
Intermediate & Advanced SEO | | sickle3110 -
Best Sitemap for Large Website
i have more than 3500 pages on my website. Please let me know the best sitemap plugin for my website.
Intermediate & Advanced SEO | | Michael.Leonard1 -
Merging 3 websites into 1
Hi I was wondering if someone can give me a bit of advice - outside of my full time job I run three websites - all in the same area, but all do three different things 1. Directory (DA19) 2. Blog (DA 23) 3. Products (DA35) I want to merger all three website into one and all be on one website (rather do it now than keep working on three websites). I have included the DAs of each site ( I know they are not amazing, but i've only recently started working on two of them), but I want to place all three websites under the Blog url. Regarding 301's of the pages, would I be better doing at the top level and 301 all the pages to the home page, or spending the time and 301 the old product page for instance to the new product page - this is a much bigger project, but what are the potential gains. Is there anything else I should consider when switching the sites - all three are wordpress sites (I know it has its limitations but they are easy to create). Thanks in Advance Andy
Intermediate & Advanced SEO | | Andy-Halliday0 -
URL Parameter & crawl stats
Hey Guys,I recently used the URL parameter tool in WBT to mark different urls that offers the same content.I have the parameter "?source=site1" , "?source=site2", etc...It looks like this: www.example.com/article/12?source=site1The "source parameter" are feeds that we provide to partner sites and this way we can track the referral site with our internal analytics platform.Although, pages like:www.example.com/article/12?source=site1 have canonical to the original page www.example.com/article/12, Google indexed both of the URLs
Intermediate & Advanced SEO | | Mr.bfz
www.example.com/article/12?source=site1andwww.example.com/article/12Last week I used the URL parameter tool to mark "source" parameter "No, this parameter doesnt effect page content (track usage)" and today I see a 40% decrease in my crawl stats.In one hand, It makes sense that now google is not crawling the repeated urls with different sources but in the other hand I thought that efficient crawlability would increase my crawl stats.In additional, google is still indexing same pages with different source parameters.I would like to know if someone have experienced something similar and by increasing crawl efficiency I should expect my crawl stats to go up or down?I really appreciate all the help!Thanks!0 -
Getting Your Website Listed
Do you have any suggestiongs? I do not know local websites where I can get some easy backlinks. I guess a record in Google Places.would be great as well. Any sound suggestion will be appreciated. Thanks!
Intermediate & Advanced SEO | | stradiji0 -
SeoMoz Crawler Shuts Down The Website Completely
Recently I have switched servers and was very happy about the outcome. However, every friday my site shuts down (not very cool if you are getting 700 unique visitors per day). Naturally I was very worried and digged deep to see what is causing it. Unfortunately, the direct answer was that is was coming from "rogerbot". (see sample below) Today (aug 5) Same thing happened but this time it was off for about 7 hours which did a lot of damage in terms of seo. I am inclined to shut down the seomoz service if I can't resolve this immediately. I guess my question is would there be a possibility to make sure this doesn't happen or time out like that because of roger bot. Please let me know if anyone has answer for this. I use your service a lot and I really need it. Here is what caused it from these error lines: 216.244.72.12 - - [29/Jul/2011:09:10:39 -0700] "GET /pregnancy/14-weeks-pregnant/ HTTP/1.1" 200 354 "-" "Mozilla/5.0 (compatible; rogerBot/1.0; UrlCrawler; http://www.seomoz.org/dp/rogerbot)" 216.244.72.11 - - [29/Jul/2011:09:10:37 -0700] "GET /pregnancy/17-weeks-pregnant/ HTTP/1.1" 200 51582 "-" "Mozilla/5.0 (compatible; rogerBot/1.0; UrlCrawler; http://www.seomoz.org/dp/rogerbot)"
Intermediate & Advanced SEO | | Jury0