Hi Joseph
It's usually totally fine to leave your company name in there by default. Then if a particular page title is too long, you can edit the title tag on an individual page basis and create a title without the business name.
Welcome to the Q&A Forum
Browse the forum for helpful insights and fresh discussions about all things SEO.
Hi Joseph
It's usually totally fine to leave your company name in there by default. Then if a particular page title is too long, you can edit the title tag on an individual page basis and create a title without the business name.
Hey Joey
They would just need to be able to crawl and access it somehow. So you'd want to link to that content from public pages, or maybe try putting it in an XML or HTML sitemap, and as mentioned try "submitting to index" in Search Console. But yes, they'll index it as long as they can crawl it from somewhere else. Running the crawler on your site and seeing if it gets picked up is a good way to see.
Hi Joey!
I would just run a crawler like Screaming Frog - and if your PDFs and Images show up in the crawl report, then Google will be able to find them and index them. Since they are behind a login, that's the only reason they might get blocked. Also, you could try to submit them to Google with the "Fetch and render" in Search Console and "submit to index".
Hi Kate
Wanted to be sure you saw Barry's reply over on the Google forum. He also seems to think this would be fine, but I do agree with his recommendation to target all bots, not just Googlebot.
I also was able to find your page in Google and checked the cached version. Google's cache looks the same as the live version, so I think you should be fine.
Hello!
As far as the indexation - let me answer that first. Does what you see here in a site: search match more to what you'd expect? The WMT numbers can tend to lag behind. I would generally look at the site: search for the "real" number.
Also, have you tried a "fetch as Google" and then "submit to index" for pages that still may not be indexed?
-Dan
Hi David
You can probably leave the pages as-is and allow Google to crawl them. But you may want to update the part of the content that's triggering the duplicate errors. In other words - are your title tags and meta descriptions unique for each page? Or maybe the H1's are duplicates? Since the pages do have slight differences, I would use those differences to make the content unique.
In general I would stick to paginated. Scrolling may add to page load time, and can sometimes frustrate users if they are trying to just get to your footer. Paginated is a very standard and expected method by users and search engines
Hi Kate
I don't think it would be looked at as cloaking. One thing I would definitely do though to check is a fetch and render in search console. In the render, the Googlebot view should be the same as the browser view. You can also check the cached version of the page as well.
Mike and Dmytro answered this really well. It should be blocked in robots.txt
But also, you might be linking to your login page publicly. I often see links to login or for "admin" in wordpress themes in a sidebar, widget or footer. You should probably remove those as well (unless you allow public users to create their own account and log in).
You can do a "site:" search directly in Google like this and I currently see this --> http://screencast.com/t/ZVqq5iumQ - you can probably do a site: search on the whole domain, a subfolder or a specific page etc.
Thanks! OK, yes I'd make your Sitemap and HTML image URLs the same.
Also, that's a LOT of images, so I'm not surprised Google is taking time to index them.
Also, there can sometimes be a delay in Search Console data. You can always be checking Google itself to see what files are indexed.
Thanks! Hmmm did it clear Search Console without any errors? I see an error in my browser --> http://screencast.com/t/VLWhg8EyR3Dd
Edward
In the big picture, Martijin is right - unless you need traffic from the images, it's OK they are not indexed.
But secondly, looks like they are hosted on wp.com like this one - https://i2.wp.com/edwardsturm.com/wp-content/uploads/2016/03/Reddit-Gaming-Front-Page.png?resize=860%2C883&ssl=1
They show up in my crawl OK, but have you tried a "fetch and render" in Google Search Console? I would do that on any of your blog posts and see if the render shows the images.
My guess is it's not an issue for you, but maybe something wp.com is blocking access to, or not allowing indexation on.
Is this your current sitemap?
http://www.parismatch.com/var/exports/sitemaps/sitemap_parismatch-index.xml
What is the direct address of the image sitemap(s)?
Thanks!
Hi Sharon
Is there specific content or pages you're not seeing indexed that should be?
I checked with a site search and see that about 282 pages are indexed right now. I crawled the site, got about 578 active URLs, and subtracting /wp-content/ URLs and subpages, that leaves about 280ish URLs (correction, 380ish), which is the number indexed in Google. which is only 100 more than the number indexed in Google.
Perhaps things caught up, let me know if there's a URL not indexed that you expect to be.
Thanks!
-Dan
I see, thanks. Hmmm... did anything else change besides the re-design? Did the images URLs change, or did where they were being hosted change?
The current implementation doesn't show any issues, but I wonder if things were properly done in moving to the new design. Did you always have a slideshow format? Did the code change or just the design?
Hi There
There does not appear to be any accessibility issues. I can crawl and access the images just fine with my crawler.
My guess is that since the images are duplicate, and they also exist on other websites, Google may be avoiding indexing them since they already are indexed and they are technically not being linked to with a normal tag.
Is this causing a particular issue for the site? Or is it just a pesky technical bug?
Hey There
I just did a reverse image search on two of your images and they are present in Google Image search
But one issue, is that when I click 'view image' (what normally would open the image file in a new tab - instead it triggers a download box for me --> http://screencast.com/t/7LyLRRJ4CTb6 - perhaps this is because you are preventing people from doing so and just copying the images for free. But I was actually able to download the image for free straight from Google (the download worked).
Which leads me to another question... if the images are not free, maybe it makes sense to not index them? Or maybe index a watermarked version or small thumbnail?
Hey Becky
Marcus pretty well covered things, but wanted to point you to a video Matt Cutts did a few years back about discontinued products: https://www.youtube.com/watch?v=9tz7Eexwp_A - a good watch to get an idea of how Google may look at things, and he breaks out some options depending on the type of site you have.
Hi Jenny
Yes you can redirect to URLs with anchor tags, but to Gaston's point - now that you have everything on one page, they may not rank as well as before. It does depend a little on how much overlap there was across the different products to begin with. The new page might rank well for a little while, but as Google starts to take the new consolidated page into account, you may lose ranking. The root fix would be to maintain separate pages like before, it that's possible.
Hi Allie
Yes, that sounds correct. You basically don't want to default category pages indexed, because you have custom ones now.
Also - if you don't even want users to find the default category pages at all (because it might confuse them etc) I would also make sure they are not linked to or otherwise accessible. There's no need for users to find them and for Google to be even crawling them. This might not be technically possible depending on your theme, and it won't be a huge detriment, but something to think about if you have control over it.
Just want to echo Patrick's answer. He's pretty much got the right idea. Most rankings not on page #1 will jump all around. I would only be really concerned if it dropped from page #1, down to page 4-5 etc. Also, I would also agree that you don't need to focus too much on rankings week to week - and in addition - I wouldn't focus on rankings for a single keyword too much.
To see if you have some sort of site-wide issue, I would check your Impressions in Google Console Search Analytics. See if your Impressions have dropped in the same timeframe.
I would also look to queries where you already rank page #1 - and be sure those are optimized for CTR with titles and descriptions, and then see if you can find out if users are happy with those pages - maybe with goal tracking or some sort of measurement to see if they are converting etc.
Then - really focus on "local SEO", here's some resources:
Hi Joshua
Thanks, I see the ranking drop. However, have you lost traffic? I see most positions were on the 2nd page to begin with. I see this quite often, when sites don't rank on page #1, the rankings can drop/jump a lot around pages 2-5. I know it's not fun seeing a "drop" but ultimately I would focus on your traffic as a mindset.
I ask if this is a fairly new site, because it's a super competitive niche, and I'm surprised it ranked that well to begin with. If it's a new site, often Google will rank it higher for a little while to see how users react to it, and then it may settle back to a more "normal" ranking.
I'd also check your Search Analytics report in Search Console - do you see Impression and Position drops across the board? It's tempting to analyze only a few top keywords, but I'd try to look at things as a whole too.
Hi Joshua
The noodp would not be playing a role in this. It only instructs Google to not use data from the Open Directory Project in snippets, but has no other effect at all.
In general, for link value to transfer either through 301s or canonicals, the content of the page needs to be nearly identical. See Cyrus' post for more. And canonicals are not always followed by Google, they are just a "hint", so it's unlikely you'll pass much value that way.
Hi Joshua
The settings look OK - what keywords were you ranking for that have now dropped? Also, when was the site launched?
-Dan
Hi Michael
I don't think this is a concern in terms of a penalty or anything severe. But I would say, if you'd prefer the other breadcrumbs to show up, perhaps try some breadcrumb structured data (unless you have already?) - to get the preferred ones to show up.
Hello
Ahh ok, missed that detail.
I created a quick video for you ---> http://screencast.com/t/IKkEikyr
I think this is a bit of a complicated situation which will be tough to diagnose and fix in a Q&A thread. I would suggest catalog the different settings of your site in a spreadsheet like I show in the video.
Essentially, the canonical settings are just "suggestions" for Google and not "directives" so they will ignore them if they think they have been set in error.
I would start by clearly defining the end result you want (what pages should be crawled, and what should be indexed) and work backwards from there to apply the right settings.
I would probably try to use noindex, robots.txt etc before resorting to a canonical.
FYI - to check if a page is indexed try typing site:http://perfectlinens.com/collections/all into the Google search bar, or cache:http://perfectlinens.com/collections/all into your browser.
Hi There!
That page is in fact indexed and cached for me! Can you check again? And let me know?
-Dan
Hey Andy
To answer your questions:
1. So if you're 301'ing the page, it's not really a 404 page, it's a 301 So yes, you can remove the 301 redirect, making it a true 404 page (check that it returns a 404 code using fetch as google or a tool like urivalet.com).
2. If they are in the sitemap, this won't prevent Google from removing them from the index, but it will throw an error. And not that many people care about Bing, but Bing is apparently super picky about having XML sitemaps perfect.
So yes I would just 404 them without the redirects.
Hi - right, then if the URL changes for the user, you'll want to probably use the PushState method (linked above) to convey this to Google. They likely can't see the URL change by default.
You can check by trying to crawl the site with Screaming Frog SEO Spider with the user agent set to Googlebot. Then go to "outlinks" for the page with the facet links, and see if they are listed.
Hope that helps some more! Let me know if you need further direction.
-Dan
Hi - right, I should have answered your specific situation too
When the user selects a facet - does this change the URL too? Meaning, it's supposed to be a totally different page?
Google recently updated how they prefer AJAX is handled:
https://webmasters.googleblog.com/2015/10/deprecating-our-ajax-crawling-scheme.html
They now recommend using the PushState Method - I won't pretend to know all the intricacies of how it works for implementation, but that's the best method to go with. If you need any more help let us know, and I'll have another associate jump in and take a look.
-Dan
I think you're having trouble because there's really three criteria to evaluate this on. Link authority, quality and relevance. In your situation you need to find a keyword where the current results are low in at least two of those areas. In other words - right now - you'e not going to win against high link authority and high quality. Let me break down each:
Link Authority - this you already know about. Domain Authority helps assess this. It's basically the ranking power of a site/page based upon backlinks.
Quality - this is how good the current results are at fulfilling the query with quality information. This can only be judged by a human eye - meaning - you have to look at each result as a topic expert and ask yourself - can I do better? Open times I'll see pages with high link authority but the content is low quality. This is still a hole in opportunity. Lack of quality can also be due to the fact it's not written by a true expert. Old content ranking can also be "low quality" as well.
Relevance - something can be high quality and have high domain authority, but just not be a relevance match. In other words - maybe the search is "Dominican Republic Weddings" but the result is about "Dominican Republic Events". This is a slight mismatch and also an opportunity.
Link/keyword tools can give you "link authority" but not many really measure quality or relevance that well - this is where your human eye comes in.
So... you need to find a SERP with a gap in at least two of those. Low DA and low relevance. Low relevance and low quality. You get the idea
I would suggest making a quick spreadsheet and mark each keyword with the volume etc (the usual metrics) but assign a quality and relevance score based upon your judgement.
Now - this isn't magic of course - your content has to be the best and you have to do some promotion etc - but it's definitely the approach I would take for choosing keywords.
Laurie
It should be clarified that Moz's Domain Authority, while a really solid metric, is not the metric Google has or uses. And domain authority can have a few artificial quirks. So I would not be alarmed at all.
That said - can you explain where you are seeing the two different number? I see a Page Authority of 39 for both http and https - and I see a domain authority of 27 for both http and https.
Now, even IF Moz has two different numbers for http and https, again, this is not what Google is doing, it's just an approximation.
Setting a canonical from https to http is just a band-aid and I would not recommend that approach. I would recommend having a site-wide 301 redirect so if a user lands on the https version of a URL it redirects them to the same version of that page on http. Or vice vera, whichever version you are prioritizing.
I have to respectfully disagree with Dmytro and Robert - as mentioned, Moz's metrics are not Google metrics - and the best action here is always to prioritize http or https with redirects.
Hey there
Dirk pretty much hit upon the issue, which I'll reiterate with a visual. If you enter any gibberish /blog URL (like this: http://primepay.com/blog/jglkjglkjg) in the browser it returns a 200 OK which, but it should return a 404 code --> http://screencast.com/t/cStpPB5zE
Otherwise pages that are really broken will look to crawlers like they are supposed to exist.
Escaped fragments are not recommended by Google anymore. Google now recommends Progressive Enhancement such as the History API pushState. See this article for details:
https://googlewebmastercentral.blogspot.com/2015/10/deprecating-our-ajax-crawling-scheme.html
Here's another post that you may want to check out:
https://builtvisible.com/javascript-framework-seo/
-Dan
Hi Joshua
I don't think this will cause a search engine conflict, but I'll pass this to a Moz product person to see if they can answer re: Moz On-Page Grader
-Dan (Moz Associate)
Hi Luke
Matt has the right idea. If the pages are going to "exist", you should block search engines from crawling them with the robots.txt file.
I would get your dev to help, but basically you'd find the folder or path in which you want to crawler to stop at. Maybe it's /month/ or something and you'd block that in robots.txt.
Ian covers this in his recent article about "Spider Traps". And you can also read about robots.txt on Moz or on Google.
No problem. Screaming Frog (or any crawler) won't pick it up, because it's not being linked to within the website (it's an "orphaned" page).
Google could still index them because they are in the sitemap, but it took so long because they are no actually linked to from the website.
So... if it's not supposed to be indexed at all in the first place, you can add a meta "noindex" tag to the page and remove it from the sitemap. Then you'll be all set
It was probably indexed so late because Google couldn't find it
I just crawled the whole site with Screaming Frog and that URL wasn't picked up in the crawl --> http://screencast.com/t/xzunkNR3K
But it's in your sitemap --> https://www.policygenius.com/blog/post-sitemap.xml - so this makes total sense why it took Google so long to find it
Hi Ria
99.9% certain Google 'sees' all of those as the same in terms of character/word separation. I don't think OptionA/OptionB etc will be seen all as one keyword.
However Patrick has the right idea - to question if you really need one page or if things can be broken into separate pages.
I'd also optimize for readability and clicks too
-Dan
Hello
There could be a lot of issues. It's possible the way you're handling URLs can be a factor.
What specific keywords are you seeing rankings down for? And what was the exact date?
Also, I think links could be a big factor here. There are many with exact match anchors (calling cards, phone cards etc) according to open site explorer. Some of them I checked, and they look like they are on spammy sites (like this for example thenewyorkads .com)
Have you done a link disavow and/or cleanup?