Lately I have noticed Google indexing many files on the site without the .html extension
-
Hello,
Our site, while we convert, remains in HTML 4.0.
Fle names such as http://www.sample.com/samples/index.shtml are being picked up in the SERPS as http://www.sample.com/samples/ even when I use the "rel="canonical" tag and specify the full file name therein as recommended. The link to the truncated URL (http://www.sample.com/samples/) results in what MOZ shows as fewer incoming links than the full file name is shown as having incoming.
I am not sure if this is causing a loss in placement (the MOZ stats are showing a decline of late), which I have seen recently (of course, I am aware of other possible reasons, such as not being in HTML5 yet).
Any help with this would be great.
Thank you in advance
-
Can you clarify what you're concerned about for 301 redirects in terms of link juice?
301 redirects don't carry as much link juice as a direct link, but it doesn't impact correct links, just the links that, otherwise, wouldn't get link juice to your end destination at all. (Though, if your canonical is working correctly, it'll pass the same amount of link juice as a 301 redirect.)
Dr. Pete goes into this a bit more over here: https://mza.bundledseo.com/community/q/do-canonical-tags-pass-all-of-the-link-juice-onto-the-url-they-point-to
-
Many thanks for taking the time to respond Kristina.
-
I don't like to do redirects, as so many have warned of the consequences in terms of link juice
-
No, I don't link to the pages in question using "/" rather than the ".shtml" version of the page indexed.
-
A few external sources use the "/" version (recent linkers) I have found, but they likely only did so as they saw it displayed as such in the SERPs previously. No commercial or other affiliate sites do.
The reason I was really confused is that some pages are indexed using the "/", while others are not -- with no apparent reason I could locate. The "/" version for pages still remains on the first page for keywords, even with far less domain authorities and pages linking to them (for now!). We will be moving to another platform with a different default extension, so I wonder how that will be handled. Endless mysteries.
Thank you again for your time and suggestions,
Greg
-
-
Hmm, that doesn't seem good. It's hard to say whether this is causing the decline in your rankings, but either way, you want to make sure that you're not splitting your link equity between your / and .shtml pages. Here's what I'd do:
- If you can, 301 redirect / pages to .shtml pages. Obviously, it'd be easier if the canonical worked, but it sounds like it doesn't.
- Use ScreamingFrog or DeepCrawl to look through internal pages on your site to see if you're ever linking to the / version of pages rather than the .shtml pages. When Google chooses a different version of a URL over the canonical one, it's often because that's how it sees internal links pointing to the page. Make sure that you only have links to the .shtml version of the page.
- Use a tool like Moz or Ahrefs to find all internal links to your site. For any links that you built or have a partnership with the owners, make sure that they're linking to the .shtml version of the page. I could especially see your ad partners using / because it's a cleaner before parameters than .shtml.
After that, wait and see if Google fixes the problem.
Also worth noting: have you thought about changing your default to /? That's more common today, so you're probably getting a lot of external links with / instead of .shtml, and you'll never be able to fix that problem. If that's a possible solution, you may want to explore it.
Good luck!
Kristina
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Google Panda question Category Pages on e-commerce site
Dear Mates, Could you check this category page of our e-commerce site: http://tinyurl.com/zqjalng and give me your opinion about, this is a Panda safe page or not? Actually I have this as NOINDEX preventing any Panda hit, but I'm in doubt. My Question is "Can I index this page again in peace?" Thank you Clay
Intermediate & Advanced SEO | | ClayRey0 -
Very strange HTML docs - what should I do with them through site migration?
I've just been looking at a website and it includes numerous web pages with addresses like this. I click on the URL and it takes me to a fully functional web page (not an image) and when I run it through Screaming Frog this comes up as an HTML page. The site has around 150 unique pages and over 450 pages like this one - how should I deal with these pages during an SEO migration (only a few are backlinked to)? I look forward to reading your thoughts. http://www.[companyname].co.uk/property/caravan-sleeps-4/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/images/cottageTypes/blank.png
Intermediate & Advanced SEO | | McTaggart0 -
Is it possible to rank in google mexico when you don't have a local site?
Hello, someone is asking me why we don't rank in google mexico search engine. I mentioned we don't have a google mexico site, but have a USA site, so we may rank, but not as well as if we had the mexico site. IS there anyway to improve rankings or tips? THanks! Laura Robinson
Intermediate & Advanced SEO | | lauramrobinson321 -
Wrong country sites being shown in google
Hi, I am having some issues with country targeting of our sites. Just to give a brief background of our setup and web domains We use magento and have 7 connected ecommerce sites on that magento installation 1.www.tidy-books.co.uk (UK) - main site 2. www.tidy-books.com (US) - variations in copy but basically a duplicate of UK 3.www.tidy-books.it (Italy) - fully translated by a native speaker - its' own country based social medias and content regularly updated/created 4.www.tidy-books.fr (France) - fully translated by a native speaker - its' own country based social medias and content regularly updated/created 5.www.tidy-books.de (Germany) - fully translated by a native speaker - uits' own country based social medias and content regularly updated/created 6.www.tidy-books.com.au (Australia) - duplicate of UK 7.www.tidy-books.eu (rest of Europe) - duplicate of UK I’ve added the country and language href tags to all sites. We use cross domain canonical URLS I’ve targeted in the international targeting in Google webmaster the correct country where appropriate So we are getting number issues which are driving me crazy trying to work out why The major one is for example If you search with an Italian IP in google.it for our brand name Tidy Books the .com site is shown first then .co.uk and then all other sites followed on page 3 the correct site www.tidy-books.it The Italian site is most extreme example but the French and German site still appear below the .com site. This surely shouldn’t be the case? Again this problem happens with the co.uk and .com sites with when searching google.co.uk for our keywords the .com often comes up before the .co.uk so it seems we have are sites competing against each other which again can’t be right or good. The next problem lies in the errors we are getting on google webmaster on all sites is having no return tags in the international targeting section. Any advice or help would be very much appreciated. I’ve added some screen shots to help illustrate and happy to provide extra details. Thanks UK%20hreflang%20errors.png de%20search.png fr%20search.png it%20search.png
Intermediate & Advanced SEO | | tidybooks1 -
Spammy sites that link to a site
Hello, What is the best and quickest way to identify spammy sites that link to a website, and then remove them ( google disavow?) Thank you dear Moz, community - I appreciate your help 🙂 Sincerely, Vijay
Intermediate & Advanced SEO | | vijayvasu0 -
Big discrepancies between pages in Google's index and pages in sitemap
Hi, I'm noticing a huge difference in the number of pages in Googles index (using 'site:' search) versus the number of pages indexed by Google in Webmaster tools. (ie 20,600 in 'site:' search vs 5,100 submitted via the dynamic sitemap.) Anyone know possible causes for this and how i can fix? It's an ecommerce site but i can't see any issues with duplicate content - they employ a very good canonical tag strategy. Could it be that Google has decided to ignore the canonical tag? Any help appreciated, Karen
Intermediate & Advanced SEO | | Digirank0 -
Index or not index Categories
We are using Yoast Seo plugin. On the main menu we have only categories which has consist of posts and one page. We have category with villas, category with villa hotels etc. Initially we set to index and include in the sitemap posts and excluded categories, but I guess it was not correct. Would be a better way to index and include categories in the sitemap and exclude the posts in order to avoid the duplicate? It somehow does not make sense for me, If the posts are excluded and the categories included, will not then be the categories empty for google? I guess I will get crazy of this. Somebody has perhaps more experiences with this?
Intermediate & Advanced SEO | | Rebeca10 -
HTML 5 sites, segmentation and Meta data?
Hello Mozers, I am currently building an HTML 5 site. I've run into a couple of issues. While implmenting segmentation in each of my mian menu iten, I am able to pluggin Meta data only for one segement (or the page). I am unable to inser Meta data for each of the segments. For example: I have (main menu) Services ----> Submenu (teaching, upgrading, Dancing) I can implement meta data for the Services but not for teaching, upgrading and Dancing as they are segment in the same page. Whats the best logic to get around this
Intermediate & Advanced SEO | | waspmobile0