Welcome to the Q&A Forum

Hey Hassan,

I can't see what you're seeing in GSC, but it looks like your logo is showing up on Google's actual search results. In my experience, GSC is still a little buggy, so if it's working fine in the wild, you're probably safe!

Best,

Kristina

What I've usually seen with canonicals is that Google either removes the noncanonical page from its index, or it ignores your canonical and treats them as two separate pages. I haven't seen an example where a canonical lets you get the best of both worlds.

I agree with Nozzle - you can tweak your existing content to target both phrases! Google understands synonyms, so if anything, you're just creating a more all around relevant page.

Good luck!

Kristina

Hey Unique Digital,

It's a good question, and I hear what you're saying. If Google is crawling everything as a mobile crawler only, desktop rankings should basically be the same, right?

It might get there eventually, but I can say that it's definitely not there now. For one, Google is crawling mobile first, but it's also keeping an eye on desktop versions of sites. At the very least, I'm pretty sure Google is separating mobile and desktop UX metrics.

Also, on queries for my company's site (international money transfers), I can tell you that 1) Google does not show the same rankings between desktop and mobile. Mobile has more answer boxes, and I've noticed that if a site has an answer box, they're less likely to also have the #1 organic result.

So, at least for now, yes, you should track both.

Best,

Kristina

Wow, yeah, this is weird! For what it's worth, your hreflang tags seem fine, so I'd be surprised if there are canonical issues. And Google has indexed your home page.

Here's what I'd dig into: does Google think visitors want to land on your /search/ page before your homepage? I work on a site that uses a search page as a paid and organic search landing page, and it has the best conversion rate of any page on our site. Is it possible that Google thinks your homepage offers a bad user experience, and your /search/ page offers a better one, so it's picking what it thinks is best?

To see if that could be the case, I'd look into:

• What's the bounce rate of the home  vs /search/ page?
• What's the time spent on page (if you can measure that properly) of each?
• How long do the pages take to load?
• How fresh is the content on the homepage vs search? (I assume the account page is the most boring.)

If this is the case, you may want to consider letting visitors search, then requiring a registration.

Good luck! And let us know what you find out!

Kristina

Hey Sebastian,

Checking in here - does a penalty still seem to be applied? What have you done so far?

Happy to help, once I've got an idea of where you're currently at.

Best,

Kristina

I would recommend setting up each country's subdirectory as separate properties in Google Search Console. Then, go to original Search Console, and click on Search Traffic > International Targeting, click the tab Country, and identify which country you're targeting users in.

That should give GSC enough information to not flag the content as duplicate.

Good luck!

It's for different regions as well. Check out the link I shared. Google lists the reasons for hreflang. The second reason is:

"If your content has small regional variations with similar content, in a single language. For example, you might have English-language content targeted to the US, GB, and Ireland."

Very weird! Have you looked at your log files to find the IP requesting those URLs?

For any duplicate content you have between countries, use hreflang to differentiate regions. Google lays out how to do that here.

Hope this helps!

Hi there,

I agree, I don't see any actual issues with your site. Is Google still reporting a server error?

Best,

Kristina

It's best practice to redirect, but if that's not an option, the canonical route should help the problem a lot! You'll probably lose some link equity with this route, but it should clear up duplicate content issues from Google's side.

Hi Robert,

Just want to be clear: you're going from having the website lovelytraining.com and the blog lovelyblog.lovelytraining.com to having the website lovely.training and the blog lovelyblog.lovelytraining.com. Right?

If so - yup, you're going to lose that subdomain connection. There's no formal way you can tell Google that lovelyblog.lovelytraining.com is essentially part of the lovely.training site, but you can link to the blog in your header or footer so they're clearly partners.

Have you thought about migrating your blog to lovely.training/blog/? That's the best way to consolidate link equity.

Best,

Kristina

As Brett and Gaston said, this is not ideal, but also not the worst thing. You'll probably have both non-WWW and WWW versions rank for awhile, then Google will start ignoring your canonicals.

As Gaston said, it's pretty simple to change canonicals - I would recommend doing that. Otherwise, you're showing Google that your canonicals don't match your actual site, and they may start ignoring them in the future. It's worth keeping your site structure clean!

Best,

Kristina

Hi Eroc,

Yep, this is a super common issue for SEOs. My company's site has reviews load in JavaScript, and Google even crawls and indexes those as if they're separate pages.

Google wants to read everything it can, so you've got to figure out the cost/benefit of this situation. If you really do have all of the product information, I'd guess that visitors entering through page 2 of reviews wouldn't be the worst thing, especially since Google would probably choose the page with the reviews most relevant to the query.

The main issue from my perspective is wasting Google's crawl budget on your extra review pages. If that's an issue that will significantly help your site (likely if you have a lot of products), you'll want to noindex those reviews like Ikea. There's no other method that will stop Google from wasting it's crawl budget.

Hope this helps!

Kristina

Hi Jcobo,

A good way to check to see if Google understands your content is to use Google Search Console and fetch the page as Googlebot. Look at Googlebot's version of the page - does it see the content in the table?

Like Logopedia said, HTML tables are extremely SEO friendly, so I'd be surprised if Google can't read your content.

On the flip side, tables are often not particularly user friendly. Why do you have a page full of your products? How is that valuable for a visitor? If Google doesn't think a page is user friendly, it won't rank it well.

Hope this helps steer you in the right direction!

Kristina

First of all - don't stress out! You won't be penalized for broken links. Google penalizes for shady behavior, not honest mistakes.

Like Andrew said, make sure that you aren't linking to any of the pages you removed from your site, and that you've removed them from your XML sitemap.

Then you should be good!

Good luck,

Kristina

Sorry for disappearing on you - any updates here?

What a weird plugin!

Like Gaston said, Google can deal with pagination that's on separate pages, but I wouldn't be surprised if there was a period where they crawled/indexed too many pages.

Is there an option in your plugin to keep things the way they are?

Wait - so, the pages that Google says have been removed are still ranking?

You're right - this will lose you traffic! If Google says that a page is blocked, that means that Google will not list it on its results pages.

Have you checked to see if there's a noindex tag on the pages that Google says have been removed?

Hi Becky,

I'm surprised to hear that there aren't a lot of long tail searches. Is the problem that you can't make more sub-category pages, or that you can't find long tail keywords to figure out which pages to build?

Either way, here's what I recommend:

1. Use Google AdWords Keyword Planner or a 3rd party tool like Storybase to gather keywords for "lockers". I used Storybase and found: mudroom lockers, mudroom lockers with bench, school lockers, storage lockers, metal lockers, lockers for home, vintage lockers, wood lockers, cell phone lockers, wooden lockers, entry lockers, all with over 880 searches/month.
2. Export these and filter for keywords that describe the products you sell.
3. Group keywords into categories and subcategories both by logic and by search volume; for example "mudroom lockers" fit within "lockers for home" but "mudroom lockers" get more searches.
4. If you can, create category pages for every subcategory that, once grouped with similar keywords, gets at least 100 searches/month
5. Add these categories to product pages. How you do this will have to do with which other keywords you're targeting and how much control you have over the page. But you'll want both category pages AND product pages to have the right keywords to rank for the type of lockers they are.
6. Check out your competition with a product like Clearscope. It'll show you the keywords that your competitors are using for those target keywords you've identified, so you can get even more long tail searches.

Hopefully that's the sort of guidance you were looking for - let us know if you have any follow up questions!

Kristina

I would try to keep the same data, but organize it in a way that looks good on a screen the width of a mobile phone. HTML tables render the same way on a phone as they do on a desktop, they're just more likely to get smooshed.

I recommend playing around with your HTML using Google Chrome's Inspect option, and changing your device to a relatively small smartphone. Then build a table that looks good there.

If your table has too many columns, I'd recommend dividing the table into multiple tables, so you have the same data but in fewer columns for each table.

Good luck!

Kristina

Kevin's right - Google doesn't want to do this, but they're not perfect and they'll often let a site with a stronger link profile win when there are two sites with the same piece of content. Google doesn't take complaints like this, but you can go after the site itself legally.

Due to the nature of the internet, you can never completely stop scraping like this, but I would recommend repeatedly referencing your brand in each article, and linking back to your site from within your article. Then, when they scrape your article, they'll bring over your brand name and links to your site. Then you get free links and another site going on about how much great content you have.

If you want to have some fun, you may also want to try Ian Lurie's way of handling scrapers!

Canonicals can pass link juice (see Dr. Pete's comment here), but it's at Google's discretion. I've seen it go both ways.

The best way is to only link the one version you've chosen as the canonical. I'm not familiar with Magento - is that something that you can change easily? I think it'd be worth the time you'd invest in making that change.

Hey there,

The danger of having redirects to your old site, which then redirect to your new site, is that you could create long redirect strings, which pass less SEO value than one off redirects.

Ideally, you should redirect directly from what had inbound links. Say you have links to oldsite.com/awesome-article, which redirects to oldsite.com/awesome, which redirects to newsite.com/awesome. Add in a redirect directly from oldsite.com/awesome-article directly to newsite.com/awesome.

Hope this helps!

Kristina

Hi Lina,

This definitely could impact SEO. The key here is: what pages do you want to be in Google's index?

If you just want one blog homepage, the best strategy is canonicals, like Cesare said, plus telling Google that it should ignore the parameters that you use for the ESG and News blog. If you want the standard homepage, ESG, and News blog all to be  in Google's index, you'll want to canonical the homepage to it's URL with no parameters, then update the canonicals for the ESG and News blog once the filters are applied, so the canonicals include the parameters that make the ESG and News sections unique.

You can update which parameters Google pays attention to or ignores in Google Search Console, which we explained how to do in another Moz post here: https://mza.bundledseo.com/community/q/how-can-i-remove-parameters-from-the-gsc-url-blocking-tool

Good luck!

Kristina

Hi there,

This is a weird quirk of Google: sometimes, when their cached version of your page tries to execute some code and fails, the cached page looks blank. Here's the key: when you first load the cache, you'll get the gray banner at the top for about a second as the page tries to load, then everything will go white. In that second or so, if you quickly click on "Text Only Version," you'll see the text version of the cache, at least, and see that Google's reading and saving your content.

I tried this with the page you shared, and was able to get text (picture attached).

To my knowledge, this does not mean that Google isn't able to properly index your pages. But it may be worth looking closely at the text only version of the page to see if it's missing something that you load with a script.

Good luck!

Kristina

KPVGF

Whoa, this is a weird one.

I saw that you posted this on Google's forums as well, and they suggested that this might be the Japanese keyword hack. Did you look into that? If that's not it, did you try loading the URLs that are showing up on the Wayback Machine? It's possible that someone who owned this site before your client created these pages.

Either way, the answer is to double check that your 404 pages really are 404ing. If that doesn't remove them from the index fast enough, you can actually create all of those pages, with a noindex tag, add them all to a sitemap, and submit them to Google. But the 404ing is really your long term solution.

Good luck!

Hey there,

Sorry for the silence!

Okay, a few things here:

1. Yeah, you're going to run into duplicate content issues, since you're using English on all sites.

• You only need www.website.com for the US. Don't create us.website.com.
• For the UK and EU domain, localize with different currency, standard shipping, and legal information. (What's your plan, since the UK and EU use different currencies, to have them on the same site section?)
• If you have any content that's universal, like a blog post, keep that part of the site on the www.website.com section and don't duplicate it on the UK / EU site (unless you translate it).

3. If at all possible, don't use subdomains for different countries. Subdomains split your domain value (which weakens your site) and also makes it clear that you're not a truly localized site, since you're using a .com instead of the right ccTLD for the country (not bad for SEO, but not great for click through rates). Better options are:

• Get the ccTLD for each country you're operating in. This is much more expensive, but better for localizing long term.
• Put each site section on a subdirectory, like www.website.com/uk/

5. Link between the different country versions of your site, so you can be sure Googlebot will find it.

Moz has got a good article on this I recommend:https://mza.bundledseo.com/learn/seo/international-seo

I hope this helps! Let me know if you have any questions,

Kristina

Hey Srbello,

You should be fine, although if your domains are currently on different servers and are linking to each other, there's a chance that moving them to the same server will lower the value of those links.

This has been discussed more in depth here: https://mza.bundledseo.com/community/q/multiple-domains-on-1-ip-address

Now, as to your redirect question: do you want to merge all of these sites to one domain? Some of them? I'm not sure what your goal is, when you said you'd want to 301 redirect to the primary domain.

Best,

Kristina

Hey Adam,

There are a few tools out there:

http://smallseotools.com/code-to-text-ratio-checker/

http://tools.seochat.com/tools/code-to-text-ratio/

I've never used them, though: I think that the importance of code to text ratio is making sure your page doesn't take too long to load, or look like spam. I'd recommend you check out this other Moz Q&A to look into that:

https://mza.bundledseo.com/community/q/how-important-is-my-code-to-text-ratio-on-web-pages

Good luck!

Kristina

Not to toot my own horn, but I've written a whole blog post for this:

https://www.distilled.net/blog/seo/site-navigation-for-seo/

Give it a read and let me know if you have any questions!

Hi Allie,

Sounds like you've got two questions:

First, you'll need to use 301s if you have any inbound links to the pages that you're replacing.

Second, you can't use Regex in HTML. Do you have a CMS that can help you build the 301s?

Best,

Kristina

Hey Gavo,

Looks like ShedSafe.com is such a popular company, if you search for "www.shesafe.com.au", Google thinks that you just missed the "d" in "www.shedsafe.com.au". See the different messaging at the top? If you don't include "www", it asks, "Did you mean: shedsafe.com**.au**?" If you do include it, it says "Showing results for www.shedsafe.com.au."

My guess here is that when you include "www", Google thinks you're trying to use their search engine as a browser, so they're more likely to correct you. When you're not using "www", they're showing the results as you searched for them.

I think the key here is more Domain Authority, time, and branding. The more Google sees that your site is respected, they'll be less likely to recommend ShedSafe.

Good luck!

Kristina

Hey Ryzippy,

Google really doesn't want to penalize site owners for making web design decisions without taking their crawler into account, so I don't think there will be a problem. That said, I'd recommend that you upload the new design to a test group first, just in case.

This is not a risk-free solution, though, so I'd also recommend that you push back on your web developer. Is there really no way to use the same H1 for both the mobile and desktop design? This is exactly what responsive design is for, to use the same HTML elements, but different CSS based on the screen width.

Anyway, good luck, and check back in here to let other SEOs know if using multiple H1s will cause a problem!

Best,

Kristina

It's not a good or bad thing if you have 27,000 pages that Google has indexed on your site. What's important is, how many of your pages has Google indexed, how many of those are duplicates, and how many of those are unimportant pages?

To find how many of your pages Google has indexed:

1. Make sure that you have an XML sitemap that includes EVERY PAGE on your site.
2. Submit that to Google Search Console.
3. Check back in a day or so, and Google Search Console will show you how many of the pages on your XML sitemap are in their index.

If you only have 5,000 pages on your site, but Google shows that you have 27,000, then you're probably dealing with duplicate pages, or tags that are built by your site and have gotten out of control.

To find out if Google has indexed duplicate versions of your pages:

1. Use Google Analytics' Search Channel report (Acquisition All Traffic Channels Organic, then change the Dimension to Landing Page) to download a list of all pages on your site Google or Bing has sent visitors to.
2. Create a new column, stripping out portions of the URL that can change without changing the content of the page. Some common offenders:

• Parameters
• Different categories earlier on in the URL
• www vs non-www
• https vs http

4. Now, find the duplicates in the "unique" column. I like using the countif function for this.
5. For any page that's got duplicates, find a way to fix those! It varies by the problem, but the answer is generally to 301 redirect all versions to one canonical version.

To find out if you have too many tags, because they're autogenerated

1. Use an internal tool, like an XML sitemap creator, to figure out how many tags you have
2. Compare that to the number of tags actually bringing you traffic
3. Check to see if you're creating tag pages that are the same content as your category pages
4. Check to see how many tag pages you have with fewer than five products matching - those probably aren't great pages!

Going through these steps should let you know if your site is well optimized - again, it's not the total count that matters!

Good luck!

Kristina

There's no magical number of internal links that you want to strive for, though in general, you want to maximize the number of internal links to your homepage and other important pages, and link to the most important pages on your site from your homepage.

I don't want to toot my own horn too much, but I did put together a guide on SEO internal linking (which is really site navigation) a couple of years ago: https://www.distilled.net/blog/seo/site-navigation-for-seo/ It still holds up, since site structure is forever.

In case you don't want to dive into that, here's what I'd recommend for internal linking for ecommerce sites:

• Put your logo in the upper right hand corner of every page on your site, and use that to link back to the homepage. It's a really common navigational tactic, and it tells Google what your homepage is (in case it's not simply www.yoursite.com)
• Use a top navigation bar that's on every page. In the top navigation bar, link to your shopping categories.
• Use a side navigation to list subcategories when a visitor is working within a category.
• Make sure to include breadcrumbs when you're on a product page.
• Limit the number of links on any given page to below 200 (bigger sites do more, but that's because they can get away with it)

Really, that should do 99% of the SEO organization you want to do within your site.

Good luck!

Kristina

Personally, I would submit a clean sitemap ASAP. It's helpful whenever you upload it, and SEO fixes are best made as soon as possible, otherwise you're just leaving traffic on the table.

Plus, I'm skeptical that a move to https will be fast.

That said, there's no reason why you can't move your site to https without already having a clean XML sitemap for the http version. So it's really up to you.

Sorry, that's a little ambiguous! Such is SEO.

Good luck!

Kristina

Hey there,

I think you've given your client some good advice. Just to make sure we're all on the same page about how to handle duplicate content created by tracking parameters:

• Make sure to keep the XML sitemap up to date, and only include the canonical versions of URLs
• Canonical all parameter-ed URLs back to a single source, without parameters
• Mark those tracking parameters as "Doesn't affect page content (tracks usage)" in Google Search Console

The parameter issue is something I would fix ASAP, then tackle https when that comes around. At that point, you'll need to make sure ALL http pages 301 redirect to https versions of the page. I haven't worked with Omniture much, but make sure that doesn't break tracking.

Good luck!

Kristina

It's hard to say how much traffic you'll lose from the merge. Like Logan said, you'll definitely lose a bit when you first move, but long term, you'll need to look at your competition to figure out if it's better to keep the pages separate or combine them.

I don't recommend keeping pages A, B, and C if you're going to hide them from the main structure of your site. Pages get most of their Page Authority from internal links (unless they're link bait), so they won't be able to rank anyway.

That said, here's how I'd estimate the loss of traffic from the move:

1. Use Google Search Console to determine the primary keyword/s for page A, B, and C
2. Use a tool like Open Site Explorer to determine the number of links A, B, and C have. (Bonus: look at the websites linking to A, B, or C. If those are resource pages, there's a good chance their webmaster will update their links to page D, which will help with the traffic dip. If they're from news articles, you'll probably have to rely on 301s.)
3. Search for each of those top keywords and look at your competition. Does the competition closely target the term? Will page D seem as relevant to the keyword as A, B, or C did?
4. Now, look at the Page Authorities of the competition for each keyword. Will page D, which will have a combo of links from A, B, and C, blow your competition out of the water? About match it? Still be a bit behind?
5. Here's the part that's really tough: for each keyword, estimate where page D would rank, given how well it targets the keyword and how many inbound links it has.
6. Estimate the % increase or drop in traffic based on adjusted click through rate. You can find this by playing around in Google Search Console to find a time when your site ranked in a different position, or by using average click through rates, like here.
7. Once you're done, put together your estimated percent increases or drops in traffic to estimate how the new page will perform. (I recommend you look at a percent change because adding up totals only for top keywords won't take long tail keywords into account, and you'll almost definitely come up with a much lower count than you're currently getting.)

Not the easiest process in the world, and your estimate will almost definitely be wrong, since you make a lot of assumptions along the way. But it should give you an idea of whether you'll eventually gain or lose traffic from the move, once that initial Googlebot confusion wears off.

Hope this makes sense! Let me know if you have any questions!

Kristina

Hey John,

I'm going to try to go through all of your answers, one by one:

1. Can it hurt us to blog at a high volume (4 blogs per day) in an effort to include all of our keywords and attach them to state and city specific keywords (ie. "keyword one" with "keyword one city" and "keyword one different city")?

Yes, if the articles end up too short or too similar. 4 posts per day isn't necessarily a bad pace, it's just the quality of the writing that can get you into trouble.

I don't know what your service is, but, do you need to have a variation for every city? I work for a business that provides city-specific services, but we write blog posts that are relevant for the whole nation, then have pages describing what we offer for each city. We do have some location-specific content, but most of our blog posts are not.

Here's the way I'd figure it out: when people are looking for a given keyword, do they usually use the city name? When you search for the keyword without the city name, does Google return map results? If the answers are no, you can write one article that isn't location specific, and Google will rank it in all locations.

**2. Is it more valuable to blog only a couple of times per month with deeper content, or more times per month with thinner connect but more keyword involvement? **

Totally depends on your industry. And company. Deeper content can rank for more keywords and has a better chance of getting inbound links, but it can limit the keywords you target. You really need to think about ... (next question) ...

3. Our customers are forced to use our type of product by the government. We are one of the vendors that provide this service. Because of this our customers may not care at all about anything we would blog about. Do we blog for them, or do we blog for the keyword and try and reach partners and others who would read the content and hope that it also ranks us high when our potential customers search?

You should write content that appeals to whoever makes the decision to use your product.

How often does the government choose whether or not to provide your service? If there's any chance at all that you could eventually lose your contract, write content that appeals to the people who choose vendors. How can you make their decision to work with you easier? Write articles about how your service solves their problem easily. Interview some government officials. Make it clear that you understand them. Also, write about whatever your company is knowledgeable in, so it's clear that you're the thought leader in your space.

If it's really unlikely that you'll lose your government contract, or the government track doesn't work, I'd just focus on the last part: what knowledge does your company have that others don't? OKCupid, for example, got great links through their blog, which showed in depth analysis of how we date (making it a great purchase for Match.com, who, alas, stopped blogging). Pinterest developers have a Tumblr called Pingineering that talks about how their engineering team tackles big data issues (granted, this isn't on their site, but they're not struggling for links or rankings).

4. Is there an advantage/disadvantage or does it matter if we have multiple blog authors?

Nope. Go for it.

Hope this helps! I wasn't able to be very specific without knowing your specific industry, but this should point you in the right direction.

Let me know if you have any questions!

Kristina

Gotcha - I'd actually start with Google Search Console, then.

Good luck!

Are you looking for landing pages on your own site? If so, Google Search Console is your best bet. Go to Search Analytics, click on the query you're interested in, and Google will show you all of your site's pages that have ranked for that term within the last 90 days, along with their impressions and clicks.

Moz and most ranking trackers will also show this. You can go to your campaign in your Moz Pro account and choose Analyze Keyword under rankings. Choose a keyword and scroll to the bottom to see a list of what ranks for the first 5 pages on Google.

Does this help? Do you know which keywords you think you might be ranking for multiple times, or are you trying to find those keywords?

Best,

Kristina

That's true, they append parameters tracking where you came from, which looks like it can affect the navigation you're seeing on the left. They're making sure that Google doesn't get confused by using a canonical on their pages, like Mike, Eric and I have recommended.

Hi there,

Like Mike and Eric have said, I'd recommend using a canonical tag on the men's and women's pages to the version of the page that shows both genders' shoes/clothing.

That said, I just want to make sure this is the best path for your site. If it makes more sense for your site to point people to shoes and clothing by gender, shouldn't that be what you show in Google's search results? I'm a woman, and generally search for "women's shoes" since otherwise I often end up on pages that show men's options.

Let us know if these solutions work!

Kristina

Agree with Oleg above; that's not a good solution if this lasts years, though.

Is there any chance you could serve the old site using the current URLs?

Google uses URLs as unique identifiers, which means that if you redirect to URLs that are only for the old site, it'll think you've created new pages. New pages mean a month or so of low rankings, before your page has a proven click through rate and gets some links.

If you use your current (and future) URLs to load the old site, though, Google won't see that as a site overhaul, it'll see it as content modifications on a pre-existing page.

Hope this helps! If 302s or using the same URL isn't an option, let us know so we can brainstorm more solutions with you.

Best,

Kristina

Can you clarify what you're concerned about for 301 redirects in terms of link juice?

301 redirects don't carry as much link juice as a direct link, but it doesn't impact correct links, just the links that, otherwise, wouldn't get link juice to your end destination at all. (Though, if your canonical is working correctly, it'll pass the same amount of link juice as a 301 redirect.)

Dr. Pete goes into this a bit more over here: https://mza.bundledseo.com/community/q/do-canonical-tags-pass-all-of-the-link-juice-onto-the-url-they-point-to

Hmm, that doesn't seem good. It's hard to say whether this is causing the decline in your rankings, but either way, you want to make sure that you're not splitting your link equity between your / and .shtml pages. Here's what I'd do:

1. If you can, 301 redirect / pages to .shtml pages. Obviously, it'd be easier if the canonical worked, but it sounds like it doesn't.
2. Use ScreamingFrog or DeepCrawl to look through internal pages on your site to see if you're ever linking to the / version of pages rather than the .shtml pages. When Google chooses a different version of a URL over the canonical one, it's often because that's how it sees internal links pointing to the page. Make sure that you only have links to the .shtml version of the page.
3. Use a tool like Moz or Ahrefs to find all internal links to your site. For any links that you built or have a partnership with the owners, make sure that they're linking to the .shtml version of the page. I could especially see your ad partners using / because it's a cleaner before parameters than .shtml.

After that, wait and see if Google fixes the problem.

Also worth noting: have you thought about changing your default to /? That's more common today, so you're probably getting a lot of external links with / instead of .shtml, and you'll never be able to fix that problem. If that's a possible solution, you may want to explore it.

Good luck!

Kristina