Welcome to the Q&A Forum

Great, thanks.

With those 2 recommendations I have more than enough for the next crawler. Thank you both!

Hi, thanks for answering

Well, it looks doable. Will try t do it on next programmed crawler, trying to minimize exposed time.

Hw, your idea seems very compatible with my first approach, maybe I could also allow rogerbot through htaccess, limiting others and only for that day remove the security user/password restriction (from joomla) and leave only the htaccess limitation. (I know maybe I'm a bit paranoid  just want to be sure to minimize any collateral effect...)

*Maybe could be a good feature for Moz to be able to access restricted sites...

I'm in the middle of site development and wanted to start crawling my site with Rogerbot, but avoid googlebot or similar to crawl it.

Actually mi site is protected with login (basic Joomla offline site, user and password required) so I thought that a good solution would be to remove that limitation and use .htaccess to protect with password for all users, except Rogerbot.

Reading here and there, it seems that practice is not very recommended as it could lead to security holes - any other user could see allowed agents and emulate them. Ok, maybe it's necessary to be a hacker/cracker to get that info - or experienced developer - but was not able to get a clear information how to proceed in a secure way.

The other solution was to continue using Joomla's access limitation for all, again, except Rogerbot. Still not sure how possible would that be.

Mostly, my question is, how do you work on your site before wanting to be indexed from Google or similar, independently if you use or not some CMS? Is there some other way to perform it?
I would love to have my site ready and crawled before launching it and avoid fixing issues afterwards...

Thanks in advance.

Yes... analyzing data would be the best way to start... will try to bring more information for tomorrow and paste it here so we could speak with real numbers.

Thanks!

What happens with sites that have the complete list of products in the home page and in the "Products" section.

Would like to know your opinion as web developers and users and like SEO professionals.

I'm in duality like web developer, somehow my subconscience is telling me that is stupid to have it on both places... And the home page is getting nice results and comments from the users, so, would consider quitting the "Products" teaser view. But of course, eliminate the "Products" section is somehow... un natural....

And what happens with SEO, I read somewhere that it's not very good to have all the "rank" spread in so many links in the home page... I don't even understand it completely or neven sure they said rank or juice..., but it's something that is concerning me...

We are talking about 20 products, that can grow. I'm looking to have a better SEO, but also the UX is very important, even more than SEO (I know good UX is better SEO).

Thanks in advance to any contribution.

I started with one very nice tutorial in https://tutsplus.com/, it's a pity that can't find it now because I have to continue soon with it and it was very useful and free...

Ok, profiles made, now to wait to see some results.

Do you think that the profile that I already have could be ok as raw, there are no special changes, and for the working one use one of new¿?

I'm more interested in new data than the past one, but not sure if in some moment not having the past data could make me loose some calcs....

Hi Paul, make a lot of sense. Thank you for this guidance.

I'm going to do it in few hours, will post any question if I have.

Thanks again!!

Hi Sanket,

confirmed was the filter.... Visits started counting already.... Will keep it all in mind for any possible future need.

Thanks Sanket

Hi Sanket

• I have checked javascript code and it's GA's default for root domain and subdomains.
• What do you mean by republishing the site?
• I`m not able to find the option to Update Settings...

BUT, looking for options you wrote, I found that I applied one filter in the profile.... For one subdomain... And, it's very possible that was not filtering no data.....

I think that you can add it to your list, special for newbies Hw, not sure yet if really was that, but dates are same....

Thanks!