Hi,

just an idea I found somewhere else.

It's about the old dublicate content issue....what do you think:

If I change my robots.txt in that way, that all spiders shouldn't spider urls with -2.html, -3.html, -4.html, what do you think? The dublicate content problem should be solved. Does anybody know, what the robots.txt should look like? Help needed.

Another idea, is to use JoomSef edit url for that problem. O.k, more work. Have a look:

Meta Content-Language: What is it for? EN for English or what
Meta Robots: How to handle this? YES or No or what?
Meta Googlebot: Same, is it boolean exp. Yes or No.

I hope that I can get answers to my questions and the solving by robots.txt dublicate content problem.

Tia

Sorry, to open that thread again.

Is nobody here, who is able to verify that idea?

Hello,

my opinion is, that the robots.txt solution would be quite complicated and would not give the results you would like. There would be a risk that some of the multipage articles/galleries/forum threads/etc. would not be spidered correctly, which is probably something you do not want. Also the links in pages would still stay the same (duplicated).

If you have problem with duplicates, I would recommend trying the new option in JoomSEF 2.0 - \"Ignore multiple sources (Itemids)\".

It works so, that only 1 Itemid (by default the first found) is used for the content item. So there should be no duplicates. If you are not satistifed with the default id that was used (and thus e.g. you do not see the menu you want to have in page), you may change it in URL editor.

Just note, you have to clear all the already created duplicates that already ARE in database. (there are not removed automatically).

We are using this configuration now on this pages, and it has reduced almost all duplicates. Just in cases, when some link is created wrongly (there is something extra there should not be or so), we have to adjust the link manually a bit, but in 95% of cases, this works automatically.

When you hit a duplicate, always go to URL editor and try to compare where the original URLs are different. If just in Itemid, you may just delete the duplicated (if you have \"Ignore multiple sources (Itemids)\" on, it will not be created again). If it differst in something else, you need to consider why and think about a solution -- either renaming the link, or adjust the link source to be the same as the other one and thus not creating the duplicity.

Hopefully this will help you a bit against fighting the duplicates.

Best regards,
michal