Menu Content

Support

> Forums, FAQs & Paid Support
Welcome, Guest
Username Password: Remember me

Why are my non SEF urls being crawled?
(1 viewing) (1) Guest
Support forum for users using free edition of JoomSEF 4 (Joomla 1.6/1.7/2.5+ compatible). These forums are mainly for mutual help between users.

Please note that due to our capacity limitations, we do not monitor these forums regularly.

TOPIC: Why are my non SEF urls being crawled?

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30854

  • dajo
  • OFFLINE
  • Posts: 5069
And when you open the non-SEF URL, is it redirected to the SEF one automatically?

The URLs you posted are pagination URLs, so the problem is that all the pages in list have the same title. Our K2 extension currently doesn't support adding page number to page title, but we already track it as a requested feature.

I don't know why Google sees your non-SEF URLs. Have you checked your website's source code if there are non-SEF URLs present? Or have tried fetching it as google bot in Webmasters Tools?
ARTIO Support Team

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30861

Thanx for the reply

No when I open the non-SEF URL it remains non-SEF, it's not being redirected on any of the non-SEF URLS

Good news about the pagination issues support, will the update be free of charge?

I know the code fairly well, there is no manuually added nonSEF URLs, i think they are mostly as you say pagination type URLS, same for PRINT and SHARE URLS - how do you deal with those through JoomSEF?

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30951

Any ideas here? I now have over 380 duplicate pages, titles and tags.

Here is an example
www.mydomain.co.za/?option=com_k2&vi...id=83&Itemid=131
www.mydomain.co.za/?option=com_k2&view=item&id=83
mydomain.co.za/?option=com_k2&view=item&id=83

This is really bothering me now because it is affecting my SEO and I can't do anything about it

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30963

Hi Gazza, Dajo

I've been wrangling with this issue for months. When I managed to configure JoomSEF and the JoomSEF Virtuemart Extension to remove the NON SEF ?limit=5&limitstart=0 part of the URL pagination fails to move to the next page and product ids get mixed up. I.e. click one product browse listing takes you to an entirely different product.

I had over 10,000 duplicate links due to the issue. I did manage to create a temporary workaround. I exported 30,000 links out of the database using an external app rather than JoomSEF. Once the rows of urls were out of the database I set them all to locked. Then I filtered the columns to show columns that had the NON SEF parts of the URL in them. With them filtered I Canonicalised the links to their route paths and set them not to be included in the sitemap index. I set JoomSEF to not create any new SEF URLS and exported them all back into the database.

Then I generated a new sitemap index before submitting to Google Webmaster Tools. After, I switched JoomSEF to start creating new SEF URLS but set the default state for new URLs as not indexed. I narrowed the 30,000 down to 1,700 of which Google has now indexed 800.

New URL's are still being created with this same problem but I'm keeping on top of them.

Having a solution to this problem would really solve an immense load of headaches.
Mark
ID 23447
Joomla Version 1.5.23
JoomSEF 3.9.5
Virtuemart 1.1.7
Virtuemart Extension Version 2.0.34

AND

JoomSEF Version 4.2.9
Virtuemart Cart 2.0.8e

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30981

  • dajo
  • OFFLINE
  • Posts: 5069
gazza777:
That's strange that the redirection is not working. Could you post an example of some SEF URL from JoomSEF's URLs Manager and its corresponding non-SEF URL for which the redirection doesn't work?

As long as you have the update period active our updates are free of charge for you (one year from purchasing the extension).

The print and share URLs are usually not converted to SEF because they contain the tmpl=component variable. You can enable SEF for those by enabling the "SEF URLs that use component template" option in JoomSEF's Advanced configuration.

I see 2 problems in the URLs you posted:
1. The second URL is missing the Itemid variable. This is usually caused either by the K2 component not adding it correctly or there are some manually created URLs without the correct Itemid set. The solution is not easy, it depends on your website's structure. If all your K2 items use the same Itemid, you can set the K2 extension to Override the Itemid and always use the one you set.
2. The third URL does not start with www. This is really bad for SEO as it causes duplicate content. You can either use "www and non-www domain handling" option in JoomSEF's Configuration or create a custom RewriteRule in your .htaccess so the URLs are always redirected to www domain.

Please note that those are really general advices. I would recommend you to purchase our paid support time and we would check your website directly to find the problems.
ARTIO Support Team

Re: Why are my non SEF urls being crawled? 12 years, 5 months ago #30982

  • dajo
  • OFFLINE
  • Posts: 5069
markkenyon:
As I understand you have a different problem, that there are many duplicate SEF URLs generated by JoomSEF on your website? For gazza777 the main problem is that Google indexes his non-SEF URLs instead of SEF ones.
Please start a new thread with description of your problem and some examples of the SEF and corresponding non-SEF URLs as generated by JoomSEF. Also don't forget to state the versions of your software (mainly Joomla, JoomSEF, VirtueMart).
ARTIO Support Team
User Login Empty