When I try to have Artio show all the duplicate URLS it thinks for a long time then gives me a 500 server error. The URL database is very large, and many of them have been locked over the years to prevent broken links as I have switched out various components. I know that a large number of the URLS are duplicates and want to get rid of them but can't seem to do it.

On the same topic, when and if I finally do get the system to show me the duplicates how do I know which one(s) to keep?

-MaestroC

Hi,
This may happen if the database is large. Try increasing your server's memory or script execution time limit.

If your current URLs are working correctly for some time, you can usually keep the URLs that have the highest hits count. But if you want to be sure that you keep the right URLs, you can disable SEF for a while, check which non-SEF URLs are used and keep the SEF URLs that correspond to them.

Of course having a backup of your website is recommended prior to deleting any URLs.

Okay, I have gone into my php.ini files and set memory limits to 512MB and the max execution time to 240. I confirmed that the settings have taken effect by looking at phpinfo.

Sadly, even with these higher limits it still gives me a 500 error when trying to find duplicities. I believe that these settings are at the top limit of what my host will allow.

I hoped that if I tried to limit the scope of my search it might work. I set it to only look at Articles and at only those that have more than 1000 hits. That search came up fine, but when I then add the Find Duplicities to it it times out again.

Any more ideas? Is there a way to cut down on the amount of URLS that it will sort through?

-MaestroC

Some more information, I think I can do this process from phpmyadmin with no problems. In fact it comes up almost instantly if I use the query-

SELECT sefurl, COUNT(*) FROM jos_sefurls
Group BY sefurl HAVING COUNT(*) > 1

Is this the same query that is being used to find duplicates in JoomSEF?

I am not an SQL guru, and patched this together from posts on other sites. Please tell me if it is wrong or if there is some way to use this to accomplish what I cannot accomplish through the JoomSEF control panel.

-MaestroC

Your SQL displays only one record for each duplicated SEF URL, so we basically use the following nested SQL, that selects them all:

Please let me know if it works for you in phpmyadmin, how long it takes and how many URLs it returns.

I ran it as you requested. This is what got returned.

(808 total, Query took 26.0599 sec)

I assume I can go in there and drop rows the same way I would do it from inside the back end of the site?

One other thing, since I am not fluent in SQL, is there a way to take that same result and only show the sefurl, origurl, and hit count columns? Otherwise it shows all the columns and truncates the full sefurls and origurls.