I've imported 6K categories and 16K products to magento using a custom import profile.
When I'm trying to reindex everything works except for 'Catalog URL Rewrites' that keep showing PROCESSING but never completes.
log and exceptions files don't show anything.
Is there something I can do to make the index work?
Can I just ignore this index and not use it? (I don't know what it does).
Thanks
You can sort of ignore this index if you do not care about pretty, search engine "friendly" URL's. The products will still appear in the catalog but will have their default "Zend Framework" type URL (/catalog/product/view/id/123)
My store has 150,000+ SKUs in two store views. On my development environment it took almost two weeks to complete.
The best way to determine how fast it is running is to look at the core_url_rewrite database table. It appears that the process starts at product ID 1 in store ID 1 and looks to make sure all of its URLs exist and creates the ones that do not yet exist.
For me the reason it took so long was that it had to go through my entire catalog twice to make all the URLs. One thing I did notice was that this process creates a ton of URLs that are completely unnecessary. In our store easily 90% of our products belong to a configurable product so their visibility is set to "Not Visible Individually" so they never would need to have a URL. This index creates those URLs anyways.
Hopefully this will shed some light on to how this URL works. I would keep an eye on that database table so you sorta know how far the process has come. I would also seriously consider running the reindex process for the command line. I have included a link that explains how to do this (disclosure: it's from my blog)
http://overlycaffeinated.com/2011/02/when-reindexing-in-magento-fails-use-the-command-line/
My own attempt of reindexing URL rewrites was with 10,000 products and it took me like an Hour and Half to finish up.
this link will help you do it from Command Line , which is preferable than doing it from the same Magento admin panel.
http://www.yireo.com/tutorials/magento/magento-administration/340-magento-14-cronjobs
Make sure you read the part about skipping some basic configuration when facing memory limits , this is the one i usually use.
You need catalog URL rewrites to make your catalog work properly. How long are you letting it run for? There's a really substantial amount of data to be written to it...
If you cannot make it work programmatically, it may be possible to shove the data into the table manually and force the index to feel refreshed? Caveat emptor, I have not tried this.
I was encountering the same problem, the server that I am running this on is a FreeBSD server. With the help of the sample of the above code I was able to get the issue to resolve buy running the following command in the shell.
/usr/local/bin/php /usr/local/www/magento/shell/indexer.php --reindex catalog_url
It runs very quickly and resolved my issue. I then created a cron for this to run every 6 hours on my server.
You can reindex using command line it will reduce 75% of load and your magento admin panel working as normal while reindexing.
Related
I've posted this question on magento.stackexchange.com but got no response due to lower traffic there than here... So here it is...
I'm working on a website for a client and right before publishing the
website, I noticed that all the products went out of stock itself. So
I put them all back in stock and everything looked fine, but when I
checked back after a few hours all the products were back out of
stock.
I tried a few times to re-enter the stock quantities, re-index and
clear the cache, but I'm still having this weird issue - everything
looks fine until like 2-3 hours later everything goes out of stock.
Any suggestion what could be causing this issue?
Thanks in advance!
PS. I didn't share the code because I would have to share the code of the entire website...
Any suggestion what could be causing this issue?
Any imaginable 3rd party application or module that has access to cataloginventory model or its tables in database.
Suggestion how to debug this:
put this snippet into inventory observer
app/code/core/Mage/CatalogInventory/Model/Observer.php
somewhere at the begin of saveInventoryData method. (Or create local module and observe a save event)
Mage::log(Varien_Debug::backtrace(true, true), null, 'inventory.log');
after you reproduce the behaviour just check the log file under <doc root>/var/log/inventory.log
I am running Magento 1.8.1 with embeddedERP extension.
Not sure if extension is to blame for the behaviour, i don't think so as it seemed to work fine for a while after I installed the extension. The issue appears to relate to indexing.
I reindexed the site in response to a magento index aler. When I try to view products in the backend the page is blank but only from from the horizontal line below the Add New product button, there is no list of products but also no products table at all, rather than the products table with no rows one would expect.
I have tried deleting cache and session folders.
I've tried disabling cache.
I enabled logging and there isn't an exceptions.log file so presuming this means no exceptions happening.
I've logged in and out.
On one occasion when i tried creating a new product after saving it it displayed the products (new one and existing) but now that doesn't work either.
Previously, deleting session and cache and logging back in appeared to work but now doesn't.
Can anyone suggest how I can go about working out what is wrong. My magento debugging skills probably need some enhancing.
Thanks,
Came back to this and the site was then displaying an error:
Warning: Unknown: Unable to allocate memory for pool in [filepath]/Autoload.php
This thread:
http://www.magentocommerce.com/boards%20/viewthread/216180/#t462005
highlighted this is as an apc issue.
Altering the memory size to 256M as suggested solved this.
Hope this helps if anyone else gets this, still don't know why this issue showed a blank screen to start with, perhaps an APC caching issue?
So I've been asked by a client for an online store in which he need each night to delete all the products and import again aprox. 80.000 products.
Of course I thought about Magento. But as we all know the importing process is a pain in the ass.
The best option is to use MAGMI of course and the import doesn't take long and it also has a nice utility that quickly empties the database and resets the product ID counter.
The problem is with the bloody reindex operation.
MAGMI has a plugin called Magento Reindexer which doesn't work in my case. I guess it needs to ensure security configuration enable "shell_exec()" calls from php. I dunno how to do that.
To my only choice is to reindex using the browser from the admin.
Of course it's not the best solution specially since it times out and it doesn't work.
So I need to give the client two options:
to create a cron job that would use MAGMI's Clear Catalog utility to empty the database and then use MAGMI to import the same CSV file every night. So all the client would have to do is reupload the CSV file in the same location every night.
Give the client the possibility to manually make the import when he wants using MAGMI and of course empty the database using the same Clear Catalog utility from the MAGMI interface.
Either way I have to find a solution for the reindexing problem.
Does anyone have a similar experience or maybe a solution?
Thanks in advance!
You do not need to empty the database with each import. Magmi has the ability to update products on the fly without deleting them first.
If your concern is removing products that are no longer in the CSV, you can use a plugin I wrote and posted on StackOverflow here: https://stackoverflow.com/a/18447266/1332068
The indexer issue has also been asked about and answered on this site as well.
https://stackoverflow.com/a/19384326/1332068
I urge you to search thoroughly before asking questions as there are several solutions already available for Magmi on this site.
This is really strange. Some of the products from the catalog gives 404 error, even if targeted directly.
The product can be seen in admin and is enabled with stock set. I checked the URL rewrite section but couldn't find anything. Also, tried reindexing everything, but couldn't resolve the issue. Even flushed the cache still nothing.
After searching the issue, I found a solution saying running an SQL command. I did that but it said the entry already exists.
Even when I search the product in catalog, nothing returns. The item is there but its just redundant and not showing.
You can check that the product is...
Assigned to a website
Inventory level is above 0 and status is in stock
Enabled
Visible in catalog/search
I have highlighted point 1 as it does tend to catch people out in a multi store environment.
Also, you may need to reindex/clear cache etc.
Finally, double check you are using the correct suffix if relevant i.e. you may have to add .html to the url.
Make sure that your product's status is 'Enabled'.
Re-indexing may help you to resolved the issue.
Go to admin panel System > Index Management. Here select all and do re-indexing. After doing re-indexing try to clear your magento cache and check.
We have noticed that the shopping cart banners seem to frequently loose the link to the related promotion without anyone making this change within admin. Is there a setting that only allows a certain number of customers to see this banner and then it would be removed? I can’t think of another reason it might just drop the link. It isn’t dropping all the links, and it has usually been the lower value ones.
You need to setup cron.php to be accessed in a cronjob. Verify by hitting cron.php manually and see if that resolves your issue.