when the new products get uploaded they are not visible on the front end. They told me we need to re-index every time. However re-index gets stuck.Please look into this issue for me and let me know if we can have it reindex automatically.
Reindex mode is Update on save and currently showing processing in index management
If it shows as processing then it's probably running. It may take a long time depending on how many products you have. The core url rewrite index is the most intensive. The reason you need to index is most likely because you have enabled the Use Catalog Flat Tables options under system configuration which is good for performance and speed of the site. Downside is you need to re-index these flat tables when you add new items. You can have the indexes run automatically by configuring cron to call magentos index process.
e.g.
* * * * * /usr/bin/php /var/www/magentosite/cron.php
5 0 * * * /usr/bin/php /var/www/magentosite/shell/indexer.php --reindex all
If it is stuck and you are sure it's not running, then you can change the status of the index process back to pending manually in the database by editing the table index_process. Re-run it manually from the admin and see if it fails. The index most likely causing your products not to show up is Flat Catalog Product Indexes so run this one first.
Yes, When you insert new product so re-indexing is require and also disable you magento cache.
and also flush your magento cache.
Related
I have a Magento site in which the cross-selling products do not seem to be appearing.
After looking on Stack and Google it seems that 'reindexing the data' has solved this issue for a lot of individuals.
My question is, are there any risks associated with performing this task? Or is it a relatively straight forward procedure?
Indexing is a fundamental part of Magento and will not effect your site in a negative way.
Magento uses a complex EAV (entity-attribute-value) database structure that can sometimes require heavy database queries to retrieve simple results. Because of this, the Magento developers have implemeted Index tables that query all of this data, and store it into a single table structure. This allows Magento to quickly query the single Index table, rather than making complex joins across multiple tables.
With that being said, Reindexing does not alter your existing data. It simply queries your existing data and copies it to it's own tables.
To reindex your site, you can simple go to System > Index Management, check off all the indexes that you wish to reindex, then submit.
If you have a large set of products, I recommend reindexing your site from the shell command line.
Login to your site using an SSH program (such as Putty)
Once logged in, cd to your magento/shell/ (where magento is your Magento root directory)
Run the following command to reindex your site: php indexer.php reindexall
Wait for the index processes to complete.
Lastly, ensure that your Catalog is using the Flat index tables. To do this:
Go to System > Configuration > Catalog > Frontend (section)
Set Use Flat Catalog Category to Yes
Set Use Flat Catalog Product to Yes
Click Save Config
No, you're safe to reindex whenever you see the notice appear.
If you know you're going to make a lot of changes, you can wait until you're done, saving yourself some time but only running it once at the end.
The only exception where this is not safe is if you have tens of thousands of products and/or lots of store views. It may end up running for hours and hours, slowing down your site leading to an undesirable experience for the customer.
I have found on sites with a large number of products, running the price reindex can cause a database lock, which can cause certain actions to be unavailable and for orders to be duplicated during that time. It also can affect performance and eat resources. I recommend performing this late at night only if possible.
I am facing one issue in Magento. I am having one Magento store with multi website functionality which containing approx 4500 products. I want to re-indexing product.
I had import 4500 product by csv through magento default functionality. after importing product the changes is not showing on front side so i went to index management and i found there are two indexes are in processing status
1 Product Attributes 2 Product Flat Data
I had already done following steps:
1 try to re-index it from admin side system->index management
2 try to do manually by calling php script
require_once 'app/Mage.php';
umask( 0 );
Mage :: app( "default" );
$process = Mage::getSingleton('index/indexer')->getProcessByCode('catalog_product_flat');
$process->reindexAll();
OR
$indexingProcesses = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach ($indexingProcesses as $process) {
$process->reindexEverything();
}
Also change the var/locks folder permission to 777 and also rename that folder and also try to delete the .lock file which was created in this lock folder but not got the solution.
I am not having SSH rights. So is there any other solution which will help me to solve re-indexing problem.
NOTE: While the below answer has, in my experience, been relevant to this type of issue, it does not appear to be the cause of the situation currently experienced by the asker
Processing means one of two things:
The indexes may actually still be runnning, in which case, sit back and wait for them to finish. Magento indexing can take a very long time (for 4500 products, "hours" is possible, depending on the server).
An indexer process may have died, leaving stale locks behind. Meanwhile, the dead indexer may have been run by a different user. The most common case is cron being incorrectly configured to run the indexer as root, or a normal user account, rather than as the same user as is used by the website (eg: apache or www).
All that Processing really means is "Magento failed to acquire a lock for these indexer jobs". The first case is trivial and uninteresting. Wait a couple of hours, and if the same indexers are still listed as Processing, then you may have the second case.
Check the permissions of the lock files, found under var/locks in your magento root. Are they owned by the same user as the web server is running as? If not, and you are absolutely certain that the indexes are no longer running, it is safe to delete the locks. The next step is to find out why the locks had the wrong permissions in the first place. That is a conversation which might be better had with your host, if you don't have ssh access.
The errors from the indexer will be caught and not logged by default. The typical workaround is to use the CLI re-index tool; which will be very verbose with any errors.
Eg.
php shell/indexer.php --reindex ...
But given you don't have SSH access - you can either look at the indexer.php file to see how they generate the errors, or you could just launch a shell_exec or exec from a web-based PHP script that would emulate the CLI.
Reason
There are references to inexisting/deleted products in catalog_product_flat_% tables.
Solution
Try to truncate ´catalog_product_flat_%´ tables (catalog_product_flat_1, catalog_product_flat_2, catalog_product_flat_3 etc.) in MySQL console or via phpMyAdmin:
mysql > truncate table ´catalog_product_flat_1´;
mysql > truncate table ´catalog_product_flat_2´;
mysql > truncate table ´catalog_product_flat_3´;
Then reindex.
I was not able to reindex catalog_product_flat index process. After spending a day and trying several solutions on the Internet. i was able to fix the issue. Below is the steps.
Create a clone of the Magento Db which is facing the indexing issue.
Dump the database into the newly created db.
truncate the table catalog_product_flat_1 for single store and for multiple store there will be multiple table catalog_product_flat_* where * is the store id. Here truncate all that table.
point the mangento running instance to the newly created database and configure the databse, so that site is functioning normally.
Now run the command php document_root/shell/indexer.php --reindex catalog_product_flat or try reindexing from Admin. php document_root/shell/indexer.php --reindexall for reindexing all the process.
I ran into the same issue, when creating new customer group I was unable to reindex prices.
Found the solution here http://www.magikcommerce.com/blog/how-to-resolve-magento-reindexing-errors-in-your-magento-store/
Here the procedure:
Locate var/locks directory and remove all files under this directory. This will clear all the locks for re-indexing to take place again.
Now, login to your MysQSL/phpMyAdmin to run the following MySQL query (Ensure that your have taken full backup before committing this MySQL query)
DELETE cpop.* FROM catalog_product_option_price AS cpop
INNER JOIN catalog_product_option AS cpo
ON cpo.option_id = cpop.option_id
WHERE
cpo.type = 'checkbox' OR
cpo.type = 'radio' OR
cpo.type = 'drop_down';
DELETE cpotp.* FROM catalog_product_option_type_price AS cpotp
INNER JOIN catalog_product_option_type_value AS cpotv
ON cpotv.option_type_id = cpotp.option_type_id
INNER JOIN catalog_product_option AS cpo
ON cpotv.option_id = cpo.option_id
WHERE
cpo.type <> 'checkbox' AND
cpo.type <> 'radio' AND
cpo.type <> 'drop_down';
Log back in to your Magento Admin panel and go to System tab > Index Management hit index again and you will notice no such errors will appear again. You can follow these same steps again if re-indexing stops in future to resolve Magento ReIndexing issues.
We have 4000 products with 4 stores and around 80 categories. We are running a dedicated DB server having SSD and followed the white paper of optimizing DB as well as App Server. Each product also have 12 custom options. Now Indexing is very slow. Any suggestions. Would it make any difference to reduce number of categories in the store etc.
Regards,
Steve
Indexing is dramatically slowed when multi store setups are used. On one of our servers 3000 products on a single store setup takes approx 60 seconds to index everything. But 3000 products on a multi store setup (7 stores) can take up to 10 minutes. We have it set up to manual index only. You could set the indexing to run on a cron which might help you. (Throwing more resources at the server will obviously help too).
Reduce Catalog price rules if in use
Reduce all attributes not required for Search Index - the default Magento install includes attributes like "Tax Status" in the Search Index and then this is multiplied 4x for your multi stores
Same can be said for URL rewrites - 4x increase
12 custom options per product ? Could common attributes be used instead ?
Run indexing from SSH rather than from the Magento admin
In the web root in the shell/ folder you will find indexer.php
php indexer.php --help will give you some options
You can then time each of the nine indexes to see which is taking the most time which may help narrow the problem down
By running from command line you can increase the php memory limit just for that one process which may improve results
I set up a cron job to re-index the site in off peak times and in Magento admin, change to Manual Update only
I am importing about 10.000 products and updating their data with import custom script on regular basic. I use Magento object to save product data. The problem is that for each product save the process is slower. On 1000 products save it becomes really slow. When I clear cache, everything is ok again.
I have now couple of questions to understand the thing:
Does anybody have any idea why is that?
Should I disable "Collections Data" cache, or maybe any other type of cache as well?
Or is there any way to tell Magento not to cache collection data on product save?
If not, will disabling Collections Data cache slow the page a lot?
Thank you
The reason for the slowness is your indexes are getting larger. Unless specified Magento will reindex for each new product, you can speed this up during your imports by disabling it, however you will need to reindex at some point to be able to present the newly imported products to the frontend.
A solution to consider:
Magento API: Rebuild Indexes after adding new products
I am developig an import module which update product data. To speed up the process, I put index to manual mode.
$processes = Mage::getSingleton('index/indexer')->getProcessesCollection();
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_MANUAL));
$processes->walk('save');
and after the import is finished, I reindex data and put index mode back to auto
$processes = Mage::getSingleton('index/indexer')->getProcessesCollection();
$processes->walk('reindexAll');
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_REAL_TIME));
$processes->walk('save');
But I am not sure if I also need to clear cache. So my question is how index and cache are related. For example if I clear cache, does it also reindex all data? And on the other site, if I reindex all data, does it clear cache? Or do I need to trigger everytime both processes if I have index mode set to manual? I am not quite sure about this, I hope anybody could confirm it for sure.
Thank you
Magentos System -> Cache Managment and System -> Index Managment are both stand-alone features. If you rebuild such index, no matter whether thru backend or directly using reindexAll(), Magento will not automatically refresh any cache and vice versa.
The answer to Do I need both? (caches and indexes) is: it depends.
If you're running Magento with caches COLLECTION_DATA and/or EAV enabled, you should refresh those caches after importing and reindexing product data.
The refreshing is necessary because your importer has updated/inserted product data which the caches are not aware of, but not, because you've reindexed.
If you're running Magento with all caches disabled, you don't need both. Technically, there is no need to refresh disabled cache. Magento would be slower of course, but still be fully functional.