Magento Re- Indexing Issue - magento

I am facing one issue in Magento. I am having one Magento store with multi website functionality which containing approx 4500 products. I want to re-indexing product.
I had import 4500 product by csv through magento default functionality. after importing product the changes is not showing on front side so i went to index management and i found there are two indexes are in processing status
1 Product Attributes 2 Product Flat Data
I had already done following steps:
1 try to re-index it from admin side system->index management
2 try to do manually by calling php script
require_once 'app/Mage.php';
umask( 0 );
Mage :: app( "default" );
$process = Mage::getSingleton('index/indexer')->getProcessByCode('catalog_product_flat');
$process->reindexAll();
OR
$indexingProcesses = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach ($indexingProcesses as $process) {
$process->reindexEverything();
}
Also change the var/locks folder permission to 777 and also rename that folder and also try to delete the .lock file which was created in this lock folder but not got the solution.
I am not having SSH rights. So is there any other solution which will help me to solve re-indexing problem.

NOTE: While the below answer has, in my experience, been relevant to this type of issue, it does not appear to be the cause of the situation currently experienced by the asker
Processing means one of two things:
The indexes may actually still be runnning, in which case, sit back and wait for them to finish. Magento indexing can take a very long time (for 4500 products, "hours" is possible, depending on the server).
An indexer process may have died, leaving stale locks behind. Meanwhile, the dead indexer may have been run by a different user. The most common case is cron being incorrectly configured to run the indexer as root, or a normal user account, rather than as the same user as is used by the website (eg: apache or www).
All that Processing really means is "Magento failed to acquire a lock for these indexer jobs". The first case is trivial and uninteresting. Wait a couple of hours, and if the same indexers are still listed as Processing, then you may have the second case.
Check the permissions of the lock files, found under var/locks in your magento root. Are they owned by the same user as the web server is running as? If not, and you are absolutely certain that the indexes are no longer running, it is safe to delete the locks. The next step is to find out why the locks had the wrong permissions in the first place. That is a conversation which might be better had with your host, if you don't have ssh access.

The errors from the indexer will be caught and not logged by default. The typical workaround is to use the CLI re-index tool; which will be very verbose with any errors.
Eg.
php shell/indexer.php --reindex ...
But given you don't have SSH access - you can either look at the indexer.php file to see how they generate the errors, or you could just launch a shell_exec or exec from a web-based PHP script that would emulate the CLI.

Reason
There are references to inexisting/deleted products in catalog_product_flat_% tables.
Solution
Try to truncate ´catalog_product_flat_%´ tables (catalog_product_flat_1, catalog_product_flat_2, catalog_product_flat_3 etc.) in MySQL console or via phpMyAdmin:
mysql > truncate table ´catalog_product_flat_1´;
mysql > truncate table ´catalog_product_flat_2´;
mysql > truncate table ´catalog_product_flat_3´;
Then reindex.

I was not able to reindex catalog_product_flat index process. After spending a day and trying several solutions on the Internet. i was able to fix the issue. Below is the steps.
Create a clone of the Magento Db which is facing the indexing issue.
Dump the database into the newly created db.
truncate the table catalog_product_flat_1 for single store and for multiple store there will be multiple table catalog_product_flat_* where * is the store id. Here truncate all that table.
point the mangento running instance to the newly created database and configure the databse, so that site is functioning normally.
Now run the command php document_root/shell/indexer.php --reindex catalog_product_flat or try reindexing from Admin. php document_root/shell/indexer.php --reindexall for reindexing all the process.

I ran into the same issue, when creating new customer group I was unable to reindex prices.
Found the solution here http://www.magikcommerce.com/blog/how-to-resolve-magento-reindexing-errors-in-your-magento-store/
Here the procedure:
Locate var/locks directory and remove all files under this directory. This will clear all the locks for re-indexing to take place again.
Now, login to your MysQSL/phpMyAdmin to run the following MySQL query (Ensure that your have taken full backup before committing this MySQL query)
DELETE cpop.* FROM catalog_product_option_price AS cpop
INNER JOIN catalog_product_option AS cpo
ON cpo.option_id = cpop.option_id
WHERE
cpo.type = 'checkbox' OR
cpo.type = 'radio' OR
cpo.type = 'drop_down';
DELETE cpotp.* FROM catalog_product_option_type_price AS cpotp
INNER JOIN catalog_product_option_type_value AS cpotv
ON cpotv.option_type_id = cpotp.option_type_id
INNER JOIN catalog_product_option AS cpo
ON cpotv.option_id = cpo.option_id
WHERE
cpo.type <> 'checkbox' AND
cpo.type <> 'radio' AND
cpo.type <> 'drop_down';
Log back in to your Magento Admin panel and go to System tab > Index Management hit index again and you will notice no such errors will appear again. You can follow these same steps again if re-indexing stops in future to resolve Magento ReIndexing issues.

Related

Cloudera Hadoop Impala - Extracting last refresh date

Is there a way to get the list of all tables with the last refresh date from a database in the Cloudera Hadoop impala?
I'm trying to write a custom SQL query that can do that so I can use it to build a dashboard (in Tableau) where we can track if a table is refreshed or not. So we can take action accordingly. I tried it using a join but there are so many tables and I believe there is a better way to do it. (Database name Core_research and there are more than 500 tables)
I used to run a script that refreshed column stats on tables every Sunday. We couldn't run all the tables but we did as many as time permitted. You could do the same but actually record when the script ran in database/table. This would give you the functionality you are looking for.
Another other option would be to create a table out of the Impala logs and keep track of things that way. (With some fancy regex to track refreshes)

reindex required on product save magento

when the new products get uploaded they are not visible on the front end. They told me we need to re-index every time. However re-index gets stuck.Please look into this issue for me and let me know if we can have it reindex automatically.
Reindex mode is Update on save and currently showing processing in index management
If it shows as processing then it's probably running. It may take a long time depending on how many products you have. The core url rewrite index is the most intensive. The reason you need to index is most likely because you have enabled the Use Catalog Flat Tables options under system configuration which is good for performance and speed of the site. Downside is you need to re-index these flat tables when you add new items. You can have the indexes run automatically by configuring cron to call magentos index process.
e.g.
* * * * * /usr/bin/php /var/www/magentosite/cron.php
5 0 * * * /usr/bin/php /var/www/magentosite/shell/indexer.php --reindex all
If it is stuck and you are sure it's not running, then you can change the status of the index process back to pending manually in the database by editing the table index_process. Re-run it manually from the admin and see if it fails. The index most likely causing your products not to show up is Flat Catalog Product Indexes so run this one first.
Yes, When you insert new product so re-indexing is require and also disable you magento cache.
and also flush your magento cache.

Magento: product bulk import on cron run

I have more than 2000 product, are updating on cron run. The procedure is delete the product first and then import. I am using:
$product->delete() for delete. But its taking long time to run whole script and getting
500 Internal error. How can I optimize this one.
I really need to get out from here. I also increased my php.ini max_execution_time.
Rather then a delete break this into three parts:
Delete by checking what are going to be non existent as per feed received.
Insert/update other products.
If for some reason if you still want to delete all, delete a smaller chunk of product and loop till all are deleted.

Magento Reindexing Data - Risks

I have a Magento site in which the cross-selling products do not seem to be appearing.
After looking on Stack and Google it seems that 'reindexing the data' has solved this issue for a lot of individuals.
My question is, are there any risks associated with performing this task? Or is it a relatively straight forward procedure?
Indexing is a fundamental part of Magento and will not effect your site in a negative way.
Magento uses a complex EAV (entity-attribute-value) database structure that can sometimes require heavy database queries to retrieve simple results. Because of this, the Magento developers have implemeted Index tables that query all of this data, and store it into a single table structure. This allows Magento to quickly query the single Index table, rather than making complex joins across multiple tables.
With that being said, Reindexing does not alter your existing data. It simply queries your existing data and copies it to it's own tables.
To reindex your site, you can simple go to System > Index Management, check off all the indexes that you wish to reindex, then submit.
If you have a large set of products, I recommend reindexing your site from the shell command line.
Login to your site using an SSH program (such as Putty)
Once logged in, cd to your magento/shell/ (where magento is your Magento root directory)
Run the following command to reindex your site: php indexer.php reindexall
Wait for the index processes to complete.
Lastly, ensure that your Catalog is using the Flat index tables. To do this:
Go to System > Configuration > Catalog > Frontend (section)
Set Use Flat Catalog Category to Yes
Set Use Flat Catalog Product to Yes
Click Save Config
No, you're safe to reindex whenever you see the notice appear.
If you know you're going to make a lot of changes, you can wait until you're done, saving yourself some time but only running it once at the end.
The only exception where this is not safe is if you have tens of thousands of products and/or lots of store views. It may end up running for hours and hours, slowing down your site leading to an undesirable experience for the customer.
I have found on sites with a large number of products, running the price reindex can cause a database lock, which can cause certain actions to be unavailable and for orders to be duplicated during that time. It also can affect performance and eat resources. I recommend performing this late at night only if possible.

Magento 1.5.0.1 catalog_product_price index issue

I am using Magento 1.5.0.1 with 600,000 products. Indexing is a major issue, especially catalog_product_price index.
1/ Towards the end of the indexing process a query is run DELETE FROM catalog_product_index_price . This has the effect of removing every item from our site so that the site displays 'There are no products matching the selection.' for all categories, the home page, search results.
2/ The process to insert from catalog_product_index_price_idx into catalog_product_index_price takes 10 minutes so we have a 10 minute window with no products on the site. I am absolutely certain this is a bug, there is no way someone intended for indexing to remove all products for a period of time - even if it was only 10 seconds this is not right for an ecommerce website
3/ For some reason the process of DELETE FROM catalog_product_index_price sometimes leaves a few products in the table therefore when the process of inserting from catalog_product_index_price_idx into catalog_product_index_price runs the indexer throws up an integrity constraint issue because of duplicate entries. This ends the indexing process and leaves the site with no products on. We run indexing in the early hours of the morning so sometimes we have a number of hours with no products on site if the index fails.
Does anyone know of a fix to these issues or a better way to update prices on the site that does not require us to index?
Firstly well done for running Magento with 600k products, that is the most I have heard of.
The best way to work around this I think would be to override the indexing process so that instead of truncating and rebuilding the price index, it replaces line by line. It is likely that this would take longer over all but would resolve the issue of it having this window with no products.
One thing you could try is to replace DELETE FROM with TRUNCATE TABLE which might be more reliable for your "still some items in the table" issue.
Ultimately though I think you are going to be building a bespoke optimised indexer.

Resources