I have a Magento site in which the cross-selling products do not seem to be appearing.
After looking on Stack and Google it seems that 'reindexing the data' has solved this issue for a lot of individuals.
My question is, are there any risks associated with performing this task? Or is it a relatively straight forward procedure?
Indexing is a fundamental part of Magento and will not effect your site in a negative way.
Magento uses a complex EAV (entity-attribute-value) database structure that can sometimes require heavy database queries to retrieve simple results. Because of this, the Magento developers have implemeted Index tables that query all of this data, and store it into a single table structure. This allows Magento to quickly query the single Index table, rather than making complex joins across multiple tables.
With that being said, Reindexing does not alter your existing data. It simply queries your existing data and copies it to it's own tables.
To reindex your site, you can simple go to System > Index Management, check off all the indexes that you wish to reindex, then submit.
If you have a large set of products, I recommend reindexing your site from the shell command line.
Login to your site using an SSH program (such as Putty)
Once logged in, cd to your magento/shell/ (where magento is your Magento root directory)
Run the following command to reindex your site: php indexer.php reindexall
Wait for the index processes to complete.
Lastly, ensure that your Catalog is using the Flat index tables. To do this:
Go to System > Configuration > Catalog > Frontend (section)
Set Use Flat Catalog Category to Yes
Set Use Flat Catalog Product to Yes
Click Save Config
No, you're safe to reindex whenever you see the notice appear.
If you know you're going to make a lot of changes, you can wait until you're done, saving yourself some time but only running it once at the end.
The only exception where this is not safe is if you have tens of thousands of products and/or lots of store views. It may end up running for hours and hours, slowing down your site leading to an undesirable experience for the customer.
I have found on sites with a large number of products, running the price reindex can cause a database lock, which can cause certain actions to be unavailable and for orders to be duplicated during that time. It also can affect performance and eat resources. I recommend performing this late at night only if possible.
Related
I have more and more disabled products and I am wondering if they could slow down my site.
All operations will be much faster on a smaller database.
Each product has tens of records in the EAV tables. 10-20k inactive products will have a major impact on anything that is not cached. Reindexing will take longer.
You should take into consideration the following when you delete products:
Reorder will not work (as the same product ordered is expected to exist)
Check if you have some custom reports/extensions that full the data from the product tables
Orders/Invoices created already will display all the informations
if you know how to optimize the whole thing, and your site was working fine before, deactivating products is not much impact on the speed of the store as a whole.
I am using Magento 1.5.0.1 with 600,000 products. Indexing is a major issue, especially catalog_product_price index.
1/ Towards the end of the indexing process a query is run DELETE FROM catalog_product_index_price . This has the effect of removing every item from our site so that the site displays 'There are no products matching the selection.' for all categories, the home page, search results.
2/ The process to insert from catalog_product_index_price_idx into catalog_product_index_price takes 10 minutes so we have a 10 minute window with no products on the site. I am absolutely certain this is a bug, there is no way someone intended for indexing to remove all products for a period of time - even if it was only 10 seconds this is not right for an ecommerce website
3/ For some reason the process of DELETE FROM catalog_product_index_price sometimes leaves a few products in the table therefore when the process of inserting from catalog_product_index_price_idx into catalog_product_index_price runs the indexer throws up an integrity constraint issue because of duplicate entries. This ends the indexing process and leaves the site with no products on. We run indexing in the early hours of the morning so sometimes we have a number of hours with no products on site if the index fails.
Does anyone know of a fix to these issues or a better way to update prices on the site that does not require us to index?
Firstly well done for running Magento with 600k products, that is the most I have heard of.
The best way to work around this I think would be to override the indexing process so that instead of truncating and rebuilding the price index, it replaces line by line. It is likely that this would take longer over all but would resolve the issue of it having this window with no products.
One thing you could try is to replace DELETE FROM with TRUNCATE TABLE which might be more reliable for your "still some items in the table" issue.
Ultimately though I think you are going to be building a bespoke optimised indexer.
I am importing about 10.000 products and updating their data with import custom script on regular basic. I use Magento object to save product data. The problem is that for each product save the process is slower. On 1000 products save it becomes really slow. When I clear cache, everything is ok again.
I have now couple of questions to understand the thing:
Does anybody have any idea why is that?
Should I disable "Collections Data" cache, or maybe any other type of cache as well?
Or is there any way to tell Magento not to cache collection data on product save?
If not, will disabling Collections Data cache slow the page a lot?
Thank you
The reason for the slowness is your indexes are getting larger. Unless specified Magento will reindex for each new product, you can speed this up during your imports by disabling it, however you will need to reindex at some point to be able to present the newly imported products to the frontend.
A solution to consider:
Magento API: Rebuild Indexes after adding new products
I'm having to convert an existing e-commerce site with 50k plus products to a magento site. Everywhere I look the advice is to use the flag catalog for this amount of products.
My question is, once enabled do new products have to be created using the old EAV tables or can I just import and update new products in the newly created flat catalog?
Thanks for any advice, I'm not looking forward to this transation at all, lol. ;)
Think of Flat Catalog as a cache of the EAV structure. It does not replace the EAV system it simply creates a "flattened" or simplified version of the data store in EAV tables.
The EAV system is the most flexible way to store data, allowing any number of user or system variables without changes to the table structure. The down side is this system requires multiple and or recursive queries, this is slow and memory intensive. This is where the flat catalog comes in... The following is still quite accurate (even though it was written when flat catalog was first introduced) and quite clear: http://www.magentocommerce.com/boards/viewthread/37247/#t122010
You will need to optimise memory usage within PHP and MySQL to enable rebuilding of flat catalog for a site with a large number of products.
I don't know what version of Magento your using, but until 1.4 you need to put it into the EAV structure if you have to manage your products using backoffice.
The creation of the flat_ tables are automatic, it's part of the indexing process (which can be very long for this amount of products)
edit: I don't know for version after 1.4
I am planning a high performance e-commerce project in asp.net and need help in selecting the optimal data retrieval model for the product catalog.
Some details,
products in 10-20 categories
1000-5000 products in every category
products listed with name, price, brand and image, 15-40 on every page
products needs to be listed without table-tags
product info in 2-4 tables that will be joined together (product images not stored in db)
webserver and sql database on different hardware
ms sql 2005 on shared db-server (pretty bad performance to start with...)
enable users to search products combining different criteria such as pricerange, brand, with/without image.
My questions are,
what technique shall I use to retrieve the products?
what technique shall I use to present the products?
what cache strategy do you recomend?
how do I solve filtering, sortering, pageing in the most efficient way?
do you have any recomendations for more reading on this subject?
Thanks in advance!
Let the SQL server retrive the data.
With fairly good indexing the SQL server should be able to cope.
in SQL 2005 you can do paging in results, that way you have less data to shuffle back and forth.
I think you will end up with a lot of text searching. Give a try for either lucene or Solr ( http server on top of lucene). CNET developed solr for their product catalog search.
Have you thought about looking at an existing shopping cart platform that allows you to purchase the source code?
I've used www.aspdotnetstorefront.com
They have lots of examples of major e-commerce stores running on this platform. I built www.ElegantAppliance.com on this platform. Several thousand products, over 100 categories/sub-categories.
Make sure your database design is normalised as much as possible - use lookup tables where necessary to make sure you are not repeating data unnecessarily.
Store your images on the server filesystem and store a relative (not full) path reference to them in the database.
Use stored procedures for as much as possible, and always retrieve the least amount of data as you can from the server to help with memory and network traffic efficiencies.
Don't bother with caching, your database should be fast enough to produce results immediately, and if not, make it faster.