I'm having to convert an existing e-commerce site with 50k plus products to a magento site. Everywhere I look the advice is to use the flag catalog for this amount of products.
My question is, once enabled do new products have to be created using the old EAV tables or can I just import and update new products in the newly created flat catalog?
Thanks for any advice, I'm not looking forward to this transation at all, lol. ;)
Think of Flat Catalog as a cache of the EAV structure. It does not replace the EAV system it simply creates a "flattened" or simplified version of the data store in EAV tables.
The EAV system is the most flexible way to store data, allowing any number of user or system variables without changes to the table structure. The down side is this system requires multiple and or recursive queries, this is slow and memory intensive. This is where the flat catalog comes in... The following is still quite accurate (even though it was written when flat catalog was first introduced) and quite clear: http://www.magentocommerce.com/boards/viewthread/37247/#t122010
You will need to optimise memory usage within PHP and MySQL to enable rebuilding of flat catalog for a site with a large number of products.
I don't know what version of Magento your using, but until 1.4 you need to put it into the EAV structure if you have to manage your products using backoffice.
The creation of the flat_ tables are automatic, it's part of the indexing process (which can be very long for this amount of products)
edit: I don't know for version after 1.4
Related
In working with Magento's EAV system, I have developed a PHP-based tool which will assemble the tables' data back into a tabular format as well as identify "holes" in the attributes between or across products. It works with both the product and category system, and allows determining if an attribute has NOT been inserted into a particular product, or if data is consistent across stores, or stores which are inconsistent - things that are not easy through normal SQL queries and which might be tough to program with Magento native coding.
I plan to post a link to this via github and here when complete.
My question is, are there other platforms or system which make use of EAV? I would like to make my application as system-agnostic as possible. Thanks.
I have a Magento site in which the cross-selling products do not seem to be appearing.
After looking on Stack and Google it seems that 'reindexing the data' has solved this issue for a lot of individuals.
My question is, are there any risks associated with performing this task? Or is it a relatively straight forward procedure?
Indexing is a fundamental part of Magento and will not effect your site in a negative way.
Magento uses a complex EAV (entity-attribute-value) database structure that can sometimes require heavy database queries to retrieve simple results. Because of this, the Magento developers have implemeted Index tables that query all of this data, and store it into a single table structure. This allows Magento to quickly query the single Index table, rather than making complex joins across multiple tables.
With that being said, Reindexing does not alter your existing data. It simply queries your existing data and copies it to it's own tables.
To reindex your site, you can simple go to System > Index Management, check off all the indexes that you wish to reindex, then submit.
If you have a large set of products, I recommend reindexing your site from the shell command line.
Login to your site using an SSH program (such as Putty)
Once logged in, cd to your magento/shell/ (where magento is your Magento root directory)
Run the following command to reindex your site: php indexer.php reindexall
Wait for the index processes to complete.
Lastly, ensure that your Catalog is using the Flat index tables. To do this:
Go to System > Configuration > Catalog > Frontend (section)
Set Use Flat Catalog Category to Yes
Set Use Flat Catalog Product to Yes
Click Save Config
No, you're safe to reindex whenever you see the notice appear.
If you know you're going to make a lot of changes, you can wait until you're done, saving yourself some time but only running it once at the end.
The only exception where this is not safe is if you have tens of thousands of products and/or lots of store views. It may end up running for hours and hours, slowing down your site leading to an undesirable experience for the customer.
I have found on sites with a large number of products, running the price reindex can cause a database lock, which can cause certain actions to be unavailable and for orders to be duplicated during that time. It also can affect performance and eat resources. I recommend performing this late at night only if possible.
we have 250k products and we want to create more than 200 different site(multi site) all stores will have same products but different domain and different design. Magento multi site function is good for us but you need to match every product and every site so that means 50m(min) record for tables like "catalog_product_website"(and some other tables) and also flat catalog and indexing will be a real problem.
if we use store and store view than userbase will be same and also flat catalog will be problem.
So my question, is there any way to make all these stores work like single store? Or is there any way to make it work with nice performance?
The short answer is no.
While Magento is built for multi-site applications, I don't think it's suited to running 200+ stores with 250,000 products.
At that size, I'd say contact Magento regarding Magento Enterprise to see what they have to say. Oh, and be prepared to pay the $15,000/Server/Year price.
I found that when i reindex from backend then magento create many additional table. what is the puprose of it?
for e.g.
catalog_category_flat_store_1
catalog_product_flat_1
catalog_product_index_price_final_idx
The flat tables are for the different stores in Magento. Each store will have its own flat table since each store can have different information for it.
The price index as far as I know is used to help speed up the category pages since it would be slower to use the actual price model for those pages.
For the most part all of the indexes in Magento are there to increase the overall speed of the system.
I am planning a high performance e-commerce project in asp.net and need help in selecting the optimal data retrieval model for the product catalog.
Some details,
products in 10-20 categories
1000-5000 products in every category
products listed with name, price, brand and image, 15-40 on every page
products needs to be listed without table-tags
product info in 2-4 tables that will be joined together (product images not stored in db)
webserver and sql database on different hardware
ms sql 2005 on shared db-server (pretty bad performance to start with...)
enable users to search products combining different criteria such as pricerange, brand, with/without image.
My questions are,
what technique shall I use to retrieve the products?
what technique shall I use to present the products?
what cache strategy do you recomend?
how do I solve filtering, sortering, pageing in the most efficient way?
do you have any recomendations for more reading on this subject?
Thanks in advance!
Let the SQL server retrive the data.
With fairly good indexing the SQL server should be able to cope.
in SQL 2005 you can do paging in results, that way you have less data to shuffle back and forth.
I think you will end up with a lot of text searching. Give a try for either lucene or Solr ( http server on top of lucene). CNET developed solr for their product catalog search.
Have you thought about looking at an existing shopping cart platform that allows you to purchase the source code?
I've used www.aspdotnetstorefront.com
They have lots of examples of major e-commerce stores running on this platform. I built www.ElegantAppliance.com on this platform. Several thousand products, over 100 categories/sub-categories.
Make sure your database design is normalised as much as possible - use lookup tables where necessary to make sure you are not repeating data unnecessarily.
Store your images on the server filesystem and store a relative (not full) path reference to them in the database.
Use stored procedures for as much as possible, and always retrieve the least amount of data as you can from the server to help with memory and network traffic efficiencies.
Don't bother with caching, your database should be fast enough to produce results immediately, and if not, make it faster.