I am taking over a website from another developer, and have found the setup of attributes to be a bit of a mess.
Specifically, products have attributes that aren't associated with the relevant attribute sets.
If you take wine with an attribute set "Wines", one of whose attributes is "Grape Variety".
But I also have "Beers" with a completely different attribute set, but somehow one of my beers has a Grape Variety.
It's not assigned to the Beers attribute set, it doesn't show up in the back end for this product, (so I can't edit it) but if I look in the database it's there (in catalog_product_entity_* and catalog_product_index_eav), furthermore when I do an export it's there too, and if someone searches for "Merlot", they are coming up with this beer. There are hundreds of products like this.
What is the best way of removing all attributes from products that are not within their assigned attribute sets?
I could figure it out in SQL I'm sure, but that's not the best way of doing things as I'd be afraid of missing something and screwing up the products altogether.
This is what I ended up doing. A few weeks later and it doesn't seem to have caused any issues yet:
CREATE TABLE catalog_product_entity_int_old LIKE catalog_product_entity_int;
INSERT INTO catalog_product_entity_int_old SELECT * FROM catalog_product_entity_int;
DELETE FROM catalog_product_entity_int
WHERE value_id IN
(SELECT cpei.value_id
FROM catalog_product_entity_int_old cpei
WHERE cpei.attribute_id NOT IN
(SELECT eea.attribute_id
FROM eav_entity_attribute eea
JOIN catalog_product_entity cpe ON eea.attribute_set_id = cpe.attribute_set_id
WHERE cpe.entity_id = cpei.entity_id)
ORDER BY cpei.entity_id)
and
DELETE FROM catalog_product_index_eav WHERE
attribute_id NOT IN (
(SELECT eea.attribute_id
FROM eav_entity_attribute eea
JOIN catalog_product_entity cpe ON eea.attribute_set_id = cpe.attribute_set_id
WHERE cpe.entity_id = catalog_product_index_eav .entity_id)
);
Then regenerate the indices.
I would definitely use Magmi to clean this up:
First, do a product export from within Magento (System -> Import/Export -> Profiles) and choose "Export All Products." In the resulting CSV, you will have columns for each attribute and you can remove any irrelevant attribute values for each product. This will allow you to bypass the backend where attributes are set for a particular product, but not in the product's attribute set.
Take a good look at the Magmi Wiki, but a few quick tips: Magmi only imports the columns you specify, so you can safely remove most of the columns when you perform your import. For example, I always make sure to remove the image columns if I'm not importing images (or you may lose all of your images). Also, be sure to do a database backup before running your import in case something goes wrong.
Related
Which tables are connected with the process of reindexing of index in magento.
Please share any documents available for the same.
Can't take credit for this as it is taken from original post at: Can someone explain Magentos Indexing feature in detail?
Magento's indexing is only similar to database-level indexing in spirit. As Anton states, it is a process of denormalization to allow faster operation of a site. Let me try to explain some of the thoughts behind the Magento database structure and why it makes indexing necessary to operate at speed.
In a more "typical" MySQL database, a table for storing catalog products would be structured something like this:
PRODUCT:
product_id INT
sku VARCHAR
name VARCHAR
size VARCHAR
longdesc VARCHAR
shortdesc VARCHAR
... etc ...
This is fast for retrieval, but it leaves a fundamental problem for a piece of eCommerce software: what do you do when you want to add more attributes? What if you sell toys, and rather than a size column, you need age_range? Well, you could add another column, but it should be clear that in a large store (think Walmart, for instance), this would result in rows that are 90% empty and attempting to maintenance new attributes is nigh impossible.
To combat this problem, Magento splits tables into smaller units. I don't want to recreate the entire EAV system in this answer, so please accept this simplified model:
PRODUCT:
product_id INT
sku VARCHAR
PRODUCT_ATTRIBUTE_VALUES
product_id INT
attribute_id INT
value MISC
PRODUCT_ATTRIBUTES
attribute_id
name
Now it's possible to add attributes at will by entering new values into product_attributes and then putting adjoining records into product_attribute_values. This is basically what Magento does (with a little more respect for datatypes than I've displayed here). In fact, now there's no reason for two products to have identical fields at all, so we can create entire product types with different sets of attributes!
However, this flexibility comes at a cost. If I want to find the color of a shirt in my system (a trivial example), I need to find:
The product_id of the item (in the product table)
The attribute_id for color (in the attribute table)
Finally, the actual value (in the attribute_values table)
Magento used to work like this, but it was dead slow. So, to allow better performance, they made a compromise: once the shop owner has defined the attributes they want, go ahead and generate the big table from the beginning. When something changes, nuke it from space and generate it over again. That way, data is stored primarily in our nice flexible format, but queried from a single table.
These resulting lookup tables are the Magento "indexes". When you re-index, you are blowing up the old table and generating it again.
When in backend I can't see all custom attributes. Same occurs in frontend.
I selected EAV_ATTRIBUTE table and can see 179 product attributes (type = 4).
But backend Product > Attributes shows only 119.
Anyone know what could be happening?
Magento Version: 1.7.0.2
IN the admin grid for attributes, the attribute collection is retrieved like this:
$collection = Mage::getResourceModel('catalog/product_attribute_collection')
->addVisibleFilter();
So not all the attributes that exist are listed in there. Only the ones marked as is_visible in the catalog_eav_attribute table.
Try this select and see what you get.
SELECT
*
FROM
eav_attribute e
LEFT JOIN `catalog_eav_attribute` ce
ON e.attribute_id = ce.attribute_id
WHERE
e.entity_type_id = 4 AND
ce.is_visible = 1
This should get you the attributes that are listed in the admin grid.
Perhaps the database has inconsistent data due to use of direct SQL insert statements or such. Its logical that only the consistent data would show up.
I've seen some import scripts which could lead to such inconsistencies floating around on the 'net.
Maybe you got bitten by one?
Would anyone be kind enough to explain how to change a SKU in an existing order? We import orders from a 3rd party and the SKUs have spaces in them, which need to be removed. I looked at sales_flat_order_item but not sure how to change. Any help will be very appreciated!
One possible way to remove blanks in SKUs of order items would be s/t like this:
$iIncrementId = '911';
$oOrder = Mage::getModel('sales/order')->load($iIncrementId, 'increment_id');
foreach ($oOrder->getAllVisibleItems() as $oItem) {
$oItem
->setSku(str_replace(' ', '', $oItem->getSku()))
->save();
}
I'm guessing you mean to change the Order number, because SKUs are related to products, not orders.
There's no UI way to change the Order number (there are several extensions but I strongly recommend not using ANY of them) but you can change it yourself if you have access to the database. The table sales_flat_order has an increment_id field where the order numbers are stored. This field is a varchar(50) meaning it supports alphanumeric values.
Magento keeps a UNIQUE index on the field so will not be able to keep duplicates.... unless you are running a Magento version prior to 1.4.x (actually, make that 1.4.0.22). If that's the case, be careful not to duplicate any increment_ids and whenever possible, stick to numerical values to prevent this bug http://www.magentocommerce.com/bug-tracking/issue?issue=13625
If you don't have access to the DB, you can always create an upgrade script that does the job, but add some logic to avoid duplicates or the installation script will blow in your face and the website will not work until you fix it
I'm trying to add some custom options to already existing products in Magento. Seems to work fine, I added the needed rows in the following tables:
catalog_product_option
catalog_product_option_title
catalog_product_option_type_value
catalog_product_option_type_price
catalog_product_option_type_title
I also updated has_options and required_options for the right product, in the following tables:
catalog_product_entity
catalog_product_flat_1
catalog_product_flat_2
catalog_product_flat_3
When I open the product, the options doesn't show, actually, it shows less. The button to order it disappears. When I open the edit page, it does show the options. After saving, it appears on the front-end too.
What am I missing?
Update:
After manually going through literally every query that was executed after a save action, I discovered what I was missing. When a product has options, it has to display them in a different template (or whatever it's called in Magento). To do this, you'll have to change the value for the attribute options_container.
So, there's a really easy fix for that. Just look up the attribute_id in the table eav_attribute. Then just run the following query for each product:
UPDATE `catalog_product_entity_varchar` SET `value` = 'container1' WHERE `attribute_id` = 836 AND `entity_id` = $productId;
That'll do the trick! :)
After manually going through literally every query that was executed after a save action, I discovered what I was missing. When a product has options, it has to display them in a different template (or whatever it's called in Magento). To do this, you'll have to change the value for the attribute options_container.
So, there's a really easy fix for that. Just look up the attribute_id in the table eav_attribute. Then just run the following query for each product:
UPDATE `catalog_product_entity_varchar` SET `value` = 'container1' WHERE `attribute_id` = 836 AND `entity_id` = $productId;
That'll do the trick! :)
You really shouldn't be accessing the database directly for any reason. This undermines the power and resourcefulness of using an EAV system.
Extend Mage.php if outside of magento (disregard if not)
Make a collection of whatever entities you want to manipulate
Use said collection to write/read data
Save yourself headaches down the road!
I am facing a problem while making product configurable i used this link http://www.magentocommerce.com/knowledge-base/entry/tutorial-creating-a-configurable-product/ but In associated tab the products doesnt show up
Thankds and regards
2 things you can check:
Are you sure that the simple products you want to associate to the configurable product actually have values for the attribute you made the configurable product configurable by? So if you made a configurable product based on 'color', do the simple products have values selected for 'color'?
When you are looking at the associated products tab, and seeing the blank grid there, have you tried resetting the filter, or selecting 'No' or 'Any' in the first column? If it is set to 'Yes', it is only looking for products that have already been associated.
How does configurables products linking works in magento ?
Getting rid of being trolled by the linking of configurables products with their simple counterparts ? Let's explain how does Magento link'em... and why it sometimes doesn't work as expected.
The first point to understand is how the data persistence is managed by the application. As expected, links are stored in database. Thought it was in catalog_product_relation ? You were wrong. It should be too simple to respect the Magento's spirit :)
catalog_product_relation vs catalog_product_super_* tables
I won't tell catalog_product_relation is useless – in fact, it's necessarily there for something. But from version 1.5+, this is not there that links are stored, and I can't explain what does its purpose is.
First step
The first step of the linking process is the definition of the configurable product's configurable attributes. Supposing we're creating a new configurable product from an attribute set having a few global-select attributes.
If you try to access to your configurable product, the application will ask you which of 'em should be used as configurable attributes, in a form using a list of checkboxes associated to each attribute. By selecting some and saving your product, you insert in catalog_product_super_attribute a row by attribute (an association between product's id and attribute's one).
Second step
The second step is the linking itself with the associated simple products. This step is a bit more complex, because it implies some checks of the database's content, which we'll detail further below. Some rules to keep in mind about that :
only products having the same attribute set as the current configurable product
options must be defined for the configurable attribute
only one simple can be linked for a given configurable attribute combination
simple can be linked to as many configurables as desired
Then, what happens when we go to our configurable's associated products tab ? Don't be scray to not see any of your simple-corresponding-product in the list that display – it's filter is configured by default to display only already linked products. Here is what happens behind the scene :
the application go check the catalog_product_super_attribute table, and bring back each corresponding attribute id defined there for the current product (our configurables attributes)
from them, it will go check eav_attribute to get some details about them – in which ones the backend_type, which is the more important part of this explanation
parallely, the catalog_product_super_link table is checked too. Any product linked to current_product is a potential linked product
for each attribute, the application check if a value exists in catalog_product_entity_{backend_type} for each potentially linked product. Note that if no value exists for a given product, it won't be validated nor as a potential link nor than an effective one, even if it has already been linked (we'll see below what it implies for attribute creation).
To resume, only products with a valid value are diplayable and effectively linked / linkable. Note that all of the combination must have valid values – if any of the configurable's attribute as none, game over, you'll not have any link between your simple product and its configurable counterpart.
We should think it's ok, and everything will go right now, since we know how everything works in database. That's true if you always create your attributes from the application's back-office. But since scripting attributes is a very common way to proceed, you're much like to be in this second case, which implies a potential HUGE problem.
Programatically created attributes and... What the hell !?! This attribute has change of backend type !
Edit note: this case happens effectively when using a eav/entity_attribute_source_table source model
This a trouble I personnally runned through. If you check attributes backend models in database (only for select attributes – we do not care about which ones are not usable as configurable's ones), you'll see some varchar, some int... In short, multiple ones, and we could rightfully expect any of them works.
And they do so. Since you do not decide to add an option to your attribute from the back-office, or anything else that require to click on the « save » button on the attribute page.
The point is that when you save your attribute, Magento, in it's great goodness, ask itself what is the type of the attribute you ask him to save. The point is that before anything else, it's a select. And a select has options. And options has id. Which are the values that will be put in catalog_product_entity_{backend_type}, in any case. The label is just properly ignored in that (it's stored in it's proper table and doesn't affect anything there). Only the id is used.
And what shall we expect from Magento :) ?
It just change the attribute backend type to int systematically.
So, if you have products that used to be linked and are no more, go check the catalog_product_entity_{backend_type} tables and the eav_attribute one – compare the backend type, look for values in each values table... If you find 'em in a table that does not correspond, you get your problem. You have a few ways to correct the problem, here are the both I find myself (and I really do not promote the first, which I put only for explanation purpose).
First way: get the backend type back (The Dirty Trick)
It has changed to int ? Don't care. Bring it back to what you want it to be.
Why it's dirty : because you want you user to be able to update it's attributes when he wants. If you update the backend type to varchar, for example, any label change in the back-office for an option will rollback to int as it will be saved.
Second way: let's correct everything ! (The Right Thing)
You can pass throught this... bad situation, with a bit of MySQL without harming your database. But please, first, dump it. Then you'll require 4 steps :
check that you don't already have the data in catalog_product_entity_int table
duplicate the rows of the current ...entity{attribute} table to the ..._entity_int one by using an INSERT / SELECT statement (this query can be a bit long)
update your configurable attributes (not all your attributes, ONLY the ones you know you have created and are use to set configurable products) to set their backend type to int (and never do something else again for configurable selects :) )
delete all entries corresponding to attribute in the ..._entity{attribute} table that you'll no more use (it can do pretty much entries since you can have a lot of products if you get a lot of options)
Here are some MySQL scripts, corresponding to these steps, supposing you have at start a list of the attributes codes concerned :
Check existing entries in entity int for a given list of attribute codes
-- Check existing entries to not duplicate
SELECT ea.attribute_code,
count(*)
FROM eav_attribute as ea
INNER JOIN catalog_product_entity_int as cpei
ON ea.attribute_id = cpei.attribute_id
WHERE ea.attribute_code IN(
{attributeCodesList}
)
GROUP BY ea.attribute_code
Duplicate entries from entity_varchar table (for example) to entity_int for a given list of attribute codes
-- Duplicating missed selects from varchat entity table to int
INSERT INTO catalog_product_entity_int (entity_type_id, attribute_id, store_id, entity_id, value)
SELECT cpev.entity_type_id,
cpev.attribute_id,
cpev.store_id,
cpev.entity_id,
cpev.value
FROM eav_attribute as ea
INNER JOIN catalog_product_entity_varchar as cpev
ON ea.attribute_id = cpev.attribute_id
WHERE ea.attribute_code IN(
{attributeCodesList}
)
Update attributes to never have the problem again!
-- Update missed select attributes from varchar backend type to int one
UPDATE eav_attribute as ea
SET ea.backend_type = 'int'
WHERE ea.attribute_code IN (
{attributeCodesList}
)
Remove the no more used entries from your database
-- Kill old varchar in varchar entity table
DELETE FROM catalog_product_entity_varchar
WHERE attribute_id IN (
SELECT ea.attribute_id
FROM eav_attribute as ea
WHERE ea.attribute_code IN (
{attributeCodesList}
)
)
Hope it will help all of the ones who can't make their associated products show up to get the trick!