Magento: Updating Product Catalogs faster - magento

I have written quite a few scripts to update my product catalog based on some or other parameter. In each of them the base logic is something simillar to this...
//Get collection
$collection = Mage::getModel('catalog/product')->getCollection();
$collection->addAttributeToSelect('sku');
$collection->addAttributeToSelect('publihser');
$collection->addFieldToFilter(array(array('attribute'=>'publisher','eq'=>$publisher)));
// for each product in collection do a individual save
foreach ($collection as $product) {
$product->setSKU($newValue);
$product->save();
}
Though this work, each save is a SQL update query and the fact is that having a very large catalog, this is fairly slow.
I was wondering if this could be sped up by doing single save on the collection instead on the product.

There are several things you can do to write a much faster update script. I don't know how you are getting some of your variables so you'll need to modify to get it working in your case, but the code below should be much faster than the way you are currently doing it. e.g:
// Set indexing to manual before starting updates, otherwise it'll continually get slower as you update
$processes = Mage::getSingleton('index/indexer')->getProcessesCollection();
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_MANUAL));
$processes->walk('save');
// Get Collection
$collection = Mage::getModel('catalog/product')->getCollection()
->addAttributeToSelect('sku')
->addAttributeToSelect('publihser')
->addFieldToFilter(array(array('attribute'=>'publisher','eq'=>$publisher)));
function productUpdateCallback($args){
$product = Mage::getModel('catalog/product');
$product->setData($args['row']);
$productId = $product->getId();
$sku = 'yourSku';
// Updates a single attribute, much faster than calling a full product save
Mage::getSingleton('catalog/product_action')
->updateAttributes(array($productId), array('sku' => $sku), 0);
}
// Walk through collection, for large collections this is much faster than using foreach
Mage::getSingleton('core/resource_iterator')->walk($collection->getSelect(), array('productUpdateCallback'));
// Reindex all
$processes->walk('reindexAll');
// Set indexing back to realtime, if you have it set to manual normally you can comment this line out
$processes->walk('setMode', array(Mage_Index_Model_Process::MODE_REAL_TIME));
$processes->walk('save');

Related

How can I sort a loaded product collection in Magento?

I want to sort a product collection that is already loaded by
$_productCollection = $this->getLoadedProductCollection();
The default sort in admin Magento is Style attribute
I want to sort first by Style, then by color and then by name.
I've try
$_productCollection->setOrder(array('style', 'color','name'), asc);
and also
$_productCollection->addAttributeToSort('color', Varien_Data_Collection::SORT_ORDER_ASC);
$_productCollection->addAttributeToSort('name', Varien_Data_Collection::SORT_ORDER_ASC);
but is not working.
The default sort is working good. Can someone please help?
You can do it this way:
$_productCollection = $this->getLoadedProductCollection();
$_productCollection->clear();
$_productCollection->addAttributeToSort('color', Varien_Data_Collection::SORT_ORDER_ASC);
foreach ($_productCollection as $product) {
// ...
}
This way the collection is forced to be reloaded and then your custom sorting is applied.
You can also take clone of the loaded collection and then modify the query as you want.
// clone of the current collection.Better way of modifying the colection
$_productCollection = clone $this->getLoadedProductCollection();
// unset the cuurent coolection and append your custom filter.
$_productCollection->clear()
->addAttributeToFilter('color', Varien_Data_Collection::SORT_ORDER_ASC)
->load();
Nothing here or elsewhere worked for me. No matter what I tried it would just not sort on /Magento_Catalog/templates/product/list.phtml using 'getLoadedProductCollection()'. Also had a weird issue where products would disappear after clearing cache, until a reindex was done. No idea what was going on there, something was not right.
So I got it to work in the hacky way below (Yes I know it's not 'best practice' to use object manager but it was the only thing that worked!!)
$_objectManager = \Magento\Framework\App\ObjectManager::getInstance();
$categoryId = 2;
$category = $_objectManager->create('Magento\Catalog\Model\Category')->load($categoryId);
$productCollection = $_objectManager->create('Magento\Catalog\Model\ResourceModel\Product\CollectionFactory');
$_productCollection = $productCollection->create()
->addAttributeToSelect('*')
->setOrder('position', 'ASC')
->addCategoryFilter($category)->load();
If you have products in a collection that need more advanced sorting/oordering you can move the products to an assoc array which is much easier to sort/order as you like. Once the order is as you like you can add them back by first clearing your collection with $collection-removeAllItems() and then iterate your array as $product and move each product back to the collection with $collection->addItem($product);
I used this when I wanted a sticky that was not touched by different ordering/sorting.
You can use setOrder on collection to sort any collection data like below :
$_productCollection->setOrder('id','DESC');
Where id can be replace with your column name, & second parameter can be DESC & ASC as per your requirement.
Have it working in v2.3.7 using the following
$_productCollection = $block->getLoadedProductCollection();
$_productCollection->getSelect()->reset(Zend_Db_Select::ORDER);
$_productCollection->setOrder('price', 'ASC');

Programmatically modify related products in magento

I'm trying to programmatically manipulate the product relations in a Magento store.
From what I've read, setRelatedLinkData should be the way to go.
As I simple test, I'm just trying to replace a products related products with nothing (i.e. an empty array), however it's not working - the product in question is still showing the related product in the backend.
The test code I'm working with is:
$product = Mage::getModel('catalog/product')->load($product->getId());
$linkData = array();
print_r($linkData);
$product->setRelatedLinkData($linkData);
echo "Save\n";
$r = $product->save();
As mentioned above however the product still has a related product when I reload it in the backend.
NOTE: I don't only want to remove related products, eventually I want to be able to add new ones as well, so a DELTE FROM... SQL query isn't what I am looking for. However if I can't get it to work to remove products, then it's certainly not going to work to add them, so one step at a time :-)
The quickest way I can think of is to use the Link Resource:
app/code/core/Mage/Catalog/Model/Resource/Product/Link.php saveProductLinks
// sample code
$product = Mage::getModel('catalog/product')->load(147);
$linkData = array();
Mage::getResourceModel('catalog/product_link')->saveProductLinks(
$product, $linkData, Mage_Catalog_Model_Product_Link::LINK_TYPE_RELATED
);
and if you want to assign products use the same code but provide this as $linkData:
$linkData = array(
'145' => array('position' => 1),
'146' => array('position' => 2)
);

Which way is better to load models in magento?

I don't know if i am asking that right.I need to load a product, change some values and save it. My question is which way is the appropriate to do it.Currently i am using that way:
$id = Mage::getModel('catalog/product')->getIdBySku($sku);
$product = Mage::getModel('catalog/product')->load($id);
1) In general it works fine even with 40K products.But i read that this way is leading to memory leak? Also for that solution i read that if i disable the reindex functionality i can improve the process time.
2) If i use another way and load it as a collection and then apply some filters, ex addFieldToFilter('sku',$the_product_i_want)
would be better?
When i say better i mean: 1)the magento way, 2)time efficient 3)not doing something that i don't need to do.
Each time you call the following code snippet Mage::getModel('catalog/product') Magento will create a new model object in the memory. And that will lead to memory wastage. You can do as follows.
$model = Mage::getModel('catalog/product');
$id = $model->getIdBySku($sku);
$product = $model->load($id);
At the same time if you have the product object you can use the following code.
$collection = Mage::getModel('catalog/product')->getCollection();
$product = $collection->addFieldToFilter('sku',$the_product_i_want);

Magento get all products

I am trying to get the entire magento product collection, without any filters or restrictions, but I fail to get all products.
I've tried various methods already, but they all give me a very limited selection of products. Let's say the store contains 5000 products, but it only shows 500. When I check the catalog -> products is does show me the entire list.
Mage::getModel('catalog/product')->getCollection();
Mage::getResourceModel('catalog/product_collection')->addAttributeToSelect('*');
Mage::getModel("catalog/product")->getResourceCollection()->load();
All of them return the same amount (500), while I expect it to give me 5000 products. I would prefer not to use Zend or PHP and just stick to the Magento way to get them.
Does anyone know how to really get ALL products or can point me in the right direction why this isn't working?
The select-string that is returned is:
SELECT 1 AS `status`, `e`.`entity_id`, `e`.`type_id`, `e`.`attribute_set_id` FROM `catalog_product_flat_4` AS `e`
//to overwrite limit but you need first to increase your memory limit
$collection = Mage::getModel('catalog/product')->getCollection()
->addAttributeToSelect('*') // select all attributes
->setPageSize(5000) // limit number of results returned
->setCurPage(1); // set the offset (useful for pagination)
// we iterate through the list of products to get attribute values
foreach ($collection as $product) {
echo $product->getName(); //get name
echo (float) $product->getPrice(); //get price as cast to float
echo $product->getDescription(); //get description
echo $product->getShortDescription(); //get short description
echo $product->getTypeId(); //get product type
echo $product->getStatus(); //get product status
// getCategoryIds(); returns an array of category IDs associated with the product
foreach ($product->getCategoryIds() as $category_id) {
$category = Mage::getModel('catalog/category')->load($category_id);
echo $category->getName();
echo $category->getParentCategory()->getName(); // get parent of category
}
//gets the image url of the product
echo Mage::getBaseUrl(Mage_Core_Model_Store::URL_TYPE_MEDIA).
'catalog/product'.$product->getImage();
echo $product->getSpecialPrice();
echo $product->getProductUrl(); //gets the product url
echo '<br />';
}
And something like this:
$products = Mage::getModel('catalog/product')->getCollection();
foreach($products as $prod) {
$product = Mage::getModel('catalog/product')->load($prod->getId());
}
With this method I get more than 500 but all my product...
Several possibilities here:
1. Some inner limitation, like 500 at all.
2. Some paging limitation. Products per page(in db abstract)
3. Some lazyload limitation.
Perhaps, there is some over problem, but I think this is some inner limit.
Turn off your flat_catalog_product in admin > system > configuration > catalog > catalog. After this you will get a full product collection. Even though this is a workaround, it helped me to achieve what I needed to achieve.
You might be using Flat Catalog Product Structure . This create separate table for each store view.
Use Code below to print sql query . you will see the collection is coming from flat table like
catalog_product_flat_38
echo Mage::getModel('catalog/product')->getCollection()->getSelect();
try using
Mage::app()->setCurrentStore('0');
// same as
Mage::app()->setCurrentStore(Mage_Core_Model_App::ADMIN_STORE_ID);
this hepled me

Magento filter products by items that have this item in their related products

I wish to get a collection of products that have product x as a related product.
So, say I'm starting with
$_productCollection = $product->getCollection()
->addAttributeToSelect('*')
->addAttributeToFilter('status',1)
->addStoreFilter()
->distinct(true);
And wish to add something like (but with some reverse in instead of eq):
$_productCollection->addAttributeToFilter('related_ids',array('eq' => $idofproductx));
I can't find anything, but assume it will include a join of some description. I'd rather avoid loading all products and then all related for each cycling through them all, for obvious reasons.
You may be able to do
// #var $products array
$_productCollection->getSelect()->join(array('links' => 'catalog_product_link'), 'e.entity_id = links.product_id');
$_productCollection->getSelect()->where('links.linked_product_id IN (?)', $products);
I tested that it runs without error against my DB, but I don't have product links in my database (we fetch from a web service), so I can't be certain it returns the expected products.
I recently had the same issue, but I ended up making a small module for this so it's reusable.
But for simplicity a stripped back version of my solution can be found below.
// load by a product Id
$_product = Mage::getModel('catalog/product')->load($_productId);
$_linkInstance = $_product->getLinkInstance();
$collection = $_linkInstance->getLinkCollection();
// get products which have $_productId as a link
$collection->addFieldToFilter('linked_product_id', array('eq' => $_product->getId()));
// 1 = related products
$collection->addLinkTypeIdFilter(1);
$collection->joinAttributes();
The above code will populate $collection with an array of IDs which have assigned $_productId as a related product.
Note: This works on CE 1.9.1.0

Resources