In Magento I write a number of small command line scripts to do things like set a new attribute on a number of products. I am finding that the time it takes to update 900 products takes about 6 hours to complete.
The time it takes to load the individual products goes as fast as I would except, but the act of saving once I have made the change takes a very long time.
I am attaching how I am loading the products in case there is something I can do to better optimize the process. Any help here would be greatly appreciated.
$product = Mage::getModel('catalog/product')->load($magento_id);
$product->setMadeInUsa(1);
try {
$product->save();
} catch(Exception $e) {
echo "ERROR: " . $e->getMessage() . "\n";
}
The code runs without error, but it takes forever.
Mage::getSingleton('catalog/product_action')
->updateAttributes(array($product->getId()), $attributesData, $storeId);
This code only updates the attributes you want to change. The first paramater is an array of product IDs, the second is an array of attribute names and values, and then the third is the store ID you wish to update.
This is MUCH faster than saving the entire model.
Try first seting indexing to Manual and then reindex after update is done. This should improve the performance. However the ultimate solution, if you are going to do the import often, is to follow the code ideas you can find in update attributes mass action, which is optimized for saving many products at once.
Related
I want to test a simple strategy based on time: every day at fixed time check some conditions and go long. every day at fixed time exit.
Whatever I try to get timeOpenCondition I get syntax error.
Tried things similar to:
EntryTime = if hour=0800 and minute=0
ExitTime = if hour=1400 and minute=0
It feels like I don't get the concept of how this works. Appreciate any help!
I have about 25.000 rows in my DB table 'movies' (InnoDB, 17.5 mb)
And when I try to get them all to display in my admin panel, nothing happens. Just 5-8 seconds pending and white screen. No displayed errors, just nothing. (max execution time is 3600 seconds, because it's on my local machine). My simple as hell code:
public function index()
{
$data['movies'] = Movies::all();
dd('This var_dump & die never fires');
// return view('admin.movies', $data);
}
I just wonder why it not performs the query and just die without declaration of war.
I didn't found anything interesting in .ENV or config/database.php to explain what happens in such situations.
PS. yes, I can make serverside pagination and search, and take only 10-25 records from the DB, question is not about that.
Looks like you are running out of memory. Try quering half, of the results, or maybe just 100 to see if that at least fixes the white page, if so use chunk:
Movies::chunk(200, function($movies)
{
foreach($movies $movie)
{
var_dump($movie);
}
});
You should definitely look at your storage\logs directory to verify the error. It's quite possible that it takes to much memory getting 25k rows.
In fact as you mentioned in real life there is no need to get so many rows because unless you export them into CSV or XLS.
I'm trying to make an ajax autocomplete search box that of course uses SQL, min 3 characters, and have a SQL view of relevant fields already set up and indexed in the db. The CPU still spikes when searching, which I expected as it's running a query for every character. I want to use Zend shm cache to speed up results and reduce CPU usage. The results are stored in an array which is to be cached like this:
while($row = db2_fetch_row($stmt)) {
$fSearch[trim($row[0]).trim($row[1])] = array(/*array built here*/);
}
if (zend_shm_cache_store('fSearch', $fSearch, 10 * 60) === false) {
error_log('Failed to store search cache!');
}
Of course there's actual data inside the array instead of comments, I just shortened the code for simplicity. Rows 0&1 form the PK, and this has tested to be working properly. It's the zend_shm_cache_store that fails because the error log gets flooded with 'Failed to store search cache!'. I read that zend_shm_cache_store can store any array that can be serialized - how can I tell if my data is serialized or can be serialized? Are there any other potential causes? I did make a test page that only stored a string and that was successful, so I know caching is on.
Solved: cache size was too small for array - increased cache size and it worked fine. Sorry for the trouble.
I've written a module to import products and I'm currently using Magento's product model to add/update products accordingly. However, this is proving to be very slow (possibly because I have the catalog index enabled?).
Even doing just
$product = Mage::getModel('catalog/product')->load($id);
$product->save();
Is incredibly slow - We're talking maybe 2 seconds per product (I'm doing 5 per http request, and using javascript to make several requests).
Each product needs to have some attributes updated, category ids changed, and the stock levels updated. At the moment, I'm looping through 20 products and it's taking near enough 60 seconds. In production, it will be looping through 200-300 products (although on a much more powerful server).
Is there a better/faster way of creating/updating the products? Obviously I could just use SQL but I don't fancy figuring out Magento's intense EAV database structure!
Sorry if this is a naff question, I'm not sure how best to word it!
Setting the indexer mode to manual whilst importing will give you at least some performance gains. You can obviously set this in the admin area, but you can also do it via your script:
//Set to manual mode
$processCollection = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach($processCollection as $process) {
$process
->setMode(Mage_Index_Model_Process::MODE_MANUAL)
->save();
}
//Set back to real time mode
$processCollection = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach($processCollection as $process) {
$process
->setMode(Mage_Index_Model_Process::MODE_REAL_TIME)
->save();
}
If you are looking for a way to reindex directly in your script after importing...
$processCollection = Mage::getSingleton('index/indexer')->getProcessesCollection();
foreach($processCollection as $process) {
$process
->reindexEverything();
}
But, Magmi - http://sourceforge.net/projects/magmi/files/magmi-0.7/ - is not only amazingly fast at importing products, but it also provides some really nice features.
I don' know of any other product import tool for Magento that is as fast (would be interested if anyone else knows of one that is as fast or faster?)
I have a mobile app that is using LinqToDatasets to update/insert into a SQL Server CE 3.5 File.
My Code looks like this:
// All the MyClass Updates
MyTableAdapter myTableAdapter = new MyTableAdapter();
foreach (MyClassToInsert myClass in updates.MyClassChanges)
{
// Update the row if it is already there
int result = myTableAdapter.Update(myClass.FirstColumn,
myClass.SecondColumn,
myClass.FirstColumn);
// If the row was not there then insert it.
if (result == 0)
{
myTableAdapter.Insert(myClass.FirstColumn, myClass.SecondColumn);
}
}
This code is used to keep the hand held database in sync with the server database. Problem is if it is a full update (first time for example) there are a lot of updates (about 125). That makes this code (and more loops like it take a very long time (I have three such loops that take over 30 seconds each).
Is there a faster or better way to do updates/inserts like this?
(I did see this Codeplex Project, but I could not see how to make it work with both updates and inserts.)
You should always use SqlCeResultSet for data access on mobile devices for maximum performance and memory usage. You must identify the data to be inserted and then use code like the SqlCeBulkCopy sample, and use similar code by using the Seek and Update methods of the SqlCeResultSet.