It takes about 20 seconds to login to the admin backoffice. During this time load would spike (from 0.02 to 0.20)
Yet I see no "slow queries" in the mysql log. I've optimized all the tables.
Is there some way to hunt down the source of this slow down?
Which browser are you using? The Magento admin can be very slow in some browsers i.e. Internet explorer. I recommend using Chrome or Firefox for admin work, especially adding/editing products. If you are generally looking at ways to speed up Magento then here is a good place to start:
https://web.archive.org/web/20140327173649/http://www.gxjansen.com/101-ways-to-speed-up-your-magento-e-commerce-website/
The admin panel pulls in a lot of files and is heavily javascripted. Turning on things like mod_deflate and mod_expires can help tell your browser to not download these files as often.
Magento is extremely resource intensive and the admin interface does not use any caching. A lot of it depends on the hardware your site is on. For any site is decent traffic and heavy admin use, the base line should be a dedicated dual quad core server with 24-32mb of ram, especially if you have the db and web on the same server. Any shared server will just not cut it. Cloud based solutions vary, we have had some good results with splitting db and load balanced web in a cloud environment. Make sure you have apache and mysql properly tuned based on the memory of the server and use a php accelerator like apc.
Related
In the prestashop performance page, it offers caching with APC, memcached, File System, and XCache. There's a warning about making sure the infrastructure contains one front-end server. If not sure, ask your hosting company. What's available and best here?
I'm currently using the free 3 gear set up. 1 small gear with PHP 5.4 and scaling turned on (minimum 1, maximum all available), 1 small gear for mysql 5.5. the HA Proxy shows one of my gears as active/down (red) with 0 bytes of anything. I'm assuming because I don't have the High Availability turned on correctly. I'm getting page load times of 1.5 to 3 seconds on average. I know the small gear is low cpu, but is that a normal response time? How can I tune this to sub 1 with what I have? I'd like to see the performance capabilities and scaling abilities before I kick over a CC to grab bigger gears and more of them.
Speaking of scaling I see this The code in the git repository is copied to each new gear, but the data directory begins empty. I just recently made some post_deploy hooks to symlink an images directory so that my git updates from my local machine didn't destroy custom uploads. Am I going to have to switch to some kind of CDN? I believe I saw an option to set CDN servers in the application.
Thanks in advance,
Alan
Prestashop is probably a pretty beefy application to run on a small gear, even with moving the database to it's own gear. Your web gear would have to receive enough traffic to scale up to another web gear, but as long as you are just getting a few requests to it, it might respond slowly and not scale up. For a few cents an hour, it might be worth spinning up a medium/large gear and checking it's performance.
As for the data directory symlink. On a scaled application, anything you upload into your OPENSHIFT_DATA_DIR will not be synced across gears that are spun up. You might want to consider finding some kind of Amazon S3 plugin or similar.
I am new with Magento and have developed a website using CE 1.7.0.2. Now its ready to go live but I have issues with slow page load.
My website product home, list and detail page initially takes time to 10-13 sec to page load but after that first load it only takes 1-2 second to page load.
Also I have installed APC, Memcache and CDN on server and full page cache extension for website but yet it is slow. I am so frustrated why this happening with my website?
If anybody knows how can we resolve speed up issues that will be helpful for me.
Thanks!
You can go through the below steps for Magento Optimization:
High Performance Dedicated Server. Ex: Amazon EC2 cloud
Swap Apache for NGINX
Minimize Javascript use
Minify and Compressed CSS files
‘Combine CSS’ seeks to reduce the number of HTTP requests made by a
browser
Optimize images
Use lazyload for images
Specify Image dimensions
Combine images into CSS sprites
Use a Content Delivery Network (CDN) for delivering static files
like JS, CSS and Images to offload your server
Disable modules/extension which are not required
Enable all Magento Caches
Use a Full Page Cache / Varnish Cache / Memcache / RedisCache
Don’t use layered navigation if you don’t really need it, it needs
a lot of resources
Enable Compilation
Limit the number of products on a product overview page.
Set only those attribute frontend properties to ‘Yes’ that you’re
actually going to use. Set all other to ‘No’.
Don’t use in quick search, advanced search compare, etc. : Catalog
-> Attributes -> Manage Atributes -> Frontend Properties.
Install Google Page Speed Module
Minimize redirects – Minimizing HTTP redirects from one URL to
another cuts out wait time for users.
Prefer asynchronous resources – Fetching resources asynchronously
prevents those resources from blocking the page load.
This list may help you for Magento performance improvement:
Enable Magento caching
This is ofcourse the first step in optimization: Enable all the available caches in the Magento Admin Panel.
Compress images
Many people forget that images (PNG, JPG) can be compressed, which lowers the bandwidth between the browser and the webserver. Not only the images used by the Magento skin need optimizing, but catalog images as well. Various tools allow you to compress batches of images, for instance the online tool Smush.It.
Disable unneeded Magento modules
By disabling Magento modules that you do not need, less resources are needed – as simple as that. Modules could be disabled through the configuration in the Magento Admin Panel, or by editing XML-files in app/etc/modules. For instance, disable Mage_Log which performs queries on every request, but is not needed if you gather site statistics using external programs.
Enable flat catalogs for smaller webshops
For smaller webshops switching from the complex EAV-structure to a flat catalog could save time. This optimization is dubious and depends on many parameters, so do not take this step lightly.
W3C compliance
While it could be argued that this is less important with the coming of HTML5, it is still a fact that if your webpages are filled with ugly errors, the browser will have a harder time interpreting it. If you stick to W3C compliance, it is made sure the browser engine has an easy job parsing your HTML-code.
Compress output in general
By enabling the PHP-setting zlib.output_compression the output generated by PHP is compressed when sent to the browser. This saves bandwidth. Instead of using this, you could use the Apache mod_deflate module as well, which allows also for compression of non-PHP output (CSS, JavaScript, other plain text-files).
Configure PHP options
Most PHP settings actually do not influence the performance of Magento, but just set certain limits. For instance, settings like memory_limit and max_execution_time do not increase the page load but just make sure that certain actions do not timeout or run into memory problems.
Session storage
With Magento, sessions could be stored in files or in the database (by configuring app/etc/local.xml). Which option performs best, really depends on how the hosting environment is setup. If MySQL databases perform well, session storage in the database could benefit your site. But if MySQL is not setup correctly, the best choice might be files.
Use a PHP accelerator
By opcode caching, PHP-execution could be fastened. There are various PHP accelerators doing this job (APC, ZendOptimizer+, eAccelerator, XCache). Both APC and ZendOptimizer+ are working flawless with Magento.
Tune PHP realpath_cache
By tuning the PHP realpath_cache_size to for instance 128K (default is 16K) and the realpath_cache_ttl to 86400, things might be speeding up. Make sure you don’t run out of memory, because every Apache child will consume the configured caching size.
Use Apache mod_expires
By telling the browser which files to keep in cache for how long, you can optimize the browser cache. For instance, JavaScript files tend to change much less then CSS files (at least in the first stages of the site), but perhaps after the site is running smooth for some months you can maximize the expiration date.
Beware 404 errors
Every time a file (like a stylesheet or image) is not found, this generates a 404 error. Because the default 404 of Magento is caught by the application of Magento itself, this causes the Magento application to start for every 404 encountered. Check your Apache logs to make sure all 404 errors are solved.
Disable Magento logging
Within the Magento configuration, you can enable logging under the Developers-tab. Depending on the Magento modules this could lead to more logs needed to be written to the filesystem, slowing down your shop. Because the logging-abilities are only needed to debug something, in daily usage, it’s best to disable logging altogether.
MySQL table optimization
Through phpMyAdmin, you can perform the command OPTIMIZE TABLE on specific Magento database-tables. When a table is cluttered, this could lead to improved performance. This does not only count for the complex EAV-tables, but also for regular MySQL tables that are used frequently (for instance, core_config_data).
Merge CSS and JavaScript
By merging all CSS and JavaScript files together as one big file, only one single HTTP-request is needed by the browser to fetch this content. This saves bandwidth. For this merging, the FooMan Speedster module could be used. Magento 1.4 contains an option to merge CSS, while JavaScript-files are merged by default.
Besides merging, crunching is also an option offered by FooMan Speedster: It removes whitespaces from the output, but when compression is already applied to CSS, this option is less needed.
Use Magento Compiler module
The Magento Compiler module limits the number of directories that PHP has to search through when looking for PHP-files. This decreases the PHP execution-time, which speeds up the Magento application in general.
Be warned that you need to be careful when making changes to your Magento system, while the Magento Compiler is enabled. Upgrades should only be undertaken when the compiler is (temporarily) disabled.
One very neat trick that speeds up things tremenduously is to create a tmpfs-mount specifically for the includes/src folder. Note that this tmpfs-mount needs to be at least 100Mb – preferably 200Mb.
MySQL server tuning
The default MySQL setup is a lot of times sufficient to run a general hosting environment, but not all optimized for Magento. Tuning settings like query_cache_size could dramatically increase performance, but is also dangerous because it hugely depends on other variables (number of databases, number of tables per database, number of queries, peak usage).
Serve static content through a CDN
Static content like images, CSS-stylesheets or JavaScript-files, could be served through other servers that are more optimized for static content. For instance, a CDN could be used so that the static content is always served from a location that is closest to the visitor. This is vital for webshops serving customers worldwide.
Disable local Magento modules
If your site does not need local Magento modules, you could choose to skip the search for local modules alltogether. Within the app/etc/local.xml file, you will find an XML-tag allowing you to do so.
Be carefull with HTTPS
Every time you use SSL between webserver and browser, the process of encrypting and decrypting is added on both sides. Also there is a slight overhead in bandwidth. The Magento site runs slightly faster if you disable SSL for all or just a few pages. However, this “win” is so small compared to the other wins on this page, that it should only be handled with caution. The gained bandwidth is non-vital, while almost all computers nowadays have CPU-power with which the encryption/decryption process takes place in microseconds.
Magento in the cloud
While CDNs could be used to optimize the bandwidth for static content, the Magento application could also be optimized in the same way by using cloud computing.
Memory-based filesystem for dynamic data
By storing dynamic data (var/cache, var/session) on a memory-based filesystem like RAMdisk or tmpfs, the disk I/O is decreased.
Disable Apache htaccess-files
When allowing the usage of htaccess-files, Apache needs to inspect every directory in its path to see if an htaccess-file is present. By moving the Apache configuration-directives from the htaccess-file to the VirtualHost configuration-file, and disabling htaccess-files all together, the Apache execution-time is decreased. This tip probably applies in most cases only to dedicated servers.
Use Nginx or Litespeed
While the Apache webserver is very flexible in its configurations, there are other webservers that are better optimized regarding memory usage: By replacing Apache with either Nginx or Litespeed, you could speed up the Magento scripts even more. Both webservers require manual configuration to allow for SEF URLs.
Use lazyload for images
When a page is loading, a visitor is often waiting for images on that page to load. Depending on the number and size of these images, this can take some time. Instead of loading images at once when the page is loaded, you can also add the LazyLoad JavaScript effect that makes sure only visible images (within the browser screen) are loaded, while remaining images are only loaded once the visitor scrolls down.
Minimize Apache logging
If Apache logging is minimized, less file operations are needed for every incoming request. Ofcourse less logging also means less insight when something goes wrong. An alternative is to optimize the filesystem on which Apache logs are stored. By default, Apache logs to the /var filesystem – but there’s no need to enable things like journalling for that filesystem.
ref: http://magentotutorialbeginners.blogspot.in/2014/05/magento-performance-improvement.html
If you have done all kind of server and caching optimization.Go to code level.
1) See are you loading a collection with in a foreach loop.
2) Try to optimize the code.
3) If you are loading a collection, filter the collection for the required attributes only.
4) Check for your product images.Use png images and try to keep images size under 500 KB.
5) Try commenting the custom functionality that you are providing on that page.And check with some tool like GTMetrix. How much time you achieve to load the page.Try to find out the code which is taking long time to load.
6) Keep only necessary attributes used for filter purpose in Layered navigation.
7) Try disabling unnecessary modules.
8) Try after enabling the compilation.
Hope these suggestions will work for you.
I've been researching this quite a bit and just wanted some professional opinions on this. I am working on an eCommerce site that is really slow for submitting orders. Would creating a web farm be beneficial? If not, what would - server, or network wise (load balancers, etc...)?
Assume the app is optimized as much as can be for now and we need to look at other alternatives.
Environment:
Windows 8 RC 2
IIS 7.5
SQL Server 2008
Ideas
IIS and database on separate server
Load balancers
Maybe we can find a cheaper solution.
If it is only the submit process which is slow perhaps you can just improve the code and use some background workers that won't block the main thread.
Otherwise what about just upgrading your server?
Static file compression.
Create separate application pools for static and dynamic pages with in the website.
Get the heavy feature onto a separate app pool. Check if that helps.
Do optimization on SQL server end. Indexing?
Other than this I would look in to the code. IIS performance is highly dependent upon the pages being executed.
You may also want to check from browser developers tool as to which part of the request is actually clocking the most amount of time. That will give you a better idea of which aspect of performance you should really be concerned about.
Is cloud hosting the way to go? Or is there something better that delivers fast page loads?
The reason I ask is because I run a buddypress site on a bluehost dedicated server, but it seems to run slow at most times of the day. This scares me because at the moment the sites not live and I'm afraid when it gets traffic it'll become worse and my visitors will lose interest. I use Amazon Cloud to handle all my media, JS, and CSS files along with a catching plugin, but it still loads slow at times.
I feel like the problem is Bluehost, because I visit other sites running buddypress and their sites seem to load instantly. Im not web hosting savvy so can someone please point me in the right direction here?
The hosting choice depends on many factors such as technical requirements, growth rates, burst rates, budgets and more.
Bigger Hardware
To scale up hosting operation, your first choice is often just using a more powerful server, VPS, or cloud instance. The point is not so much cloud vs. dedicated but that you simply bring more compute power to the problem. Cloud can make scaling up easier - often with a few clicks.
Division of Labor
The next step often is division of labor. You offload database, static content, caching or other items to specific servers or services. For example, you could offload static content to a CDN. You could a dedicated database.
Once again, cloud vs non-cloud is not the issue. The point is to bring more resources to your hosting problems.
Pick the Right Application Stack
I cannot stress enough picking the right underlying technology for your needs. For example, I've recently helped a client switch from a Apache/PHP stack to a Varnish/Nginx/PHP-FPM stack for a very business Wordpress operation (>100 million page views/mo). This change boosted capacity by nearly 5X with modest hardware changes.
Same App. Different Story
Also just because you are using a specific application, it does not mean the same hosting setup will work for you. I don't know about the specific app you are using but with Drupal, Wordpress, Joomla, Vbulletin and others, the plugins, site design, themes and other items are critical to overall performance.
To complicate matter, user behavior is something to consider as well. Consider a discussion form that has a 95:1 read:post ratio. What if you do something in the design to encourage more posts and that ratio moves to 75:1. That means more database writes, less caching, etc.
In short, details matter, so get a good understanding of your application before you start to scale out hosting.
A hosting service is part of the solution. Another part is proper server configuration.
For instance this guy has optimized his setup to serve 10 million requests in a day off a micro-instance on AWS.
I think you should look at your server config first, then shop for other hosts. If you can't control server configuration, try AWS, Rackspace or other cloud services.
just an FYI: You can sign up for AWS and use a micro instance free for one year. The link I posted - he just optimized on the same server. You might have to upgrade to a small server because Amazon has stated that micro is only to handle spikes and sustained traffic.
Good luck.
I have a Drupal site with a lot of calculations and database requests on each page load (running on an Amazon EC2 server). I am curious how my site would hold up if it became popular or in some other way received heavy traffic. Perhaps the most important thing for me is to locate potential bottlenecks in my code.
What are the best tools for stress testing and finding bottlenecks in a Drupal site? Right now I'm not using any cache module. I've read about the MemCached module and some about Varnish.
Anyone who can share their experiences?
Generally php does not do great in really large scale projects. That is why google does not support php in their app engine.
Facebook which was made in php had to be compiled into c++ for better performance.
Having said that here are the some of the tools ( I have not used them )
http://www.webload.org/
http://xdebug.org/ - to profile your php code besides debugging
cheers,
Vishal
Apache Benchmark.
Send no cookies to simulate anonymous traffic and stress the Varnish caching layer.
Send rando cookie to stress the Drupal caching layer (where you are hopefully using memcache)
Send a login cookie to stress the DB layer.
We run massive Drupal sites on php. Scaling does cost in resources for multiple webheads, database clustering, separate memcache and file servers, but you have to balance that cost against hiring developers for refactoring your code into a different language, and code maintenance.