We increased our products from 12k to 50k.
Now we're faceing a huge loading time issue. Sites are loading over 1-2 minutes. It seems like the first ttfb takes much to long and our developers can't find a solution/reason for that. someone knows a solution for this or someone who eveb could fix it? (paid)
Loading Time in Screenshot
Slow example url would be: https://gear2game.ch/unterhaltungselektronik/tv-home-cinema/tv.html
Also good to know, the homepage is fast, but all categories and products are slow :(
Best regards
Sandro
When the homepage is fast, but categories and products are slow, that is frequently an indication that way too much data is being loaded somewhere in a custom module, or you're not making use of indexers (which are a necessity with 50k products).
Slow ttfb also indicates that it's an issue on your backend (and given you just added a bunch of products, I'd say it's because some of the settings in Magento to handle large amounts of data are not enabled).
These are the things I'd start with:
Have you reindexed?
Are you using flat tables?
Are your caches enabled?
Are you using the Magento swatch module? This is notoriously bad with many products.
With a 50K products you should really think about Varnish.
Here are some useful links :
https://varnish-cache.org/
https://www.magentocommerce.com/magento-connect/turpentine-varnish-cache.html
Related
Over the past couple of months, I've been on a rampage optimising a Joomla website that I'm managing. When I first started, the homepage used to open in around 30-40 seconds, in spite of repeatedly upgrading my dedicated server, as suggested by the hosting firm.
I was able to bring the pagespeed down to around 800ms by religiously following all the recommendations of the likes of GT Matrix and PingdomTools, (such as using JCH-optimize, .htaccess caching and compression settings, and MaxCDN) but now I'm stuck optimising my my.cnf settings, trying various settings suggested on a number of related articles. The fastest I'm getting the homepage to open - with the current settings - is 777ms after refresh, which might not sound too bad, but look at the configuration of my dedicated server:
2 Quads, 128GB, 2x480GB SSD RAID
CloudLinux/Cpanel/WHM
Apache/suEXEC/PHP5/FastCGI
MariaDB 10.0.17 (all tables converted to XtraDB/InnoDB)
The site traffic is moderate, 10,000 and 20,000 visitors per day, with around 200,000 pageviews.
These are the current my.cnf settings. My goal is to bring the pagespeed down to under 600ms, which should be possible with this kind of hardware, provided it is tuned the right way.
[mysqld]
local-infile=0
max_connections=10000
max_user_connections=1000
max_connect_errors=20
key_buffer_size=1G
join_buffer_size=1G
bulk_insert_buffer_size=1G
max_allowed_packet=1G
slow_query_log=1
slow_query_log_file="diskar/mysql-slow.log"
long_query_time=40
connect_timeout=120
wait_timeout=20
interfactive_timeout=25
back_log=500
query_cache_type=1
query_cache_size=512M
query_cache_limit=512K
query_cache_min_res_unit=2K
sort_buffer_size=1G
thread_cache_size=16
open_files_limit=10000
tmp_table_size=8G
thread_handling=pool-of-threads
thread_stack=512M
thread_pool_size=12
thread_pool_idle_timeout=500
thread_cache_size=1000
table_open_cache=52428
table_definition_cache=8192
default-storage-engine=InnoDB
[innodb]
memlock
innodb_buffer_pool_size=96G
innodb_buffer_pool_instances=12
innodb_additional_mem_pool_size=4G
innodb_log_bugger_size=1G
innodb_open_files=300
innodb_data_file_path=ibdata1:400M:autoextend
innodb_use_native_aio=1
innodb_doublewrite=0
innodb_user_atomic_writes=1
innodb_flus_log_at_trx_commit=2
innodb_compression_level=6
innodb_compression_algorithm=2
innodb_flus_method=O_DIRECT
innodb_log_file_size=4G
innodb_log_files_in_group=3
innodb_buffer_pool_instances=16
innodb_adaptive_hash_index_partitions=16
innodb_thread_concurrency
innodb_thread_concurrency=24
innodb_write_io_threads=24
innodb_read_io_threads=32
innodb_adaptive_flushing=1
innodb_flush_neighbors=0
innodb_io_capacity=20000
innodb_io_capacity_max=40000
innodb_lru_scan_depth=20000
innodb_purge_threads=1
innodb_randmon_read_ahead=1
innodb_read_io_threads=64
innodb_write_io_threads=64
innodb_use_fallocate=1
innodb_use_atomic_writes=1
inndb_use_trim=1
innodb_mtflush_threads=16
innodb_use_mfflush=1
innodb_file_per_table=1
innodb_file_format=Barracuda
innodb_fast_shutdown=1
I tried Memcached and APCU, but it didn't work. The site actually runs 2-3 times faster with 'Files' as the caching handler in Joomla's Global Configuration. And yes, I ran my-sqltuner, but that was of no help.
I am newby as far as Linux is concerned and suspect that above settings could be improved. Any comments and/or suggestions?
long_query_time=40
Set that to 1 so you can find out what the slow queries are.
max_connections=10000
That is unreasonably high. If you come anywhere near it, you will have more problems than failure to connect. Let's say only 3000.
query_cache_type=1
query_cache_size=512M
The Query cache is hurting performance by being so large. This is because any write causes all QC entries for the table to be purged. Recommend no more than 50M. If you have heavy writes, it might be better to change the type to DEMAND and pepper your SELECTs with SQL_CACHE (for relatively static tables) or SQL_NO_CACHE (for busy tables).
What OS?
Are the entries in [innodb] making it into the system? I thought these needed to be in [mysqld]. Check by doing SHOW VARIABLES LIKE 'innodb%';.
Ah, buggers; a spelling error:
innodb_log_bugger_size=1G
innodb_flus_log_at_trx_commit=2
inndb_use_trim=1
and more??
After you get some data in the slowlog, run pt-query-digest, and let's discuss the top couple of queries.
Have problem with loading homepage of my site. PHP execution time is about 14sec !!! Other pages like categories have 1,5sec to execution. Cache are enabled also I clean DB with no results.
The cache being enabled doesn't help if the blocks you have on your homepage aren't setup to be cached. If you have any custom blocks generating content on your homepage you should set the cache lifetime and override the getCacheKeyInfo. Especially if these blocks are doing large DB queries involving joins across many EAV tables. Obviously there's not much anyone can do with the limited information you've given, but start by looking at your homepage layout XML to see which blocks are being used, and investigate whether they are being cached.
Install FPC.
Pre-warm Cache.
Make sure if any listing then flat tables are used.
No block level code in .phtml
In short you need to know how to optimise magenta for performance
Solved problem with Full Page Cache Thanks!
I saw #parscale tweet: How many queries are you happy with for a home page? When do you say this is Optimized?
I saw responses that < 50 is good, 30 or less is best, and 100+ is danger zone. Is there really any proper number? And if say you do have > 50 queries running on your pages, what are some ways to bring it down?
I generally have sites that run the gamut that are under 50 queries and some more, though the "more" don't seem to be too slow, I'm always interested in making it faster. How?
How to reduce queries will vary from site to site, template to template, but there's been a few articles on EE optimisation and performance:
http://expressionengine.com/wiki/Reduce_Queries/
http://expressionengine.com/blog/entry/troubleshooting_site_performance_issues/
http://www.netmagazine.com/tutorials/optimise-your-expressionengine-site
http://www.leezilla.net/post/12377053779/ab-seeing-your-sites-performance
http://eeinsider.com/articles/using-cache-wisely-with-expressionengine/
But if you've done all that and still need to speed things up, then your next step is to look at add-ons like CE Cache.
Thing to remember is not all queries are created equal. You can have 1,000 queries that do very little in the way of impacting performance, or a single query that can slow everything way down.
In EE its actually better to look at the template debug output and identify key slow down spots in the template build then to always focus on just the query count.
As others have pointed out products like CE Cache, Solspace's Template Morsels, or even adding a varnish caching server in-front of an intensive EE web site can do wonders, though with the added work required to fully get a varnish setup in front of EE setup, I would currently stick to the other solutions/directions first.
There is not a magic query number. In my opinion, your server environment dictates what can be supported. The more resources you have, the more complex your code can be.
With that said, there are lots of options you can use if issues do arise on an EE website. The links in the answer above give you a solid list but here are some first things to check:
Remove search:field_name="" parameters
Reduce use of channel tags, combine if you can
Add disable="" parameter to channel tabs to disable what you don't need
Reduce use of embeds
Turn off all EE tracking code
Stop using advanced conditionals if you have a channel tag inside
Following on from Nevin's point. I find that the JB Graphite is a huge help, it turns the debug output into a pretty graph, so you can easily spot bottleneck queries.
http://devot-ee.com/add-ons/jb-graphite
I'll expand on MediaGirl's point number 6 - you can often greatly simplify conditionals by using Croxton's Ifelse and/or Switchee add-ons. Definitely worth a look.
I used CE Cache on a really intensive build and it reduced page load from 6 seconds to 0.7 seconds. Awesome addpon, with incredible documentation and the best support you can get anywhere.
I'm currently using CakePHP for my training plan application but I have rendering times of 800 ms per page.
Do you have any tips on improving the performance?
Thanks in advance.
br, kms
In general the tool that will give you the greatest speed boost is caching, and CakePHP has a dedicated helper. The second most useful thing to do is optimizing the queries, only asking for the data you need - this can be done with the containable behavior.
You can find more detailed tips in this post.
Install APC, if you dont have it. that will instantly make it < 500ms. also without touching a single line of code.
Make sure your tables have all the proper indexes so that queries are as fast as they can be.
Next up look at some things about caching routes / urls as that is a huge drain.
Those will give you the most speed boost for the least amount of work
Have you tried any of the CSS/JS asset combiners for CakePHP? They combine/compress/minify your CSS/JS scripts and cache them where applicable. Here's one that's quite recent.
Not specific to CakePHP but you could go through all the factors in Google Page Speed, it will help you speed up your page loading times by suggesting what scripts you could combine and advice on how to reduce requests.
Other than that, look into the containable behaviour, see if you can cut out any necessary info/queries by only selecting what you need at any time.
This question is well-populated with information on speeding Cake up.
We have website, which is based on Drupal. There is a ~30 modules, which is not huge amount for our VPS. We have no high traffic, so traffic doesn't making Site overload.
On same VPS we have other Sites which are loading properly.
Site: http://jnews.am
Where I can start? How can I check what part of my server/website is causing performance issues? What investigating methods you can suggest?
You need to dig in to find out the bottle-neck. Here is a good article about it. http://www.mindyourcode.com/php/drupal-profiling-performance-analysis-for-optimization-finding-the-performance-bottlenecks-in-your-application-at-core-level/
Good question. Here's some answers:
1) Downlaod the YSlow extension to Firefox, and install it. This will let you test on a number of different items that can suggest why your site may be slow. This doesn't currently look to be your current big problem right now, though.
2) Install the Firebug extension to firefox. The 'Net' tab of firebug tells you how long every document took to download. your core page is taking 5 seconds, and for some reason system.css is taking almost as long, which is unusual in that it's a static file.
3) Check and see if you've got any slow queries, and why. Assuming you're using mysql, this page http://dev.mysql.com/doc/refman/5.0/en/slow-query-log.html tells you how to set up the slow query log, which will collect and report which queries are taking a long time to complete.
Beyond that, some suggestions: You're almost certainly not using some of Drupal's performance options, such as caching, and I would suggest using memcache to speed the site up as well. (See http://drupal.org/project/memcache)
It really looks like you've got a query that's taking too long. It seems to me that the slow query log is what's going to be the most useful tool - it'll tell you where you need to optimize your site. Do note that mysql tends to use the first index on a table that it finds in the WHERE clause instead of the fastest, and as such a query like "WHERE type='story' AND status = 1" is faster than "WHERE status = 1 AND type='story'", because there type index filters the data better than the status one. (And Views tends to put the items in the where clause in the same order that they're in the Filter section.)