After implementation of varnish cache on apache server, sessions are reduced to half on google analytics. Since it is JS code that runs on client side, shouldn't be affecting this. What could be the issue?
Related
I suddenly got a spike in CPU usage on shared hosting and my website has been taken offline for the past 48 hours. I have already tried the following, and it did not help (or only a bit):
Updated Drupal 7 and all modules to the latest version
activated CloudFlare
disabled any unused modules
Caching was already on (min cache lifetime: none; expiration cached pages: 1 hour; bandwidth optimization: all options checked). I had no problems before, and visitor numbers have remained fairly constant at 1500-2000 a day. Could it be hacked? Spam? The logs only show lots of Google bot "access denied" activity, but somehow doubt that can take the site down.
Thanks in advance for any advice!
Use boost module, this will help you improve the performance by creating HTML files.
Boost provides static page caching for Drupal enabling a very
significant performance and scalability boost for sites that receive
mostly anonymous traffic. For shared hosting this is your best option
in terms of improving performance. On dedicated servers, you may want
to consider Varnish instead.
Apache is fully supported, with Nginx, Lighttpd and IIS 7
semi-supported. Boost will cache & gzip compress html, xml, ajax, css,
& javascript. Boosts cache expiration logic is very advanced; it's
fairly simple to have different cache lifetimes for different parts of
your site. The built in crawler makes sure expired content is quickly
regenerated for fast page loading.
We are moving an asp.net site to Azure Web Role and Azure Sql Database. The site is using output cache and normal Cache[xxx] (i.e. HttpRuntime.Cache). These are now stored in the classic way in the web role instance memory.
The low hanging fruit is to first start using a distributed cache for output caching. I can use in-role cache, either as co-located or with a dedicated cache role, or Redis cache. Both have outputcache providers ready made.
Are there any performance differences between the two (thee with co-located/dedicated) cache methods?
One thing to consider is that will getting the page from Redis for every pageload on every server be faster or slower than composing the page from scratch one every server every 120 seconds but inbetween just getting it from local memory?
Which will scale better when we want to start caching our own data (i.e. pocos) in a distributed cache instead of HttpRuntime.Cache?
-Mathias
Answering to your each question individually:
Are there any performance differences between the two (thee with
co-located/dedicated) cache methods?
Definately co-located caching solution is faster than dedicated cache server, as in co-located/inproc solution request will be handled locally within the process where as dedicated cache solution will involve getting data over the network. However since data will be in-memory on cache server, getting will still be faster than getting from DB.
One thing to consider is that will getting the page from Redis for
every pageload on every server be faster or slower than composing the
page from scratch one every server every 120 seconds but inbetween
just getting it from local memory?
It will depend on number of objects on page i.e. time taken to compose the page from scratch. Though getting from cache will involve network trip time but its mostly in fractions of a millisecond.
Which will scale better when we want to start caching our own data
(i.e. pocos) in a distributed cache instead of HttpRuntime.Cache?
Since HttpRuntime.Cache is in-process caching, it is limited to single process's memory therefore it is not scalable. A distributed cache on the other hand is a scalable solution where you can always add more servers to increase cache space and throughput. Also out-proc nature of distributed cache solution makes it possible to access data cached by on application process to be used by any other process.
You can also look into NCache for Azure as a distributed caching solution. NCache is a native .Net distributed caching solution.
Following blog posts by Iqbal Khan will help you better understand the need of distributed cache for ASP.Net applications:
Improve ASP.NET Performance in Microsoft Azure with Output Cache
How to use a Distributed Cache for ASP.NET Output Cache
I hope this helps :-)
-Sameer
I am trying to reduce the latency on site goldealers.co.uk
The site appears to have a latency of anywhere between 950ms and 1500ms.
I have checked:
Processes
RAM usage
HTTP connections
Ping
Removing ALL plugins
Removing plugins doesn't make the slightest bit of difference.
The server is a VPS Cloud Server with dedicated 1.5ghz processor and 1GB RAM.
My question:
Is latency a server / programming problem?
Do wordpress sites generally have a high latency?
I have checked the latency on Forbes.com (a wordpress site) - This only has a latency of 151ms!!!
I will soon be working on caching, adding expires headers, possibly using a CDN for images etc... but to be honest, there is no point if it takes over 1 second to even start to return any data.
Any advice that you can provide is much appreciated.
Your analysis and priority are correct - starting with the base page load time first, then later optimizing the remaining front-end components.
In general WordPress sites by default can be a bit slow to deliver the HTML pages. Times in the range you mentioned 1-1.5 seconds are not uncommon. (For comparison, an unoptimized WordPress site I run is in the 1-3 second range.)
I would look into two areas:
Basic speed on that host
Database query speed
It could be that your webhost does not have a very fast connection. You can test this (and eliminate the WordPress part of the equation) by fetching a static file. On your site, for example, I can pull the robots.txt file down in about 0.3 seconds. The speed to serve a static file is about your minimum baseline.
Next I would look at the MySQL database query speed. Is MySQL being served on the same host or a different one? The Debug Queries plugin can show you the exact queries being made and performance for each. If the DB queries appear to be the problem, the DB Cache Reloaded plugin can sometimes be helpful. It adds an additional layer of caching for frequent DB calls.
There are also some good suggestions in the answers to this SO question: How can I figure out why my site pages load so slowly?
Your latency is almost certainly a server-related issue. You said you have a VPS and most VPS installations come with all Apache modules enabled - all of which you DO NOT NEED for Wordpress.
Eliminating all of the modules you don't need reduces how much memory each PHP instance will consume.
I've answered this question here on stack overflow: How can I figure out why my Wordpress pages load so slowly?
When I took a look at your site I saw that a lot of time is being killed on Facebook widgets. Testing from different locations around the world, looks like you are losing 2-3 seconds just for the facebook widgets. Drop those and you will have a much faster site.
I am using onapp api in my website and in a page it is fetching all the servers in onapp. For some users this list is very large and it extends upto thousand in some cases. The response is not only data but it contains other information too. Also I am doing a pagination. So for each api has to be called and data should be populated. Now for increasing the speed of this I am writting the response to a file and reading from it. But it also taking time. Is there anyway to spped up this operation.
Before file caching, each page was taking around 45 seconds and now it is reduced to 25. But this is also a high value . I am using Symfony Framework. I am using the following code for caching data to file.
$userStatisticsCached=unserialize(file_get_contents($filePath));
if(is_null($userStatisticsCached)||$userStatisticsCached==false){
$userStatistics = $statisticsInstance->getList(1);
file_put_contents($filePath, serialize($userStatistics));
}
else {
$userStatistics=$userStatisticsCached;
}
Is there any better method for achieving the same output with less loading time ?
First: 45seconds is a lot to load a page. How many API calls are you making?
Second: Whoa, 25seconds when having all API calls already cached in filesystem is absolutely huge, too. How many filesystem lookups does your page load perform? Are you sure all your API requests are cached when measuring the 25s page load?
In-Memory Caching:
Depending on the size of your data, I would certainly suggest storing your cached data in memory to speed up cache lookups. For caches around 1GB or less it shouldn't be an issue (depending on what server hardware/hosting provider you are running on). An excellent first choice is Memcache, which also happens to have good PHP support.
Running Memcache: When working locally on your computer you should have no problem installing memcached for yourself. When you upload your website to the server, you'll either need to ensure that Memcache is running on the same server, or ask your server hosting provider for details on how to connect to their memcache server. Most PHP hosting providers offer Memcache as part of the hosting. If they don't, you can use a hosted remote memcache provider like MemCachier, although again the latency to a remote server is going to slow down your cache lookups.
The previous 2 Heroku application stacks came with a Varnish layer which automatically reverse-proxy-cached content based on http headers.
The new Heroku cedar stack doesn't have this Varnish layer. Heroku suggests using rack-cache and memcache instead.
Does this have disadvantages compared to the previous stacks with the varnish layer? With rack-cache, aren't there fewer servers serving the caching layer, and in a less optimized way?
http://devcenter.heroku.com/articles/http-routing
http://devcenter.heroku.com/articles/http-caching
http://devcenter.heroku.com/articles/cedar
There is absolutely no question that removing the varnish layer is a huge downgrade in cache performance -- both latency and throughput -- from Heroku's bamboo to cedar stack. For one, your application requests are competing with, and may be queued behind, cache hits for dyno time.
The disadvantages, to name a few, are: interpreted ruby (vs. compiled C) application level (vs. proxy level) memcached based (vs. in process memory based) blocking I/O based (vs. nonblocking I/O based). Any suggestion that the two caching strategies could compete at scale is ridiculous. You won't see much difference on a small scale. But if you have hundreds or even thousands of cache hits per second, varnish will not substantially degrade, while the cedar stack would require many dynos just to serve static assets in a performant fashion. (I'm seeing ~ 5-10ms dyno processing time for cache hits on cedar).
Cedar was built the way it was built not to improve performance, but to introduce new levels of flexibility over your application. While it allows you to do non blocking I/O operations like long polling, and fine grained control over static asset caching parameters, it's clear that heroku would like you to bring your own caching/bandwidth if your goal is internet scale.
I don't know how the rack-cache performance compares to Varnish in terms of raw requests. The best bet would be to create a simple app benchmark it and switch stacks.
It's worth bearing in mind that because the heroku.com stack was Nginx->Varnish->App as long as you've set the correct HTTP headers you're App layer will be doing much less work. As most of the delivery will be handled by Varnish, as well as Varnish being pretty fast, this frees up your Dyno for actual app-handling-requests.
With the herokuapp.com stack hitting your app earlier, it's down to you and your app to handling caching efficiently, this might mean you choose to use rack-cache to cache full page output or you might choose to use memcached to deal with database requests or both.
In the end it depends on what your app is doing, if it's serving the same static content to a lot of users then you'd benefit from Varnish, but if you've got an application where users login and interact with content then you won't see might benefit from Varnish so caching partial content or raw database queries might be more efficient. If you install the New Relic addon you'll be able to take a peek under the hood and see what's slowing your app down.
There are also 3rd party options for hosted Varnish. I wrote a quick post about setting it up with Fastly/Varnish.
Fastly is a hosted Varnish solution that will sit in front of your Heroku app and serve cached responses.
Update link:
https://medium.com/#harlow/scale-rails-with-varnish-http-caching-layer-6c963ad144c6
I've seen really great response times. If you can get a good cache hit-rate with Varnish you should be able to throttle back a good percentage of your dyno's.
A more modern answer, with 20/20 hindsight:
To get the caching performance approaching that of bamboo on cedar-14, this is the usual pattern:
Configure Rails to generate appropriate caching headers (i.e., Etags, Cache-Control, Last-Modified, etc.)
Stick fastly (varnish as a service) or cloudflare in front of your app. If app headers were set correctly, profile-like pages that need to be fresh won't be cached, as opposed to static assets.
I would recommend redis-rails as a rack cache backend, if you're doing caching at multiple layers (FE (CF/FY), page, action, fragment, etc.).