I'm using Piwik for my web-analytics, and recently I've discovered PHPFog/CloudControl as hosting providers that set up a load balanced, fully managed server for your applications to run on. Piwik requires certain directories to be writable in order to set configuration files, and this prevents me from using load-balancing to enhance my piwik response times.
Does anyone know of an analytics package like piwik (or maybe a different version of piwik) that supports load balancing?
Piwik supports load balancing! See the doc at: http://piwik.org/faq/new-to-piwik/#faq_134
Related
I want to practice performance testing at home using some load testing tool like jmeter. Can anybody tell me some links of websites or applications on which I can practice performance testing by throwing load by load testing tool from home?
It is not a LEGAL way to conduct load/stress test on any live websites/web applications without the permission.
So, you can deploy your own sample application or download sample applications available online and deploy it in your local server like Apache HTTP server or Apache Tomcat etc.
From my knowledge, You can use WebTours sample application from HP LoadRunner as an application to put your load.
Download HP Load Runner community edition from here. free for 50 vusers for life time. You need to create an account in the website.
You can download the webtours application from the HP website (comes with Apache HTTP server, which acts as a Web server).
Setup WebTours as per the instructions here
Confirm the successful installation/configuration by accessing the application from the browser (similar to http://localhost:8080/WebTours). The app is about flight booking (though not in real time ;) ).
Now, you can use the WebTours application as an AUT (Application Under Test)
Either you can continue with Load Runner or download latest JMeter version (3.1 as of now)
JMeter tutorials:
Getting Started
Component Reference
Builiding a Test plan
For Load Runner, there is documentation available in the following link to start with:
http://lrhelp.saas.hpe.com/en/12.53/help/WebHelp/Content/WelcomeContent/c_Welcome.htm
There is community support available managed by HP.
Use the sample applications which ship with the tool
Take your choice of open source application, install them on servers you own, manage and control. Use these applications as targets.
You are welcome to use api.jmeter.ninja. I built it for that purpose. A more formal API declaration is on it''s way but you can start with
http://api.jmeter.ninja/example.html.
http://api.jmeter.ninja/objects.xml
http://api.jmeter.ninja/objects/${OBJECT}.XML
Where ${OBJECT} is taken from the objects.xml page.
Or swap xml for dot json for the same in Jason format.
Exercises/Tutorial is available at http://api.jmeter.ninja/jmeter.pdf
Currently I just use this service for my own training sessions. But I hope to make it more generally available in the near future. There a currently no automatic limits so please just use your common sense and don't run high at throughput for a sustained periods of time. Anything under a total 100k requests is no problem.
Only caveat is that service is provided on a best effort basis at this point. Any abusive/problematic users may be blocked without notice.
I use the node application purely for socket.io channels with Redis PubSub, and at the moment I have it spread across 3 machines, backed by nginx load balancing on one of the machines.
I want to replace this node application with a Phoenix application, and I'm still all new to the erlang/Elixir world so I still haven't figured out how a single Phoenix application can span on more than one machine. Googling all possible scaling and load balancing terms yielded nothing.
The 1.0 release notes mention this regarding channels:
Even on a cluster of machines, your messages are broadcasted across the nodes automatically
1) So I basically deploy my application to N servers, starting the Cowboy servers in each one of them, similarly to how I do with node and them I tie them nginx/HAProxy?
2) If that is the case, how channel messages are broadcasted across all nodes as mentioned on the release notes?
EDIT 3: Taking Theston answer which clarifies that there is no such thing as Phoenix applications, but instead, Elixir/Erlang applications, I updated my search terms and found some interesting results regarding scaling and load balancing.
A free extensive book: Stuff Goes Bad: Erlang in Anger
Erlang pooling libraries recommendations
EDIT 2: Found this from Elixir's creator:
Elixir provides conveniences for process grouping and global processes (shared between nodes) but you can still use external libraries like Consul or Zookeeper for service discovery or rely on HAProxy for load balancing for the HTTP based frontends.
EDITED: Connecting Elixir nodes on the same LAN is the first one that mentions inter Elixir communication, but it isn't related to Phoenix itself, and is not clear on how it related with load balancing and each Phoenix node communicating with another.
Phoenix isn't the application, when you generate a Phoenix project you create an Elixir application with Phoenix being just a dependency (effectively a bunch of things that make building a web part of your application easier).
Therefore any Node distribution you need to do can still happen within your Elixir application.
You could just use Phoenix for the web routing and then pass the data on to your underlying Elixir app to handle the distribution across nodes.
It's worth reading http://www.phoenixframework.org/v1.0.0/docs/channels (if you haven't already) where it explains how Phoenix channels are able to use PubSub to distribute (which can be configured to use different adapters).
Also, are you spinning up cowboy on your deployment servers by running mix phoenix.server ?
If so, then I'd recommend looking at EXRM https://github.com/bitwalker/exrm
This will bundle your Elixir application into a self contained file that you can simply deploy to your production servers (with Capistrano if you like) and then you start your application.
It also means you don't need any Erlang/Elixir dependencies installed on the production machines either.
In short, Phoenix is not like Rails, Phoenix is not the application, not the stack. It's just a dependency that provides useful functionality to your Elixir application.
Unless I am misunderstanding your use case, you can still use the exact scaling technique your node version of the application is. Simply deploy the Phoenix application to > 1 machines and use an Nginx load balancer configured to forward requests to one of the many application machines.
The built in node communications etc of Erlang are used for applications that scale in a different way than a web app. For instance, distributed databases or queues.
Look at Phoenix.PubSub
It's where Phoenix internally has the Channel communication bits.
It currently has two adapters:
Phoenix.PubSub.PG2 - uses Distributed Elixir, directly exchanging notifications between servers. (This requires that you deploy your application in a elixir/erlang distributed cluster way.)
Phoenix.PubSub.Redis - uses Redis to exchange data between servers. (This should be similar to solutions found in socket.io and others)
I got information about haproxy from Stack Exchange Gives Back 2014 page. Stackoverflow is aso using this excellent application. After visiting haproxy website, I found its uses like - load balancer. So,
Does it work like F5 (reverse proxy) and can it replace F5?
Can someone list down its all features and similar competitors application?
You may check this list for alternative Load Balancing tools.
Cloud providers (Amazon, Rackspace, Google Compute Engine, Softlayer etc), but also some dedicated/VM server providers, usually offer some cheap Load-Balancing solutions as a service.
Haproxy currently seems to be one of the most popular opensource software for Reverse-Proxy, Load-Balancing and failover. Latest versions 1.5+ support SSL-offloading but it still doesn't support content caching (F5 does).
The usual configuration is combining it with a Content Caching system or/and CDN and using the last version you now don't even need a separate SSL-offloader. It offers a nice set of load-balancing rules (RR, leastconn, ipHash etc), apart from their website check this nicely-formatted link.
We are working on a mobile portal using Worklight 5.0.6 with the Mobile Web environment.
While loading the application, all the resources like js, css, images etc.. are loaded together. Due to this it takes too much time to load the page. How can we restrict the resources (which are not required) to be loaded together with the application. And how can we force to load only the required resources?
Sounds like what you're asking is:
How to reduce the size of the application
How to reduce the network traffic produced by the application
Indeed, in Worklight 6.0.0.0 there is: the Minification and Concatenation feature as well as application cache (auto-manifest generation). Since you don't have that in 5.0.6, what you could do is:
Regardless of any feature, only keep in your app what you're actually using.
Remove images and libraries you're not using
Use png-crush to minimize the file size of images
Remove dead code
For network traffic:
Add a manifest file to your application and list in it all the resources you'd like to cache as opposed to always fetch. Application Cache is a web standard so you can simply follow articles on the web.
You could "manually" minify your application resources; you can accomplish this using Google Closure.
BE CAREFUL! If you choose this path, make sure to only use the WHITESPACE_ONLY or SIMPLE_OPTIMIZATION compilation levels. Do not use ADVANCED_OPTIMIZATIONS on the Worklight framework files - behavior will be unknown (it may work, it may not, it could suddenly fail at some parts - this is not supported by Worklight).
As for only loading what you want to, ala RequireJS, Worklight is not yet equipped to handle this, so this is not a viable option for you.
There is IMO not much to do in 5.0.6 about this; 6.0.0.0 took it a step forward and the upcoming 6.1.0 handles it better as well (the Mobile Web environment is much lighter in regards to its structure and number of files), but you cannot do moduler loading of resources just yet.
As asked here, I'm looking for a clean solution to setup a staging environment for my
Google App Engine application.
Reading the new spec of Namespaces API, Google suggest that a possible use
of this new feature is:
Creating Separate Datastore Instances for Testing and Production
In case I decide to use Namespaces for testing, could a stress test on Staging affect performance and reliability of my Production application?
Imagine a stress test that crawl and store a thousand of RSS feed in a FeedStored Model with transanctions on FeedStoreCounter and so on; could this activity on a staging namespace cause problem when the application try to do the same operation at the same time on the production namespace?
Also, do the different namespaces share the same quota?
All the data for all App Engine apps is stored in a single Bigtable. As such, data for different namespaces is just as distinct as data between different applications.
If you're also using a separate app version to serve your staging app, the two versions will be running on different app servers, too, so traffic to one version won't affect the other versions.
A stress test should not affect performance. App Engine is meant to scale.
Namespaced data (and app versions) share the same app quota.