ArangoDB Request Load balancing - cluster-computing

I'm currently doing benchmarks for my studies using YCSB on an ArangoDB cluster (v3.7.3), that I set up using the starter (here).
I'm trying to understand if and how a setup like that ( I'm using 4 VMs e.g.) helps with balancing request load?
If I have nodes A,B,C and D and I tell YCSB the IP of node A, all the requests go to node A...
That would mean that a cluster is unnecessary if you want to balance request load, wouldn't it? It would just make sense for data replication.
How would I handle the request load then? I'd normally do that in my application, but I can not do that if I use existing tools like YCSB... (or can I?)
Thanks for the help!

I had this problem as well and ended up solving it by standing-up nginx in front of my cluster, providing a stable, language-independent way to distribute query load. I found nginx surprisingly simple to configure, but take a look at the upstream module for more details.
upstream arangodb {
server coord-1.my.domain:8529;
server coord-2.my.domain:8529;
server coord-3.my.domain:8529;
}
server {
listen *:80 default_server;
server_name _; # Listens for ALL hostnames
proxy_next_upstream error timeout invalid_header;
location / {
proxy_pass http://arangodb;
}
}
It's not ideal for everyone but works well for those times when you just need to load-balance and don't want to write a bunch of code (which ends up being quite slow) to resolve coordinators.
I've asked ArangoDB for a native proxy server solution, but I would bet it's low on their to-do list as it could be tricky to support, given the huge number of configuration options.

Related

Rate Limit/Throttling with Tomcat (and Spring REST)

I read a lot of threads now, still my problem could not be solved sufficiently:
If running a tomcat webserver with a spring rest backend, there should must be a way to limit the possible requests per seconds/minute/... based on, lets say, the IP of a requestor.
My investigations led to the following possibilites so far:
Use Guava RateLimiter or https://github.com/weddini/spring-boot-throttling and check all requests in the preHandle. But since this does not take into account, which IPs at what time requested, something like a REDIS store would make more sense to check against (IP/Last access timestamp)
Put a more advanced web server in front of tomcat which offers this functionality (e.g. apache2 or nginx)
Now I don't like the first solution, since the requests already hit the application itself and the second solution builds up an additional layer, which I can't really believe is necessary for such a basic problem..
So my question is, what methods and solutions am I missing here? I read something about tomcat valve semaphore, but it seems to just limit the overall rate of requests.
Would it be most efficiently/possible to already filter with some basic functionality like iptables or fail2ban on the 8443 port and simply drop requests by the same ip in a give time frame?

How to properly scale Jelastic app servers horizontally

I have several stateless app servers packed into Docker containers. I have a lot of load on top of them and I want to horizontally scale this setup. My setup doesn't include load balancer nodes.
What I've done is simply increased nodes count — so far so good.
From my understanding Jelastic have some internal load balancer which decides to which node it should pass incoming request, e.g.:
user -> jelastic.my-provider.com -> one of 10 of app nodes created.
But I've noticed that lot of my nodes (especially last ones) are not receiving any requests, and just idling, while first nodes receive lion share of incoming requests (I have lot of them!). This looks strange for me, because I thought that internal load balancer doing some king of round-robin distribution.
How to setup round-robin balancing properly? I came to the conclusion that I have to create another environment with nginx/haproxy and manually add all my 10 nodes to list of downstream servers.
Edit: I've setup separate HAProxy instance and manually added all my nodes to haproxy.cfg and it worked like a charm. But the question is still open since I want to achieve automatic/by schedule horizontal scaling.
Edit2: I use Jelastic v5.3 Cerebro. I use custom Docker images (btw, I have something like ~20 envs which all about of custom images except of databases).
My topology for this specific case is pretty simple — single Docker environment with app server configured and scaled to 10 nodes. I don't use public IP.
Edit3: I don't need sticky sessions at all. All my requests came from another service deployed to jelastic (1 node).

Using Azure load balancer to reboot/update server with zero downtime

I have a really simple setup: An azure load balancer for http(s) traffic, two application servers running windows and one database, which also contains session data.
The goal is being able to reboot or update the software on the servers, without a single request being dropped. The problem is that the health probe will do a test every 5 seconds and needs to fail 2 times in a row. This means when I kill the application server, a lot of requests during those 10 seconds will time out. How can I avoid this?
I have already tried running the health probe on a different port, then denying all traffic to the different port, using windows firewall. Load balancer will think the application is down on that node, and therefore no longer send new traffic to that specific node. However... Azure LB does hash-based load balancing. So the traffic which was already going to the now killed node, will keep going there for a few seconds!
First of all, could you give us additional details: is your database load balanced as well ? Are you performing read and write on this database or only read ?
For your information, you have the possibility to change Azure Load Balancer distribution mode, please refer to this article for details: https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode
I would suggest you to disable the server you are updating at load balancer level. Wait a couple of minutes (depending of your application) before starting your updates. This should "purge" your endpoint. When update is done, update your load balancer again and put back the server in it.
Cloud concept is infrastructure as code: this could be easily scripted and included in you deployment / update procedure.
Another solution would be to use Traffic Manager. It could give you additional option to manage your endpoints (It might be a bit oversized for 2 VM / endpoints).
Last solution is to migrate to a PaaS solution where all this kind of features are already available (Deployment Slot).
Hoping this will help.
Best regards

mod_jk vs mod_cluster

Can someone please tell me the pro's and con's of mod_jk vs mod_cluster.
We are looking to do very simple load balancing.. We are going to be using sticky sessions and just need something to route new requests to a new server if one server goes down. I feel that mod_jk does this and does a good job so why do I need mod_cluster?
If your JBoss version is 5.x or above, you should use mod_cluster, it will give you a better performance and reliability than mod_jk. Here you've some reasons:
better load balacing between app servers: the load balancing logic is calculated based on information and metrics provided directly by the applications servers (bear in mind they have first hand information about its load), in contrast with mod_jk with which the logic is calculated by the proxy itself. For that, mod_cluster uses an extra connection between the servers and the proxy (a part from the data one), used to send this load information.
better integration with the lifecycle of the applications deployed in the servers: the servers keep the proxy informed about the changes of the application in each respective node (for example if you undeploy the application in one of the nodes, the node will inform the proxy (mod_cluster) immediately, avoiding this way the inconvenient 404 errors.
it doesn't require ajp: you can also use it with http or https.
better management of the servers lifecycle events: when a server shutdowns or it's restarted, it informs the proxy about its state, so that the proxy can reconfigure itself automatically.
You can use sticky sessions as well with mod cluster, though of course, if one of the nodes fails, mod cluster won't help to keep the user sessions (as it would happen as well with other balancers, unless you've the JBoss nodes in cluster). But due to the reasons given above (keeping track of the server lifecycle events, and better load balancing mainly), in case one of the servers goes down, mod cluster will manage it better and more transparently to the user (the proxy will be informed immediately, and so it will never send requests to that node, until it's informed that it's restarted).
Remember that you can use mod_cluster with JBoss AS/EAP 5.x or JBoss Web 2.1.1 or above (in the case of Tomcat I think it's version 6 or above).
To sum up, though your use case of load balancing is simple, mod_cluster offers a better performance and scalability.
You can look for more information in the JBoss site for mod_cluster, and in its documentation page.

Programmatically add nodes to a load balancer like Haproxy?

I'm very sure this problem has been solved, but I can't find any information anywhere about it...
How do sysadmins programmatically add a new node to an existing and running load balancer ? Let's say I have a load balancer running and already balancing say my API server between two EC2 instances, and suddenly there's a traffic spike and I need a third node in the load balancer but I'm asleep... It would be wonderful if I had something monitoring probably RAM usage and some key performance indicators that tell me when I should have another node, and even better if it could add a new node to the load balancer alone...
I'm confident that this is possible and even trivial to do with node-http-proxy and distribute, but I'd like to know if this is possible to do with HAproxy and/or Nginx... I know Amazon's elastic load balancing is probably my best bet but I want to do it on my own (I want to spawn instances from rackspace, EC2, Joyent and probably others as it's convenient).
Once again, spawning a node is easy, I'd like to know how to add it to haproxy.cfg or something similar with Nginx without having to reload the whole proxy, and doing that programatically. Bash scripting is my best bet for this but it still does have to reload the whole proxy which is bad because it loses connections...
You have a few questions in there. For the "add nodes to haproxy without restarting it":
What I do for a similar problem is prepopulate the config file with server names.. e.g. web01, web02 ... web20 even if I only have 5 web servers at the time. Then in my hosts file I map those to the actual ips of the web servers.
To add a new server, you just create an entry for it in the hosts file and it will start passing health checks and get added.
For automated orchestration, it really depends on your environment and you'll probably have to write something custom that fits your needs. There are paid solutions (Scalr comes to mind) to handle orchestration too.
What I do: I have a line in my backend section in haproxy.cfg which says:
# new webservers here
And with a sed script I update haproxy.cfg with something like:
sed -i -e "/new webservers here/a\ server $ip_address $ip_address:12080 check maxconn 28 weight 100"
And then reload haproxy. Works transparently.
HAProxy has a Runtime API that allows you to do just that dynamically.
Please read the official documentation:
Dynamic Configuration HAProxy Runtime API

Resources