I am new to caching mechanism and just started learning about Hazelcast. I gone through couple of tutorials and hazelcast site but still I am not clear.
I am trying to build a caching for my springboot & angular application. It is a single standalone application.
So in my case, since my application single and no plan in running as multiple instance can I just go with Hazelcast member without client. Is client is needed?
No, the client is not mandatory, and for your case it would seem unnecessary.
The idea is around abstraction, you ask Hazelcast for item X and it is returned if it exists. Hazelcast works out where that item is held, and mostly this is hidden from you.
X could be found in your process:
Your process is a client, has near-caching active, and has a copy.
Your process is one of 1 or more servers, and happens to be the server responsible for storing item X.
X could be found in another process:
Your process is a client, has no near-caching, so is not storing anything
Your process is one of several servers, and it happens that one of the other servers is responsible for item X.
"Mostly this is hidden from you" == There will be a retrieval time difference between data found in the same process and data retrieved from another process, as it has to pass across the network. If this is a significant difference at low volumes, it's time to upgrade the network.
Related
We are building a reporting app on Laravel that need to fetch users data from a third-party server that allow 1 request per seconds.
We need to fetch 100K to 1000K rows based on user and we can fetch max 250 rows per request.
So the restriction is:
1. We can send 1 request per seconds
2. 250 rows per request
So, it requires 400-4000 request/jobs to fetch a user data, So, loading data for multiple users is very time-consuming and the server gets slow.
So, now, we are planning to load the data using multiple servers, like 4-10 servers to fetch users data, so we can send 10 requests per second from 10 servers.
How can we design the system and process jobs from multiple servers?
Is it possible to use a dedicated server for hosting Redis and connect to that Redis server from multiple servers and execute jobs? Can any conflict/race-condition happen?
Any hint or prior experience related to this would be really helpful.
The short answer is yes, this is absolutely possible and is something I've implemented in production apps many times before.
Redis is just like any other service and can run anywhere, with clients from anywhere, connecting to it. It's all up to your configuration of the server to dictate how exactly that happens (and adding passwords, configuring spiped, limiting access via the firewall, etc.). I'd reccommend reading up on the documentation they have in the Administration section here: https://redis.io/documentation
Also, when you do make the move to a dedicated Redis host, with multiple clients accessing it, you'll likely want to look into having more than just one Redis server running for reliability, high availability, etc. Redis has efficient and easy replication available with a few simple configuration commands, which you can read more about here: https://redis.io/topics/replication
Last thing on Redis, if you do end up implementing a master-slave set up, you may want to look into high availability and auto-failover if your Master instance were to go down. Redis has a really great utility built into the application that can monitor your Master and Slaves, detect when the Master is down, and automatically re-configure your servers to promote one of the slaves to the new master. The utility is called Redis Sentinel, and you can read about that here: https://redis.io/topics/sentinel
For your question about race conditions, it depends on how exactly you write your jobs that are pushed onto the queue. For your use case though, it doesn't sound like this would be too much of an issue, but it really depends on the constraints of the third-party system. Either way, if you are subject to a race condition, you can still implement a solution for it, but would likely need to use something like a Redis Lock (https://redis.io/topics/distlock). Taylor recently added a new feature to the upcoming Laravel version 5.6 that I believe implements a version of the Redis Lock in the scheduler (https://medium.com/#taylorotwell/laravel-5-6-preview-single-server-scheduling-54df8e0e139b). You can look into how that was implemented, and adapt for your use case if you end up needing it.
I'm trying to get to grips with service fabric and I'm struggling a little bit. Some questions:
are all service fabric service instances single-threaded? I created a stateless web api, one instance, with a method that did a Task.Delay, then returned a string. Two requests to this service were served one after the other, not concurrently. So am I right in thinking then that the number of concurrent requests that can be served is purely a function of the service instance count in the application manifest? Edit Thinking about this, it is probably to do with the set up of OWIN Wep Api. Could it be it is blocking by session? I assumed there is no session by default?
I have long-running operations that I need to perform in service fabric (that can take several hours). Is there a recommended pattern that I can use for this in service fabric? These are currently handled using a storage queue that triggers a webjob. Maybe something with Reliable Queues and a RunAsync loop?
It seems you handled the first part so I will comment on the second part: "long-running operations".
We can see long running operations / workflows being handled far before service fabric came about. For this reason, we can build on the shoulders of giants by looking on the design patterns that software experts have been using for decades. For example, the famous and all inclusive Process Manager. Mind you that this pattern is sometimes an overkill. If it is in your case, just check out the rest of the related patterns in the Enterprise Integration Patterns book (by Gregor Hohpe).
As for the use of reliable collections, those are implementation details when choosing a data structure supporting the chosen design pattern.
I hope that helps
With regards to your second point - It really depends on the nature of your long running task.
Is your long running task the kind of workload that runs on an isolated thread that depends on local OS/VM level resources and eventually comes back with a result (A)? or is it the kind of long running task that goes through stages and builds up a model of the result through a series of persisted state changes (B)?
From what I understand of Service Fabric, it isn't really designed for running long running workloads (A), but more for writing horizontally-scalable, highly-available systems.
If you were absolutely keen on using service fabric (and your kind of workload tends to be more like B than A) I would definitely find a way to break down those long running tasks that could be processed in parallel across the cluster. But even then, there is probably more appropriate technologies designed for this such as Azure Batch?
P.s. If you are going to put a long running process in the RunAsync method, you should design the workload so it is interruptable and its state can be persisted in a way that can be resumed from another node in the cluster
In a stateful service, only the primary replica has write access to
state and thus is generally when the service is performing actual
work. The RunAsync method in a stateful service is executed only when
the stateful service replica is primary. The RunAsync method is
cancelled when a primary replica's role changes away from primary, as
well as during the close and abort events.
P.s.s Long running operations are the devil when trying to write scalable systems. Try and tackle that now and save yourself the future pain if possibe.
To the first point - this is purely a client issue. Chrome saw my requests as indentical and so delayed the 2nd request until the 1st got a response. Varying the parameter of the requests allowed them to be served concurrently.
To be more specific, I'm studying sessions, and I'm reading about the <distributable> tag in the deployment descriptor (for example). The text states,
"...it is possible - for the sake of load balancing of fail-over or both - to mark a web application as distributable, if it supported by your application server."
Can someone provide a little more info/context? If possible, I don't need a full background on how the mechanism works (I'm studying for the Web Components exam), just enough to understand in the context of sessions.
Thanks!
Here are some useful lines,
If an application is run in a cluster without being marked as distributable, session changes will only occur on a single JVM. Therefore, when the user connects to one of the other JVM's, their session will not be recognised, and a new session will be created. This may force them to log in again, establishing a 2nd session on the other JVM. As they switch between the two servers, various other problems may arise.
I'm working on a web application frontend to a legacy system which involves a lot of CPU bound background processing. The application is also stateful on the server side and the domain objects needs to be held in memory across the entire session as the user operates on it via the web based interface. Think of it as something like a web UI front end to photoshop where each filter can take 20-30 seconds to execute on the server side, so the app still has to interact with the user in real time while they wait.
The main problem is that each instance of the server can only support around 4-8 instances of each "workspace" at once and I need to support a few hundreds of concurrent users at once. I'm going to be building this on Amazon EC2 to make use of the auto scaling functionality. So to summarize, the system is:
A web application frontend to a legacy backend system
task performed are CPU bound
Stateful, most calls will be some sort of RPC, the user will make multiple actions that interact with the stateful objects held in server side memory
Most tasks are semi-realtime, where they have to execute for 20-30 seconds and return the results to the user in the same session
Use amazon aws auto scaling
I'm wondering what is the best way to make a system like this distributed.
Obviously I will need a web server to interact with the browser and then send the cpu-bound tasks from the web server to a bunch of dedicated servers that does the background processing. The question is how to best hook up the 2 tiers together for my specific neeeds.
I've been looking at message Queue systems such as rabbitMQ but these seems to be geared towards one time task where any worker node can simply grab a job form a queue, execute it and forget the state. My needs are a little different since there could be multiple 'tasks' that needs to be 'sticky', for example if step 1 is started in node 1 then step 2 for the same workspace has to go to the same worker process.
Another problem I see is that most worker queue systems seems to be geared towards background tasks that can be processed anytime rather than a system that has to provide user feedback that I'm dealing with.
My question is, is there an off the shelf solution for something like this that will allow me to easily build a system that can scale? Would love to hear your thoughts.
RabbitMQ is has an RPC tutorial. I haven't used this pattern in particular but I am running RabbitMQ on a couple of nodes and it can handle hundreds of connections and millions of messages. With a little work in monitoring you can detect when there is more work to do then you have consumers for. Messages can also timeout so queues won't backup too greatly. To scale out capacity you can create multiple RabbitMQ nodes/clusters. You could have multiple rounds of RPC so that after the first response you include the information required to get second message to the correct destination.
0MQ has this as a basic pattern which will fanout work as needed. I've only played with this but it is simpler to code and possibly simpler to maintain (as it doesn't need a broker, devices can provide one though). This may not handle stickiness by default but it should be possible to write your own routing layer to handle it.
Don't discount HTTP for this as well. When you want request/reply, a strict throughput per backend node, and something that scales well, HTTP is well supported. With AWS you can use their ELB easily in front of an autoscaling group to provide the routing from frontend to backend. ELB supports sticky sessions as well.
I'm a big fan of RabbitMQ but if this is the whole scope then HTTP would work nicely and have fewer moving parts in AWS than the other solutions.
Session variables are normally keept in the web server RAM memory.
In a cluster, each request made by a client can be handled by a different cluster node. right?!
So, in this case...
What happens with session variables? Aren't they stored in the nodes RAM memory?
How the other nodes will handled my request correctly if it doesn't have my session variables, or at least all of it?
This issue is treated by the web server (Apache, IIS) or by the language runtime (PHP, ASP.NET, Ruby, JSP)?
EDIT: Is there some solution for Classic ASP?
To extend #yogman's answer.
Memcached is pure awesomeness! It's a high performance and distributed object cache.
And even though I mentioned distributed it's basically as simple as starting one instance on one of your spare/idle servers, you configure it as in ip, port and how much ram to use and you're done.
memcached -d -u www -m 2048 -l 10.0.0.8 -p 11211
(Runs memcached in daemon mode, as user www, 2048 MB (2 GB) of RAM on IP 10.0.0.8 with port 11211.)
From then on, you ask memcached for data and if the data is not yet cached you pull it from the original source and store it in memcached. I'm sure you are familiar with cache basics.
In a cluster environment you can link up your memcached's into a cluster and replicate the cache across your nodes. Memcached runs on Linux, Unix and Windows, start it anywhere you have spare RAM and start using your resources.
APIs for memcached should be generally available. I'm saying should because I only know of Perl, Java and PHP. But I am sure that e.g. in Python people have means to leverage it as well. There is a memcached wiki, in case you need pointers, or let me know in the comments if I was raving too much. ;)
There are 3 ways to store session state in ASP.NET. The first is in process, where the variables are stored in memory. The second is to use a session state service by putting the following in your web.config file:
<sessionState
mode="StateServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString="data source=127.0.0.1;user id=sa;password="
cookieless="false"
timeout="20" />
As you can see in the stateConnectionString attribute, the session state service can be located on a different computer.
The third option is to use a centralized SQL database. To do that, you put the following in your web.config:
<sessionState
mode="SQLServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString=
"data source=SERVERHAME;user id=sa;password="
cookieless="false"
timeout="20"
/>
More details on all of these options are written up here: http://www.ondotnet.com/pub/a/dotnet/2003/03/24/sessionstate.html
Get a Linux machine and set up http://www.danga.com/memcached . Its speed is unbeatable compared to other approaches. (for example, cookies, form hidden variables, databases)
As with all sorts of thing, "it depends".
There are different solutions and approaches.
As mentioned, there's the concept of a centralized store for session state (database, memcached, shared file system, etc.).
There are also cluster wide caching systems available that make local data available to all of the machines in the cluster. Conceptually it's similar to the centralized session state store, but this data isn't persistent. Rather it lives within the individual nodes and is replicated using some mechanism provided by your provider.
Another method is server pinning. When a client hits the cluster the first time, some mechanism (typically a load balancer fronting the cluster) pins the client to a specific server. In a typical client lifespan, that client will spend their entire time on a single machine.
For the failover mechanism, each machine of the cluster is paired with another machine, and so any session changes are shared with the paired machine. Should the clients pinned machine encounter an issue, the client will hit another machine. At this point, perhaps due to cookies, the new machine sees that it's not the original machine for the client, so it pings both the original machine, and the paired machine for the clients session data.
At that point the client may well be pinned to the new machine.
Different platforms do it in different ways, including having no session state at all.
With Hazelcast, you can either use Hazelcast distributed map to store and share sessions across the cluster or let Hazelcast Webapp Manager do everything for you. Please check out the docs for details. Hazelcast is a distributed/partitioned, super lite and easy, free data distribution solution for Java.
Regards,
-talip
http://www.hazelcast.com
To achieve load balancing for classic ASP, you may store the user specific values in the database and pass a reference unique id in the URL as follows.
Maintain a session table in the database which generates a unique id for each record. The first time you want to store session specific data, generate a record in your session table and store the session values in it. Obtain the unique id of the new session record and re-write all links in your web application to send the unique id as part of querystring.
In every subsequent page where you need the session data, query the session table with the unique id passed in the querystring.
Example:
Consider your website to have 4 pages: Login.asp, welcome.asp, taskList.asp, newtask.asp
When the user logs in using login.asp page, after validating the user, create a record in session table and store the required session specific values (lets say user's login date/time for this example). Obtain the new session record's unique id (lets say the unique id is abcd).
Append all links in your website with the unique id as below:
welcome.asp?sessionId=abcd
tasklist.asp?sessionId=abcd
newtask.asp?sessionId=abcd
Now, if in any of the above web pages you want to show the user's login date/time, you just have to query your session table with the sessionID parameter (abcd in this case) and display to the user.
Since the unique value identifying the session is a part of the URL, any of your web servers serving the user will be able to display the correct login date/time value.
Hope this helps.
In ASP.NET you can persist session data to an SQL Server database which is common to all web servers in the cluster.
Once configured (in the web.config for your site), the framework handles all of the persistance for you and you can access the session data as normal.
As Will said, most load-balancing approaches will use some sort of stickiness in the way the distribute forthcoming requests from the same client, meaning, a unique client will hit the same server unless that actual server goes down.
That minimizes the need of distribution of session-data, meaning that only in the eventual failure of a server, a client would loose his session. Depending on your app, this is more or less critical. In most cases, this is not a big issue.
Even the simplest way of loadbalacing (round-rubin the DNS-lookups) will do some sort of stickiness since most browsers will cache the actual lookup and therefor keep going to the first record it received, AFAIK.
It's usually the runtime that is responsible for the sessiondata, in for exampla PHP it's possible to define your own session-handler, which can persist the data into a database for instance. By default PHP stores sessiondata on files, and it might be possible to share these files on a SAN or equivalent in order to share session-data. This was just a theory I had but never got around to test since we decided that loosing sessions wasn't critical and didn't want that single point of failure.