I have a websocket server and it keeps record of the users which are online. So if any change is made by a single user then it should reflect to all ther users which my server has maintained in an array.
Let's say a user changed name and total 100 users are online so server could easily loop through an array and send it to all connections.
Problem, what if I have like a good amount of user so there might be a case where my server stays in loop. So, to overcome this is there any other way to send data to all the users in websocket itself?
Related
A project I'm working on requires users to insert themselves into a list on a server. We expect a few hundred users over a weekend and while very unlikely, a collision could happen in which two users submit the list concurrently and one of them is lost. The server has no validation, it simply allows you to get and put data.
I was pointed in the direction of "optimistic locking" but I'm having trouble grasping when exactly the data should be validated and how it prevents this from happening. If one of the clients reads the data, adds itself and then checks again to ensure that the data is the same with the use of an index or timestamp, how does this prevent the other client from doing the same and then one overwriting the other?
I'm trying to understand the flow in the context of two clients getting data and putting data.
The point of optimistic locking is that the decision to accept or reject a write is taken on the server, and is protected against concurrency by a pessimistic transaction or some sort of hardware protection, such as compare-and-swap. So a client requests a write together with some sort of timestamp or version identifier, and the server only accepts the write if the timestamp is still accurate. If it isn't the client gets some sort of rejection code and will have to try again. If it is, the client gets told that its write succeeded.
This is not the only way to handle receiving data from multiple clients. One popular alternative is to use a reliable messaging system - for example the Java Messaging Service specifies an interface for such systems for which you can find open source implementations. Clients write into the messaging system and can go away as soon as their message is accepted. The server reads requests from the messaging system and acts on them. If the server or the network goes down it's no big deal: the messages will still be there to be read when they come back (typically they are written to disk and have the same level of protection as database data although if you look at a reliable message queue implementation you may find that it is not, in fact, built on top of a standard database table).
One example of a writeup of the details of optimistic locking is the HTTP server Etag specification e.g. https://en.wikipedia.org/wiki/HTTP_ETag
I have a PUB server. How can it tell what filters are subscribed to, so the server knows what data it has to create?The server doesn't need to create data once no SUB clients are interested in.
Say the set of possible filters is huge ( or infinite ), but subscribers at any given time are just subscribed to a few of them.
Example: Say SUB clients are only subscribed to a weather feed data for a few area codes in New York and Paris. Then the PUB server shouldn't have to create weather data for every other area code in every other city in the world, just to throw it all away again.
How do you find out all the subscribed to filters in a PUB server?
If there is no easy way, how do I solve this in another way?
I'll answer my own question here in case its of use to anyone else.
The requirements where:
The client should be able to ask the server what ids (topics) are available for subscription.
The client should chooses the id's it is interested in and tell the server about it.
The server should created data for all subscribed too id's and send that data to clients.
The client and server should not block/hang if either one goes away.
Implementation:
Step 1. Is two way traffic, and is done with REQ/REP sockets.
Step 2. Is one way traffic from one client to one server, and is done by PUSH/PULL sockets.
Step 3. Is one way traffic from one server to many clients, and is done by PUB/SUB sockets.
Step 4. The receives can block either the server or client if the other one is not there. Therefore I followed the "lazy pirate pattern" of checking if there is anything to receive in the queue, before I try and receive. (If there is nothing in the queue I'll check again on the next loop of the program etc).
Step 4+. Clients can die without unsubscribing, and the server wont know about it, It will continue to publish data for those ids. A solution is for the client to resends the subscription information (with a timestamp) every so often to the server. This works as a heartbeat for the ids the client has subscribed too. If the client dies without unsubscribing, the server notices that some subscription ids have not been refreshed in a while (the timestamp). The server removes those ids.
This solution seems to work fine. It was a lot of low level work though. It would be nice if zeromq was a bit higher level, and had some common and reliable architectures/frameworks ready to use out of the box.
I'm using spring's imap mechanism in order to recieve emails from my account into my server.
this works like a charm.
Anyhow, a new requirmemnt came up - instead of listening to a single email account i will have to listen on a multiple number of accounts.
Iv'e tried creating a new channel for each of these account. it WORKS!
problem is that each channel i added meaning a new thread running.
since i'm talking about a large number of accounts it is quiet an issue.
My question is:
Since all the email accounts (I would like to listen to) are in the same domain i.e:
acount1#myDomain.com
acount2#myDomain.com
acount3#myDomain.com
....
Is it possible to create a single channel with multiple accounts?
Is there any alternative for me than defining N new channels?
thanks.
Nir
I assume you mean channel adapter, not channel (multiple channel adapters can send messages to the same channel).
No, you can't use a single connection for multiple accounts.
This is a limitation of the underlying internet mail protocols.
If you are using imap idle adapters, yes, this will not scale well because it needs a thread for each. However, if you are only talking about a few 10s of accounts, this is probably not an issue. For a much larger number of accounts, it may be better to use a polled adapter.
But, even so, unless it's a fixed number of accounts, the configuration could be burdensome (but you could programmatically spin up new adapters).
For complex scenarios like this, you may want to consider writing your own "adapter" that uses the JavaMail API directly and manages the connections in a more sophisticated way (but you still need a separate connection for each account). It wouldn't have to be a "real" adapter, just a POJO that interracts with JavaMail. Then, when you receive a message from one of the accounts, send it to a channel using a <gateway/>.
I understand most of the POP3 protocol, but one thing that bothers me is how POP3 clients efficiently get a list of unretrieved message ids from the POP3 server. Many services like Yahoo and Gmail now offer gigs of space and most people (myself included), rarely if ever delete an email message.
I'm currently implementing a simple POP3 client in C#, though the question that I'm asking should be language agnostic.
On the client side I store a list of ALL the retrieved message ids that I've ever retrieved. There is no need for this client to ever delete messages, mark messages ad having been read etc. All it needs to do is get the newest email messages since the last time that it connected to the POP3 server.
Am I right in assuming that the algorithm goes something like this:
Retrieve ALL message IDs from the POP3 server. This list grows daily and can become megabytes in size easily.
Compare this list of message IDs with the message IDs I have already retrieved (this list being stored client side) and identify what mail messages I actually have to retrieve from the server.
Retrieve the mail messages one at a time from the server using the results from step 2 above.
Is there any way to make this more efficient?
There's no really efficient way to do this, because POP was never designed for leaving messages in the mailbox permanently - it's designed for the use-case where you fetch your mail once and delete it.
A better approach to the whole problem would be to use IMAP instead, which is designed for efficient storage.
We are currently developing a server whereby a client requests interest in changes to specific data elements and when that data changes the server pushes the data back to the client. There has vigorous debate at work about whether or not it would be better for the client to poll for this data.
What is considered to be the ideal method, in terms of performance, scalability and network load, of data transfer in a near real time environment?
Update:
Here's a Link that gives some food for thought with regards to UI updates.
There's probably no ideal method for every situation, but push is usually better and used more often. It allows to optimize server caching and data transfers, which helps performance and scalability, and cuts network traffic a bit by avoiding client requests and empty responses. It can be important advantage for a server to operate in it's own pace and supply clients with data when it is ready.
Industry standarts - such as OPC, GID - support both. Server pushes updates to subscribed clients, but client can pull some rarely used data out without bothering with subscription.
As long as the client initiates the connection (to get passed firewall and NAT problems) either way is fine.
If there are several different type of data you need to send, you might want to have the client specify which type he wants, but this is only needed once per connection. Then you can have the server continue to send updates as it has them.
It would be less network traffic to have the server send updates without the client continually asking for updates.
What do you have on the client's side? Many firewalls allow outgoing requests but block incoming requests. In other words, pull may be your only option if you are crossing the Internet unless you are sending out e-mails.