I am building a scalable chat application using Go and Redis w/ websockets.
I need to publish a new message using redis pub-sub model to other websocket servers to inform all the users (saved in memory of other servers) about the new joined user.
But the issue is, the publisher(also a redis client) receives the same message. Is there a direct way to solve this?
Workaround:
Check if the user for new user in the received event (for publisher) is in the list of current local users everytime.
WHY NEGATIVE VOTES? I'm so pissed at stack-overflow these days. People have no tolerance or too much arrogance
Related
I have seen a lot of examples on the internet of chats using web sockets and RabbitMQ (https://github.com/videlalvaro/rabbitmq-chat), however I do not understand why it is need it a message queue for a chat application.
Why it is not ok to send the message from the browser via web sockets to the server and then the server to broadcast that message to the rest of active browsers using again web sockets with broadcast method? (maybe I am missing something)
Pseudo code examples (using socket.io):
// client (browser)
socket.emit("message","my great message that will be received by all"
// server (any server can be, but let's just say that it is also written in JavaScript
socket.on("message", function(msg) {
socket.broadcast.emit(data);
});
// the rest of the browsers
socket.on("message", function(msg) {
// display on the screen the message
});
i don't think RabbitMQ should be used for a chat room, personally. at least, not in the "chat" or "room" part of the application.
unless your chat rooms don't care about history at all - and i think most do care about that - a message queue like RMQ doesn't make much sense.
you would be better off storing the message in a database and keeping a marker for each user to say what message they last saw.
now, you may end up needing something like RMQ to facilitate the process of the chat application. you can offload process from the web servers, for example, and push all messages through RMQ to a back-end service that updates the database and cache layers, for example.
this would allow you to scale the front-end web servers much faster, and support more users per web server. and that sounds like a good use of RMQ, but is not specific to chat apps. it's just good practice for scaling web apps / systems.
the key, in my experience, is that RMQ is not responsible for delivery of the messages to the users / chat rooms. that happens through websockets or similar technologies that are designed to be used per user.
Simple answer ...
For a simple chat app you don't need a queue (e.g. signalr would do exactly this without the queue).
Typically though real world applications are not just "a simple chat app", the queue might represent the current state of the room for new users joining perhaps, so the server knows what list of messages to serve up when that happens.
Also it's worth noting that message queues are often implemented when you want reliable messaging (e.g. Service bus) to ensure that all messages definitely get to where they should go even if the first attempt fails. So it's likely that the queue is included in many examples as a default primer in to later problem solving.
I may be late for the answer as the messaging domain changed rapidly in last few years. Applications like WhatsApp do not store messages in their database, and also provide E2E encryption.
Coming to RabbitMQ, they support MQTT protocol which is ideal for low latency high scalability applications. Thus using such queuing services offload the heavy work from your server and provide features like scalability and security.
uhmm I didn't understand exactly for are you looking for...
but In RabbiMQ you always publish a messages to an exchange and consume the message using a queue.
to "broadcast that message" you need to consume it.
hope it helps
I have a project that is related to job postings. Consultants or employers register on my website and then start posting jobs. I want to make push notifications for all users. When a consultant or employer posts a job, all online users must get notified that an employer has posted this job without any page refreshes on jquery setInterval or timeout.
I am using Spring framework. I have searched for the solution but found nothing. I want to know whether Spring provided WebSockets in their latest version. Is this possible to do with WebSockets?
I want a proper resource so that I can implement it on my website.
There are two ways to satisfy your need;
First is polling in which you repeatedly send requests from client to the server. On server side you somehow need have a kind of message queue for each client to deliver the incidents on a request. There also is a different type of polling in which you send a request from client and never end the request on the server-side thus you have a kind of pipe between two ends. This is called long polling.
Disadvantage of polling is that you have to send requests to the server forever from the client and in many cases server sends empty messages as there is no events happened.
The real application of pushing messages is recently avaliable with websockets (thanks to html5). However this requires the application server to be capable of websocket functionality. afaik jetty and tomcat has this ability. Spring 4 has websocket here you can find the tutorial; http://syntx.io/using-websockets-in-java-using-spring-4/
You can find a related stackoverflow post here
Most of the articles on the web dealing with WebSockets are about in-memory Chat.
I'm interested in kind of less instant Chat, that is persistent, like a blog's post's comments.
I have a cluster of two servers handling client requests.
I wonder what could be the best strategy to handle pushing of database update to corresponding clients.
As I'm using Heroku to handle this cluster (of 2 web dynos), I obviously read this tutorial aiming to build a Chat Room shared between all clients.
It uses Redis in order to centralize coming messages; each server listening for new messages to propagate to web clients through websocket connections.
My use case differs in that I've got a Neo4j database, persisting into it each message written by any client.
My goal is to notify each client from a specific room that a new message/comment has just been persisted by a client.
With an architecture similar to the tutorial linked above, how could I filter only new messages to propagate to user? Is there an easy and efficient way to tell Redis:
"(WebSocket saying) When my client initiates the websocket connection, I take care to make a query for all persisted messages and sent them to client, however I want you (Redis) to feed me with all NEW messages, that I didn't send to client, so that I will be able to provide them."
How to prevent Redis from publishing the whole conversation each time a websocket connection is made? It would lead to duplications since the database query already provided the existing contents at the moment.
This is actually a pretty common scenario, where you have three components:
A cluster of stateless web servers that maintain open connections with all clients (load balanced across the cluster, obviously)
A persistent main data storage - Neo4j in your case
A messaging/queueing backend for broadcasting messages across channels (thus across the server cluster) - Redis
Your requirement is for new clients to receive an initial feed of the recent messages, and any consequent messages in real-time. All of this is implemented in your connection handlers.
Essentially, this is what your (pseudo-)code should look like:
class ConnectionHandler:
redis = redis.get_connection()
def on_init():
self.send("hello, here are all the recent messages")
recent_msgs = fetch_msgs_from_neo4j()
self.send(recent_msgs)
redis.add_listener(on_msg)
self.send("now listening on new messages")
def on_msg(msg):
self.send("new message: ")
self.send(msg)
The exact implementation really depends on your environment, but this is the general flow of things.
I'm trying to understand whether the HTML5 Web Notifications API can help me out, but I'm falling short in understanding how it works.
I'd like user_a to be able to send user_b a message within my webapp.
I'd like user_b to receive a notification of this.
Can the web notifications API help here? Does it let me specifically target a user (rather than notify everyone the site has been updated_? I can't see how I would create an alert for one person.
Can anyone help me understand a little more?
The notifications API is client side, so it needs to get events from another client-side technology. Here, read THIS: http://nodejs.org/api/. Just kidding. Node.js+socket.io is probably the best way to go here, you can emit events to one or all clients (broadcast). That's a push scenario. Or each user could be pulling their notifications from the server.
HTML5 Web Notifications API gives you ability to display desktop notifications that your application has generated.
What you are trying to achieve is a different thing and web notification is just a part of your scenario.
Depending upon how you are managing your application, for chat and messaging purpose as humbolight mentioned, you should look into node.js. it will provide you the necessary back-end to manage sending and receiving messages between users.
To notify a user that (s)he has received a message, you can opt for ajax polling on client side.
Simply create a javascript that pings the server every x seconds and checks if there is any notification or new message available for this user.
If response is successful, then you can use HTML5 notification API to show a message to user that (s)he has a new message.
The main problem with long polling is server load, and bandwidth usage even when there are no messages, and if number of users are in thousands then you can expect your server always busy responding to poll calls.
An alternate is to use Server Sent Events API, where you send a request to server and then server PUSHES the notifications/messages to the client as soon as they are available.
This reduces the unnecessary client->server polling and seems much better option in your case.
To get started you can check a good tutorial at
HTML5Rocks
What you're looking for is WebSocket. It's the technology that allows a client (browser) to open a persistent connection to the server and receive data from it at the server's whim, rather than having to "poll" the server to see if there's anything new.
Other answers here have already mentioned node.js, but Node is simply one (though arguably the best) option for implementing websockets on your server. You might also be comfortable with Ratchet, which is a websocket server library for PHP, or Tornado which is in Python.
How you handle your real-time communication is up to you. Websockets are merely the underlying technology that you can use to pass data back and forth. The client side of this will be fairly easy, but on the server side, you'll need a mechanism for websocket handlers to get information from each other. Look at tools like ZeroMQ for handling queues, and Memcached or Redis to handle large swaths of data which don't need to be stored permanently.
I'm using spring's imap mechanism in order to recieve emails from my account into my server.
this works like a charm.
Anyhow, a new requirmemnt came up - instead of listening to a single email account i will have to listen on a multiple number of accounts.
Iv'e tried creating a new channel for each of these account. it WORKS!
problem is that each channel i added meaning a new thread running.
since i'm talking about a large number of accounts it is quiet an issue.
My question is:
Since all the email accounts (I would like to listen to) are in the same domain i.e:
acount1#myDomain.com
acount2#myDomain.com
acount3#myDomain.com
....
Is it possible to create a single channel with multiple accounts?
Is there any alternative for me than defining N new channels?
thanks.
Nir
I assume you mean channel adapter, not channel (multiple channel adapters can send messages to the same channel).
No, you can't use a single connection for multiple accounts.
This is a limitation of the underlying internet mail protocols.
If you are using imap idle adapters, yes, this will not scale well because it needs a thread for each. However, if you are only talking about a few 10s of accounts, this is probably not an issue. For a much larger number of accounts, it may be better to use a polled adapter.
But, even so, unless it's a fixed number of accounts, the configuration could be burdensome (but you could programmatically spin up new adapters).
For complex scenarios like this, you may want to consider writing your own "adapter" that uses the JavaMail API directly and manages the connections in a more sophisticated way (but you still need a separate connection for each account). It wouldn't have to be a "real" adapter, just a POJO that interracts with JavaMail. Then, when you receive a message from one of the accounts, send it to a channel using a <gateway/>.