I am developing a new notification microservice using SpringBoot and Kafka. It works like this:
The notification service has an API where consumers can send a notification message request.
The notification service puts this request on a kafka queue and sends back an accepted response.
The notification service reads the Kafka queue and sends the notifications to an external email/SMS service.
For point 1, the message details (e.g. sender, message) are put into the kafka queue. Ideally I don't want to put all these details in the kafka message. I could insert the notification into a DB and return a notification ID which is put into the Kafka message. The only concern I have here is due to the volume of requests, I don't want to overload the DB or if there is a DB issue I lose requests.
Any advice/recommendations of what is best practice?
Related
I am implementing spring boot stomp message broker socket to interact with webclient. i need to send sms to a specific user by username at some application point,means the message will be trigger from server to client. client will subscribe to a topic/queue. i heard #SendtoUser send sms to the perticular user, but here in my case user is just subscribing a topic, then from backend i need to send sms time to time to specific user. user will not send any sms to server.
its just push based sms.
messagingTemplate.convertAndSendToUser(sessionId,"/queue/something", payload,
headerAccessor.getMessageHeaders());
but here from where will i get the session id for the targeted user. here user is just subscribing the topic once.
You can find answer to a similar question (with project exemple) here :
Spring websocket send to specific people
The fact that user is subscribing once is not a problem. One the connection is established, the server can send has much message as needed.
We have a scenario where we need to update(send notification) to a X user belongs to a Y (subscribe)group on activity of other group members in real-time.
We decided the tech stack as -
MongoDB -- store user activity in MongoDB,
Kafka - Push activity event in Kafka (Messaging Queue) as well,
Spring Boot --- Backend API,
Angular2/Android/iOS -- Front End,
Websockets -- real-time data update
So, whenever there is an activity from user, it will be logged in both Mongo and Kafka.
Client will subscribe to Websocket /activity/{group-id}
Websocket will talk to kafka through kafka consumer and send notification to client if there is any new message in kafka.
My Question is -
How do I keep Kafka consumer process listening to Kafka-topic (I know how to read messages from Kafka) and send if any new message is there to client over socket.
In other words, One way communication from Server to Clients,of subscribed group.
Thanks
Pari
Currently I have a frontend service that makes calls to a notification service. The frontend service does not depend on the response from the notification service. I want the frontend service to make a call to the notification, but not wait for a response from the notification service. Could do this within the frontend service itself? Do I have to use some messaging service that can 'proxy' the calls between the two services?
You need a queue, the queue can be independent service (RabbitMq, Hornet, etc) o you can use a queue inside your notification service.
Independent Queue, in this case you can use a message broker in the middle of the services, the frontendservices send the message to the queue and the notification service read the message from the queue and then send it.
Internal Queue, you can have in your notification service an internal queue, like the queues in SpringIntegration. Whe the service receive a message inmediately put it on the queue and return an OK code. The frontendservice doesn't need to wait until the notification is send. Then the notification service process the messages in its queue on its own pace.
You could go for asynchronous execution. Just add #EnableAsync to your application class and annotate the method calling the notification service with #Async.
I am new to world of JMS and stuck in an issue. Here it is;
Suppose I have one topic and all the clients are publishing and subscribing to same topic. What i want to achieve is this if ClientA publishes a message to the topic then all the other clients should receive that message but ClientA (Sender should not receive his own message returned back).
You need to configure the subscriber to ignore messages published from the current connection.
This is done by setting the noLocal option when creating a subscriber.
See How can I prevent receiving JMS messages that I have produced?.
I'm dealing with a standalone MQ JMS application, our app need to "aware" that client already consumed the message producer put on the queue. Because client app is not responsible by us. So we cannot let them to write something like "msg.acknowledge();" thing on their side (msg.acknowledge() is not the right approach on my condition.). I search the history answer in the stackoverflow. Find following is quite the same what I want:
https://stackoverflow.com/questions/6521117/how-to-guarantee-delivery-of-the-message-in-jms
Do the JMS spec or the various implementations support delivery confirmation of messages?
My question is, is there any other way to archive this in the MQ API or JMS API? I need to do the coding only on the msg produce side, it is can be queue or topic.
Another question is in the JMS the acknowledge mode CLIENT_ACKNOWLEDGE, is that produce irrelevant? I always believe that this mode can block the application when call send() method until the client consume the message and call the msg.acknowledge(), but seems not like that. The produce just exit the app after message be delivered, and the message just store in the queue until client call the acknowledge(). Is that possible let the producer app hang there wait until the message be acknowledged by the client?
If my concept is not right, just correct me, thanks.
The main intention of message queuing is to decouple producer and consumer. Producer does not need to wait for the message to be consumed by the consumer, it can continue it's job. Ideally if producer needs to know if the message has been processed by consumer or not, it should wait for consumer to send a response message on another queue.
Message acknowledgement has nothing to do with producer. Message acknowledgement is the way a consumer tells the messaging provider to remove the message from a queue after the message has been delivered to an application.
There is auto acknowledge where the JMS providers (like MQ JMS), after delivering message to an application, tell the messaging provider to remove the message from queue. Then there is client acknowledge where, after receiving a message, the application explicitly tells the messaging provider to remove message from a queue.
Is there is a reason why the producer has to wait for consumer to receive the message? One way, though not elegant, could be: Once the message is sent, use the message id of the sent message and try to browse for that message. If message is not found, you can assume it has been consumed