CONTEXT
I am trying to learn about SpringJMS and MOMS and I am using ActiveMQ Artemis for this. I created a Queue Destination address using the jakarta.jms.* API, and managed to send some message to the queue like this:
public void createUserDestination(String userId) throws JMSException {
queueDestination = setupConnection().createQueue("user" + userId);
producer = session.createProducer(queueDestination);
producer.setDeliveryMode(DeliveryMode.PERSISTENT);
producer.send(session.createTextMessage("Testing queue availability"));
connection.close();
log.info("successfully created group... going back to controller");
}
So for example, if I pass an ID of user12345abc, I get a Queue Address user12345abc, of the routing type ANYCAST with one queue underneath(with that same address) with my message placed there.
PROBLEM
Now, I wanted to write a simple web front-end with STOMP that can connect to this queue. But I have been having a ton of problems connecting to that queue address because each time I try to connect by providing the destination address, it creates a new address in the MOM and connects to that instead.
My STOMP code looks like this(the first argument is the destination address, you can ignore the rest of the code):
stompClient.subscribe("jms.queue.user12345abc", (message) => {
receivedMessages.value.push(message.body);
});
In this case, completely brand new queue is created with the address jms.queue.user12345abc which is not what I want at all.
I configured my Spring Backend to use an external MOM broker like this(I know this is important):
public void configureMessageBroker(MessageBrokerRegistry registry) {
// these two end points are prefixes for where the messages are pushed to
registry.enableStompBrokerRelay("jms.topic", "jms.queue")
.setRelayHost("127.0.0.1")
.setRelayPort(61613)
.setSystemLogin(brokerUsername)
.setSystemPasscode(brokerPassword)
.setClientLogin(brokerUsername)
.setClientPasscode(brokerPassword);
// this prefixes the end points where clients send messages
registry.setApplicationDestinationPrefixes("/app", "jms.topic", "jms.queue");
// this prefixes the end points where the user's subscribe to
registry.setUserDestinationPrefix("/user");
}
But it's still not working as I expect it to. Am I getting some concept wrong here? How do I use STOMP to connect to that queue I created earlier with JMS?
It's not clear why you are using the jms.queue and jms.topic prefixes. Those are similar but not quite the same as the jms.queue. and jms.topic. prefixes which were used way back in ActiveMQ Artemis 1.x (whose last release was in early 2018 almost 5 years ago now).
In any case, I recommend you use the more widely adopted /queue/ and /topic/, e.g.:
public void configureMessageBroker(MessageBrokerRegistry registry) {
// these two end points are prefixes for where the messages are pushed to
registry.enableStompBrokerRelay("/topic/", "/queue/")
.setRelayHost("127.0.0.1")
.setRelayPort(61613)
.setSystemLogin(brokerUsername)
.setSystemPasscode(brokerPassword)
.setClientLogin(brokerUsername)
.setClientPasscode(brokerPassword);
// this prefixes the end points where clients send messages
registry.setApplicationDestinationPrefixes("/app", "/topic/", "/queue/");
// this prefixes the end points where the user's subscribe to
registry.setUserDestinationPrefix("/user");
}
The in broker.xml you'd need to add the corresponding anycastPrefix and multicastPrefix values on the STOMP acceptor, e.g.:
<acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true;anycastPrefix=/queue/;multicastPrefix=/topic/</acceptor>
To be clear, your JMS code will stay the same, but your STOMP consumer would be something like:
stompClient.subscribe("/queue/user12345abc", (message) => {
receivedMessages.value.push(message.body);
});
Related
Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like #kafkalistener for Nats jetstream
I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.
Any insights /examples regarding this will be helpful, thanks in advance.
I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.
public static void subscribe(String streamName, String subjectKey,
String queueName, IMessageHandler iMessageHandler) throws IOException,
InterruptedException, JetStreamApiException {
long s = System.currentTimeMillis();
Connection nc = Nats.connect(options);
long e = System.currentTimeMillis();
logger.info("Nats Connect in " + (e - s) + " ms");
JetStream js = nc.jetStream();
Dispatcher disp = nc.createDispatcher();
MessageHandler handler = (msg) -> {
try {
iMessageHandler.onMessageReceived(msg);
} catch (Exception exc) {
msg.nak();
}
};
ConsumerConfiguration cc = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.maxDeliver(3)
.ackWait(Duration.ofMinutes(2))
.build();
PushSubscribeOptions so = PushSubscribeOptions.builder()
.stream(streamName)
.configuration(cc)
.build();
js.subscribe(subjectKey, disp, handler, false, so);
System.out.println("NatsUtil: " + durableName + "subscribe");
}
IMessageHandler is my custom interface to handle nats.io received messages.
First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.
Connection natsConnection = Nats.connect(
new Options.Builder()
.server("nats://localhost:4222")
.connectionListener((connection, eventType) -> {})
.errorListener(new ErrorListener(){})
.build());
Then construct a JetStream instance
JetStream jetStream = natsConnection.jetStream();
Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation (https://docs.nats.io/nats-concepts/jetstream/consumers) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:
//Subscribe to a subject.
String subject = "my-subject";
//queues are analogous to Kafka consumer groups, i.e. consumers belonging
//to the same queue (or, better to say, reading the same queue) will get
//only one instance of each message from the corresponding subject
//and only one of those consumers will be chosen to process the message
String queueName = "my-queue";
//Choosing delivery policy is analogous to setting the current offset
//in a partition for a consumer or consumer group in Kafka.
DeliverPolicy deliverPolicy = DeliverPolicy.New;
PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.deliverPolicy(deliverPolicy)
.buildPushSubscribeOptions();
Subscription subscription = jetStream.subscribe(
subject,
queueName,
natsConnection.createDispatcher(),
natsMessage -> {
//This callback will be called for incoming messages
//asynchronously. Every subscription configured this
//way will be backed by its own thread, that will be
//used to call this callback.
},
true, //true if you want received messages to be acknowledged
//automatically, otherwise you will have to call
//natsMessage.ack() manually in the above callback function
subscribeOptions);
As for the declarative API (i.e. some form of #NatsListener annotation analogous to #KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:
https://github.com/linux-china/nats-spring-boot-starter,
https://github.com/dstrelec/nats
https://github.com/amalnev/declarative-nats-listeners
There may be others.
How can I block netty to send ACK responese to client in netty 4.x ?
I'm trying to control TCP packet receive speed in netty in order to forward these packet to another server . Netty receive all client packets immediately ,but netty need more time send them out , so client think it finished after sending to netty .
So , I want to know how to block received packets when netty forwarding packets which are received before to another server .
Not sure to really understand your question. So I try to reformulate:
I suppose that your Netty server is acting as a Proxy between clients and another server.
I suppose that what you want to do is to send the ack back to the client only once you really send the forwarded packet to the final server (not necesseraly received by the final server, but at least send by Netty proxy).
If so, then you should use the future of the forwarded packet to respond back with the ack, such as (pseudo code):
channelOrCtxToFinalServer.writeAndFlush(packetToForward).addListener(new ChannelFutureListener() {
public void operationComplete(ChannelFuture future) {
// Perform Ack write back
ctxOfClientChannel.writeAndFlush(AckPacket);
}
});
where:
channelOrCtxToFinalServer is one of ChannelHandlerContext or Channel connected to your remote final server from your Netty proxy,
and ctxOfClientChannel is the current ChannelHandlerContext from your Netty handler that receive the packet from the client in public void channelRead(ChannelHandlerContext ctxOfClientChannel, Object packetToForward) method.
EDIT:
For the big file transfer issue, you can have a look at the Proxy example here.
In particular, pay attention on the following:
Using the same logic, pay attention on receiving data one by one from client:
yourServerBootstrap..childOption(ChannelOption.AUTO_READ, false);
// Allow to control one by one the speed of reception of client's packets
In your frontend handler:
public void channelRead(final ChannelHandlerContext ctx, Object msg) {
if (outboundChannel.isActive()) {
outboundChannel.writeAndFlush(msg).addListener(new ChannelFutureListener() {
#Override
public void operationComplete(ChannelFuture future) {
if (future.isSuccess()) {
// was able to flush out data, start to read the next chunk
ctx.channel().read();
} else {
future.channel().close();
}
}
});
}
}
And finally add, using the very same logic, the final ack to your client (ack depending of course on your protocol): (see here and here)
/**
* Closes the specified channel after all queued write requests are flushed.
*/
static void closeOnFlush(Channel ch) {
if (ch.isActive()) {
ch.writeAndFlush(AckPacket).addListener(ChannelFutureListener.CLOSE);
}
}
I'm new to ZeroMQ (and to networking in general), and have a question about using ZeroMQ in a setup where multiple clients connect to a single server. My situation is as follows:
--1 server
--multiple clients
--Clients send messages to server: I've already figured out how to do this part.
--Server sends messages to a specific client: This is the part I'm having trouble with. When certain events get handled on the server, the server will need to send a message to a specific client -- not all clients. In other words, the server will need to be able to choose which client to send a given message to.
Right now, this is my server code:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateResponseSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.Send("ack"); // There is no overload for the 'Send'
method that takes an IP address as an argument!
}
}
}
I have a feeling that the problem is that my design is wrong, and that the ResponseSocket type isn't meant to be used in the way that I want to use it. Since I'm new to this, any advice is very much appreciated!
when using the Response socket you always replying to the client that sent you the message. So the Request-Response socket types together are just simple request response.
To more complicated scenarios you probably want to use Dealer-Router.
With router the first frame of each message is the routing id (the identity of the client that sent you the message)
so your example with router will look like:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateRouterSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
byte[] routingId = server.Receive();
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.SendMore(routingId).Send("ack");
}
}
}
I also suggest to read the zeromq guide, it will probably answer most of your questions.
I am trying to get notified when an user subscribes to a stomp user destination using #SubscribeMapping annotation. My ideia is to send some initialization data to when it joins.
Although i am not being able to get this working:
Javascript:
stompClient.subscribe('/user/monitor', function(msg) {
console.log(msg);
});
Java side:
#SubscribeMapping("/monitor-user{clientId}")
public void init(#DestinationVariable("clientId") String clientId) {
messagingTemplate.convertAndSend("/user/" + clientId + "/monitor", getOps());
}
I tried many mapping combinations such as "/user/monitor-{clientId}", "/user/monitor" (Removing clientId), no success at all.
What is the expected mapping value so this get called?
Thank you!
Since the client subscribes to the "/user/monitor" queue, the server #SubscribeMapping annotation must be to "/user/monitor" and not to "/user/monitor{something}".
The server can understand what client is depending on your authentication scheme. If you use a sockjs websocket you can use HTTP authentication, and hence leverage Spring Security, and add a "Principal" parameter to your function that will hold user information:
#SubscribeMapping("/monitor")
public void init(Principal p) {
String user = p.getName();
messagingTemplate.convertAndSendToUser(user, "/monitor", getOps());
}
If you don't use http authentication you may send the server the user info, for example adding a custom STOMP header that can be accessed from the server using a SimpMessageHeaderAccessor:
#SubscribeMapping("/monitor")
public void init(SimpMessageHeaderAccessor accessor) {
String user = accessor.getFirstNativeHeader("userid");
messagingTemplate.convertAndSendToUser(user, "/monitor", getOps());
}
Another way could be to subscribe to a different queue name which contains the user information (and this maybe was your proposal). The client must use a code similar to:
stompClient.subscribe('/topic/byUser/'+userId, function(msg) {
console.log(msg);
});
so that the server can access it this way:
#SubscribeMapping("/byUser/{userId}")
public void init(#DestinationVariable String userId) {
messagingTemplate.convertAndSendToUser(userId, "/monitor", getOps());
}
but in this case keep in mind that that queue is public and hence other clients can access their messages if they knows other client names.
everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.
1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.