I have a Spring Boot app with an embedded queue. I'm using the spring-boot-starter-artemis dependency and trying to upgrade my Java app.
I could not find any guide, and a few things are not clear, e.g.:
Checking if a queue exist
EmbeddedJMS:
jmsServer.getJMSServerManager().getBindingsOnQueue(queueName).length > 0
Can it be:
jmsServer.getActiveMQServer().isAddressBound(queueName)
or maybe:
jmsServer.getActiveMQServer().bindingQuery(SimpleString.toSimpleString(queueName)).isExists()
Creation of queue:
jmsServer.getJMSServerManager().createQueue(true, queueName, null, true, queueName)
with params (boolean storeConfig, String queueName, String selectorString, boolean durable, String... bindings)
Is it the same like:
QueueConfiguration queueConfiguration = new QueueConfiguration(queueName);
queueConfiguration.setDurable(true);
return jmsServer.getActiveMQServer().createQueue(queueConfiguration, true).isEnabled();
(added)
Getting address:
jmsServer.getJMSServerManager().getAddressSettings(address);
is the same like:
jmsServer.getActiveMQServer().getAddressSettingsRepository().getMatch(address);
Not sure how migration Connection Factory settings
final Configuration jmsConfiguration = new ConfigurationImpl();
jmsConfiguration.getConnectionFactoryConfigurations()
.add(new ConnectionFactoryConfigurationImpl()
.setName("cf")
.setConnectorNames(Collections.singletonList("connector"))
.setBindings("cf"));
embeddedActiveMQ.setConfiguration(jmsConfiguration);
To check if a queue exists I think the simplest equivalent solution would be:
server.locateQueue("myQueue") != null
To create a queue the simplest equivalent solution would be:
server.createQueue(new QueueConfiguration("myQueue")) != null
The queue will be durable by default so there's no reason use setDurable(true).
To get address settings you use this (as you suspect):
server.getAddressSettingsRepository().getMatch(address);
Regarding connection factories, you don't actually need to configure connection factories on the broker. You simply need to configure the properties for the InitialContext for your JNDI lookup. See this documentation for more details on that.
Related
Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like #kafkalistener for Nats jetstream
I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.
Any insights /examples regarding this will be helpful, thanks in advance.
I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.
public static void subscribe(String streamName, String subjectKey,
String queueName, IMessageHandler iMessageHandler) throws IOException,
InterruptedException, JetStreamApiException {
long s = System.currentTimeMillis();
Connection nc = Nats.connect(options);
long e = System.currentTimeMillis();
logger.info("Nats Connect in " + (e - s) + " ms");
JetStream js = nc.jetStream();
Dispatcher disp = nc.createDispatcher();
MessageHandler handler = (msg) -> {
try {
iMessageHandler.onMessageReceived(msg);
} catch (Exception exc) {
msg.nak();
}
};
ConsumerConfiguration cc = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.maxDeliver(3)
.ackWait(Duration.ofMinutes(2))
.build();
PushSubscribeOptions so = PushSubscribeOptions.builder()
.stream(streamName)
.configuration(cc)
.build();
js.subscribe(subjectKey, disp, handler, false, so);
System.out.println("NatsUtil: " + durableName + "subscribe");
}
IMessageHandler is my custom interface to handle nats.io received messages.
First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.
Connection natsConnection = Nats.connect(
new Options.Builder()
.server("nats://localhost:4222")
.connectionListener((connection, eventType) -> {})
.errorListener(new ErrorListener(){})
.build());
Then construct a JetStream instance
JetStream jetStream = natsConnection.jetStream();
Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation (https://docs.nats.io/nats-concepts/jetstream/consumers) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:
//Subscribe to a subject.
String subject = "my-subject";
//queues are analogous to Kafka consumer groups, i.e. consumers belonging
//to the same queue (or, better to say, reading the same queue) will get
//only one instance of each message from the corresponding subject
//and only one of those consumers will be chosen to process the message
String queueName = "my-queue";
//Choosing delivery policy is analogous to setting the current offset
//in a partition for a consumer or consumer group in Kafka.
DeliverPolicy deliverPolicy = DeliverPolicy.New;
PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.deliverPolicy(deliverPolicy)
.buildPushSubscribeOptions();
Subscription subscription = jetStream.subscribe(
subject,
queueName,
natsConnection.createDispatcher(),
natsMessage -> {
//This callback will be called for incoming messages
//asynchronously. Every subscription configured this
//way will be backed by its own thread, that will be
//used to call this callback.
},
true, //true if you want received messages to be acknowledged
//automatically, otherwise you will have to call
//natsMessage.ack() manually in the above callback function
subscribeOptions);
As for the declarative API (i.e. some form of #NatsListener annotation analogous to #KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:
https://github.com/linux-china/nats-spring-boot-starter,
https://github.com/dstrelec/nats
https://github.com/amalnev/declarative-nats-listeners
There may be others.
I need to bind a fanout exchange to the default exchange in Spring Boot RabbitMQ client and I can't do it. When I try the code below, I see no errors, but the exchange is not bound (at least it doesn't show in rabbitmq admin nor are the messages routed to that exchange). My code:
A message with routing key testq put into testex exchange disappears - isn't delivered to testq queue.
#Bean
FanoutExchange testExchange() {
return new FanoutExchange("testex");
}
#Bean
Binding testExchangeToDefaultBinding() {
return new Binding("", DestinationType.EXCHANGE, "testex", null, null);
}
#Bean
Queue testQueue() {
return new Queue("testq");
}
Is it impossible to bind it that way?
Edit: Is there any other way to do something like that?
The default exchange is a direct exchange with no name (empty string) pre-declared by the broker.
The default exchange is implicitly bound to every queue, with a routing key equal to the queue name. It is not possible to explicitly bind to, or unbind from the default exchange.
My Application uses ElastiCache on AWS for caching purposes. Our current set up uses a basic Redis Cluster with no sharding or failover. We need to now move to a Clustered Redis Elastic Cache with sharding, failover etc enabled. Creating a new cluster on AWS was the easy bit, but we are a bit lost on how to modify our java code to reads and write from the cluster.
Current Implementation -
Initialize a JedisPool.
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
jedisPoolConfig.setMaxTotal(100);
jedisPoolConfig.setMaxIdle(10);
jedisPoolConfig.setMaxWaitMillis(50);
jedisPoolConfig.setTestOnBorrow(true);
String host = "mycache.db8e1v.0001.usw2.cache.amazonaws.com";
int port = 6379;
int timeout = 50;
JedisPool jedisPool = new JedisPool(jedisPoolConfig, host, port, timeout)
A Jedis object is borrowed from the pool everytime we need to perform an operation
Jedis jedis = JedisPool.getResource();
The new implementation would be
JedisPoolConfig jedisPoolConfig = ...
HostAndPort hostAndPort = new HostAndPort(host, port);
jedisCluster = new JedisCluster(Collections.singleton(hostAndPort), jedisPoolConfig);
Question:
The documentation says JedisCluster is to be used in place of Jedis (not JedisPool). Does this mean I need to create and destroy a JedisCluster object in each thread. Or can I re-use the same object and it will handle the thread safety? When do I exactly close the JedisCluster then? At the end of the application?
The JedisCluster holds internal JedisPools for each node in the cluster.
Does this mean I need to create and destroy a JedisCluster object in
each thread. Or can I re-use the same object and it will handle the
thread safety?
You can reuse the same object.
When do I exactly close the JedisCluster then? At the end of the
application?
Yes.
Replacing all Jedis-calls with JedisCluster-calls is the best way to migrate.
But I wanted pipeline support which JedisCluster currently lacks. So one other idea is to extend JedisCluster to return the JedisPool>Jedis for a particular key:
protected Jedis getJedis(String key) {
int slot = JedisClusterCRC16.getSlot(key);
return connectionHandler.getConnectionFromSlot(slot);
}
The extended class has to be in namespace redis.clients.jedis to access getConnectionFromSlot.
Now a pipeline can be executed on the Jedis.
And you need a different Jedis for each key you want to operate on. Which makes sense - in cluster mode, each key can be on a different node.
everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.
1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.
I'm trying to create a listener in tibco rendezvous. I want to listen in on a particular subject. I'm aware that its supposed to look something like this:
TibrvListener(
TibrvQueue queue,
TibrvMsgCallback callback,
TibrvTransport transport,
java.lang.String subject,
java.lang.Object closure)
throws TibrvException
I have this code. However, I don't know a couple of things. How do I create a TibrvMsgCallback object? How do I pass in the transport? I have a publisher that sends the message as a seperate program. Do I recreate an identical transport in my subscribe program?
queue = new TibrvQueue();
dispatcher = new TibrvDispatcher(queue);
queue.setName(key);
this.listener = new TibrvListener(queue, null, null, subject, null);
TibrvTransport message = this.listener.getTransport();
You first open the Tibrv
Tibrv.open(Tibrv.IMPL_NATIVE);
Create transport
TibrvTransport transport = new TibrvRvdTransport(service, network, daemon);
Create Listener
new TibrvListener(Tibrv.defaultQueue(), this, transport, subject, null);
If your listener is "this", your class needs to implement TibrvMsgCallback
Messages can be processed on arrival in the onMsg(TibrvListener listener, TibrvMsg msg) method.