I am trying to expose HTTP endpoint which will drop a message to a JMS queue I want to reply with Success if the delivery is successful and FAILURE if message can not be delivered.
#Bean
public IntegrationFlow systemTaskCall(MapToServiceTaskConfigTransformer mapTransformer, CachingConnectionFactory jmsConnectionFactory) {
return IntegrationFlows.from(
Http.inboundGateway("/spartaSystemTask")
.requestMapping(r -> r
.methods(HttpMethod.POST)
.consumes("application/json")
)
.requestPayloadType(Map.class)
.replyChannel(RESPONSE_CHANNEL)
.errorChannel("errorChannel")
)
.handle((payload, headers) -> mapTransformer.transform((Map<String, String>) payload))
.enrichHeaders(Collections.singletonMap(DESTINATION_QUEUE, "request.queue"))
.enrichHeaders(Collections.singletonMap(JMS_REPLY_TO, "response.queue"))
.transform(Transformers.toJson())
.handle(
Jms.outboundGateway(jmsConnectionFactory,)
.requestDestination(message -> message.getHeaders().get(DESTINATION_QUEUE))
)
.log(LoggingHandler.Level.ERROR)
.enrichHeaders(
c -> c.header(org.springframework.integration.http.HttpHeaders.STATUS_CODE, HttpStatus.CREATED)
)
.transform(source -> "SUCCESS")
.transform(Transformers.toJson())
.channel(RESPONSE_CHANNEL)
.get();
}
#Bean
public IntegrationFlow errorFlow(){
return IntegrationFlows.from("errorChannel")
.transform(source -> "error")
.transform(Transformers.toJson())
.channel(RESPONSE_CHANNEL)
.get();
}
When I call this URL, Message is dropped but HTTP call times out. It seems post the JMS Outbound gateway call rest of the code is not executed.
In case of Failure in message delivery I am getting correct response.
You have this configuration:
.handle(
Jms.outboundGateway(jmsConnectionFactory,)
.requestDestination(message -> message.getHeaders().get(DESTINATION_QUEUE))
)
an outboundGateway. That means you send a request and expect a response from the other side, but it looks like you only send a JMS message into a queue and no one on the other listener side answers you with something into that response.queue. That's the reason for time out your get with a normal JMS publishing.
You need to sure that logic is correct in your flow and it is indeed valid in your distributed solution that you expect some reply from the server side.
Otherwise you need to think about changing your logic into a Jms.outboundAdapter() which is really one-way sender. For HTTP reply you could use a publishSubscribeChannel() with this Jms.outboundAdapter() as the first subscriber and the rest of your flow as a second one. This way the second subscriber is not going to be called until the first one finishes its logic properly. For the error cases you can wrap that Jms.outboundAdapter() with an ExpressionEvaluatingRequestHandlerAdvice: https://docs.spring.io/spring-integration/docs/5.2.3.RELEASE/reference/html/messaging-endpoints.html#message-handler-advice-chain
Related
Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like #kafkalistener for Nats jetstream
I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.
Any insights /examples regarding this will be helpful, thanks in advance.
I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.
public static void subscribe(String streamName, String subjectKey,
String queueName, IMessageHandler iMessageHandler) throws IOException,
InterruptedException, JetStreamApiException {
long s = System.currentTimeMillis();
Connection nc = Nats.connect(options);
long e = System.currentTimeMillis();
logger.info("Nats Connect in " + (e - s) + " ms");
JetStream js = nc.jetStream();
Dispatcher disp = nc.createDispatcher();
MessageHandler handler = (msg) -> {
try {
iMessageHandler.onMessageReceived(msg);
} catch (Exception exc) {
msg.nak();
}
};
ConsumerConfiguration cc = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.maxDeliver(3)
.ackWait(Duration.ofMinutes(2))
.build();
PushSubscribeOptions so = PushSubscribeOptions.builder()
.stream(streamName)
.configuration(cc)
.build();
js.subscribe(subjectKey, disp, handler, false, so);
System.out.println("NatsUtil: " + durableName + "subscribe");
}
IMessageHandler is my custom interface to handle nats.io received messages.
First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.
Connection natsConnection = Nats.connect(
new Options.Builder()
.server("nats://localhost:4222")
.connectionListener((connection, eventType) -> {})
.errorListener(new ErrorListener(){})
.build());
Then construct a JetStream instance
JetStream jetStream = natsConnection.jetStream();
Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation (https://docs.nats.io/nats-concepts/jetstream/consumers) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:
//Subscribe to a subject.
String subject = "my-subject";
//queues are analogous to Kafka consumer groups, i.e. consumers belonging
//to the same queue (or, better to say, reading the same queue) will get
//only one instance of each message from the corresponding subject
//and only one of those consumers will be chosen to process the message
String queueName = "my-queue";
//Choosing delivery policy is analogous to setting the current offset
//in a partition for a consumer or consumer group in Kafka.
DeliverPolicy deliverPolicy = DeliverPolicy.New;
PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.deliverPolicy(deliverPolicy)
.buildPushSubscribeOptions();
Subscription subscription = jetStream.subscribe(
subject,
queueName,
natsConnection.createDispatcher(),
natsMessage -> {
//This callback will be called for incoming messages
//asynchronously. Every subscription configured this
//way will be backed by its own thread, that will be
//used to call this callback.
},
true, //true if you want received messages to be acknowledged
//automatically, otherwise you will have to call
//natsMessage.ack() manually in the above callback function
subscribeOptions);
As for the declarative API (i.e. some form of #NatsListener annotation analogous to #KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:
https://github.com/linux-china/nats-spring-boot-starter,
https://github.com/dstrelec/nats
https://github.com/amalnev/declarative-nats-listeners
There may be others.
Staring with the tutorial code at benwilcock/spring-rsocket-demo I am trying to write a server that replicates messages to a second server before responding to a client.
To try to debug my issues I am only attempting a trivial ping-pong exchange between servers. Only when the second server responds to the pong message should the first server reply to the client:
#MessageMapping("request-response")
Mono<Message> requestResponse(final Message request) {
// register a mono that will be completed when replication to another server has happened
String uuid = UUID.randomUUID().toString();
Mono<Message> deferred = Mono.create(sink -> replicationNexus.registerRequest(uuid, sink));
// FIXME attempt to send a nested request-response message that will complete the outer message later
this.requesterMono.flatMap(requester -> requester.route("pong")
.data(uuid)
.retrieveMono(String.class))
.subscribeOn(Schedulers.elastic())
.subscribe( uuid2 -> replicationNexus.complete(uuid2, new Message(SERVER, RESPONSE)));
// return the deferred work that will be completed by the pong response
return deferred;
}
That logic is trying to use this answer to create a connection to the second server that will reconnect:
this.requesterMono = builder.rsocketConnector(connector -> connector
.reconnect(Retry.fixedDelay(Integer.MAX_VALUE, Duration.ofSeconds(1))))
.connectTcp("localhost", otherPort).cache();
To complete the picture here is the trivial ping-pong logic:
#MessageMapping("pong")
public Mono<String> pong(String m) {
return Mono.just(m);
}
and here is the logic that holds the state of the outer client response that is completed when the other server responds:
public class ReplicationNexus<T> {
final Map<String, MonoSink<T>> requests = new ConcurrentHashMap<>();
public void registerRequest(String v, MonoSink<T> sink) {
requests.put(v, sink);
}
public boolean complete(String uuid, T message) {
Optional<MonoSink<T>> sink = Optional.of(requests.get(uuid));
if( sink.isPresent() ){
sink.get().success(message);
}
return sink.isPresent();
}
}
Debugging the second server it never runs the pong method. It seems that the first server does not actually send the inner request message.
What is the correct way to run an inner request-response exchange that completes an outer message exchange with automated reconnection logic?
Not sure if I'm missing some of the complexity of your question, but if the middle server is just activing like a proxy I'd start with the simplest case of chaining through the calls. I feel like I'm missing some nuance of the question, so let's work through that next.
#MessageMapping("runCommand")
suspend fun runCommandX(
request: CommandRequest,
): Mono<String> {
val uuid = UUID.randomUUID().toString()
return requesterMono
.flatMap { requester: RSocketRequester ->
requester.route("pong")
.data("TEST")
.retrieveMono(String::class.java)
}
.doOnSubscribe {
// register request with uuid
}
.doOnSuccess {
// register completion
}
.doOnError {
// register failure
}
}
Generally if you can avoid calling subscribe yourself in typical spring/reactive/rsocket code. You want the framework to do this for you.
i define a queue channel
#Bean("mail-action-laundry-list-channel")
public MessageChannel mailRecipientActionMessageChannel() {
return new QueueChannel(20);
}
the flow below, i will aggrate messages from the queue channel, i tried this:
#Bean
public IntegrationFlow mailRecipientActionLaundryListMessageFlow(#Qualifier("laundryListMessageHandler") MessageHandler laundryListMessageHandler) {
return IntegrationFlows.from("mail-action-laundry-list-channel")
.log("--> laundry list messages:：")
.aggregate(aggregatorSpec -> aggregatorSpec
.correlationExpression("#this.payload.email")
.releaseExpression("#this.size() == 5")
.messageStore(new SimpleMessageStore(100))
.groupTimeout(2000))
.transform(laundryListMessageToItemProcessDtoTransformer())
.handle(laundryListMessageHandler)
.get();
}
but why it aggrate first 5 messages from the channel always, and aggrate other message no longer
You need to configure expireGroupsUponCompletion(true) on the aggregator:
When set to true (default false), completed groups are removed from the message store, allowing subsequent messages with the same correlation to form a new group. The default behavior is to send messages with the same correlation as a completed group to the discard-channel.
Looks like your subsequent messages from the queue has the same email property. Therefore an aggregator can't form a new group for the same correlation key.
https://docs.spring.io/spring-integration/docs/5.0.3.RELEASE/reference/html/messaging-routing-chapter.html#aggregator-config
New Spring has some WebSocketClient example on Spring documentation.
WebSocketClient client = new ReactorNettyWebSocketClient();
client.execute("ws://localhost:8080/echo"), session -> {... }).blockMillis(5000);
But it is very short and not clear:
How to send a message to the server (subscribe to a channel)?
Then handle incoming stream and emit Flux messages?
Reconnect to the server when the connection is interrupted?
Could some one provide more complex example?
UPD.
I tried to do something like:
public Flux<String> getStreaming() {
WebSocketClient client = new ReactorNettyWebSocketClient();
EmitterProcessor<String> output = EmitterProcessor.create();
Flux<String> input = Flux.just("{ event: 'subscribe', channel: 'examplpe' }");
Mono<Void> sessionMono = client.execute(URI.create("ws://api.example.com/"),
session -> session
.send(input.map(session::textMessage))
.thenMany(session.receive().map(WebSocketMessage::getPayloadAsText).subscribeWith(output).then())
.then());
return output.doOnSubscribe(s -> sessionMono.subscribe());
}
But that returns only one message. Like I didnt get subscription.
I assume you are using an "echo" service. In order to get some messages from the service, you have to push them into the websocket and the service will "echo" them back to you.
In your example code you are writing only a single element to the websocket. As soon as you push more messages into the socket you will get more back.
I adapted the code to connect to ws://echo.websocket.org instead of a local service. When you browse to /stream you see every second a new message appear.
#GetMapping(path = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> getStreaming() throws URISyntaxException {
Flux<String> input = Flux.<String>generate(sink -> sink.next(String.format("{ message: 'got message', date: '%s' }", new Date())))
.delayElements(Duration.ofSeconds(1));
WebSocketClient client = new ReactorNettyWebSocketClient();
EmitterProcessor<String> output = EmitterProcessor.create();
Mono<Void> sessionMono = client.execute(URI.create("ws://echo.websocket.org"), session -> session.send(input.map(session::textMessage))
.thenMany(session.receive().map(WebSocketMessage::getPayloadAsText).subscribeWith(output).then()).then());
return output.doOnSubscribe(s -> sessionMono.subscribe());
}
Hope this helps...
The documentation link above is to the temporary docs from before Spring Framework 5 was released. Currently the reference provides more information about implementing a WebSocketHandler.
everyone. I have an HTTP API for posting messages in a RabbitMQ broker and I need to implement the request-response pattern in order to receive the responses from the server. So I am something like a bridge between the clients and the server. I push the messages to the broker with specific routing-key and there is a Consumer for that messages, which is publishing back massages as response and my API must consume the response for every request. So the diagram is something like this:
So what I do is the following- For every HTTP session I create a temporary responseQueue(which is bound to the default exchange, with routing key the name of that queue), after that I set the replyTo header of the message to be the name of the response queue(where I will wait for the response) and also set the template replyQueue to that queue. Here is my code:
public void sendMessage(AbstractEvent objectToSend, final String routingKey) {
final Queue responseQueue = rabbitAdmin.declareQueue();
byte[] messageAsBytes = null;
try {
messageAsBytes = new ObjectMapper().writeValueAsBytes(objectToSend);
} catch (JsonProcessingException e) {
e.printStackTrace();
}
MessageProperties properties = new MessageProperties();
properties.setHeader("ContentType", MessageBodyFormat.JSON);
properties.setReplyTo(responseQueue.getName());
requestTemplate.setReplyQueue(responseQueue);
Message message = new Message(messageAsBytes, properties);
Message receivedMessage = (Message)requestTemplate.convertSendAndReceive(routingKey, message);
}
So what is the problem: The message is sent, after that it is consumed by the Consumer and its response is correctly sent to the right queue, but for some reason it is not taken back in the convertSendAndReceived method and after the set timeout my receivedMessage is null. So I tried to do several things- I started to inspect the spring code(by the way it's a real nightmare to do that) and saw that is I don't declare the response queue it creates a temporal for me, and the replyTo header is set to the name of the queue(the same what I do). The result was the same- the receivedMessage is still null. After that I decided to use another template which uses the default exchange, because the responseQueue is bound to that exchange:
requestTemplate.send(routingKey, message);
Message receivedMessage = receivingTemplate.receive(responseQueue.getName());
The result was the same- the responseMessage is still null.
The versions of the amqp and rabbit are respectively 1.2.1 and 1.2.0. So I am sure that I miss something, but I don't know what is it, so if someone can help me I would be extremely grateful.
1> It's strange that RabbitTemplate uses doSendAndReceiveWithFixed if you provide the requestTemplate.setReplyQueue(responseQueue). Looks like it is false in your explanation.
2> To make it worked with fixed ReplyQueue you should configure a reply ListenerContainer:
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(responseQueue);
container.setMessageListener(requestTemplate);
3> But the most important part here is around correlation. The RabbitTemplate.sendAndReceive populates correlationId message property, but the consumer side has to get deal with it, too: it's not enough just to send reply to the responseQueue, the reply message should has the same correlationId property. See here: how to send response from consumer to producer to the particular request using Spring AMQP?
BTW there is no reason to populate the Message manually: You can just simply support Jackson2JsonMessageConverter to the RabbitTemplate and it will convert your objectToSend to the JSON bytes automatically with appropriate headers.