Cannot use 'subscribe' or 'subscribeWith' with 'ReactorNettyWebSocketClient' in Kotlin - websocket

The Kotlin code below successfully connects to a Spring WebFlux server, sends a message and prints each message sent via the stream that is returned.
fun main(args: Array<String>) {
val uri = URI("ws://localhost:8080/myservice")
val client = ReactorNettyWebSocketClient()
val input = Flux.just(readMsg())
client.execute(uri) { session ->
session.send(input.map(session::textMessage))
.thenMany(
session.receive()
.map(WebSocketMessage::getPayloadAsText)
.doOnNext(::println) // want to replace this call
.then()
).then()
}.block()
}
In previous experience with Reactive programming I have always used subscribe or subscribeWith where the call to doOnNext occurs. However it will not work in this case. I understand that this is because neither returns the reactive stream in use - subscribe returns a Disposable and subscribeWith returns the Subscriber it received as a parameter.
My question is whether invoking doOnNext is really the correct way to add a handler to process incoming messages?
Most Spring 5 tutorials show code which either calls this or log, but some use subscribeWith(output).then() without specifying what output should be. I cannot see how the latter would even compile.

subscribe and subscribeWith should always be used right at the end of a chain of operators, not as intermediate operators.

Simon already provided the answer but I'll add some extra context.
When composing asynchronous logic with Reactor (and ReactiveX patterns) you build an end-to-end chain of processing steps, which includes not only the logic of the WebSocketHandler itself but also that of the underlying WebSocket framework code responsible for sending and receiving messages to and from the socket. It's very important for the entire chain to be connected together, so that at runtime "signals" will flow through it (onNext, onError, or onComplete) from start to end and communicate the final result, i.e where you have the .block() at the end.
In the case of WebSocket this looks a little daunting because you're essentially combining two or more streams into one. You can't just subscribe to one of those streams (e.g. for inbound messages) because that prevents composing a unified processing stream, and signals will not flow through to the end where the final outcome is expected.
The other side of this is that subscribe() triggers consumption on a stream whereas what you really want is to keep composing asynchronous logic in deferred mode, i.e. declaring all that will happen when data materializes. This is another reason why composing a single unified chain is important. So it can be triggered after it is fully declared.
In short the main difference with the imperative WebSocketHandler for the Servlet world, is that instead of it being a handler for individual messages, this is a handler for composing the complete streams. Here the handling of an individual message is just one step of the overall processing chain. So the only place to subscribe is at the very end, where .block() is, in order to kick off processing.
BTW since this question was first posted a few months ago, the documentation has been improved to provide more guidance on how to implement a WebSocketHandler.

Related

At what point does the subscription take place? (spring webflux)

At what point does the spring webflux do the subscription? Everywhere I have read that there must be a subscription otherwise no change happens. In my short time with Spring Webflux, I have never seen a subscribe() neither in the controller or services.
My doubt is also when using flatMap(), map(),... etc.. at what point does the subscription take place?
What I have read does not really resolve my doubts.
public Flux method(){
....
myFlux.flatMap(data -> {
....
}).flatMap(e -> { .... });
}
I know this is an asynchronous issue, but each flatMap runs at the same time?...and so sometimes some data I have noticed is null.
It's the framework (spring-webflux) that subscribes to the returned Mono or Flux. For example if you use Netty (that's the default), then subscription happens here based on my debugging:
https://github.com/reactor/reactor-netty/blob/db27625064fc78f8374c1ef0af3160ec3ae979f4/reactor-netty-http/src/main/java/reactor/netty/http/server/HttpServer.java#L962
Also, this article might be of help to understand what happens when:
https://spring.io/blog/2019/03/06/flight-of-the-flux-1-assembly-vs-subscription
You need to call a .subscribe() or block() function after your flatmap. Here's an example.
Assuming that myFlux is of type Flux, the following will execute the subscription based on the example above
myFlux.subscribe(System.out::println);
Here's an explanation on a separate StackOverflow thread.
But in your method function, you are returning a Flux object - so it's up to the consumer of the method() function how it wants to subscribe to the Flux. You shouldn't be trying to subscribe to the Flux from within
The answer is: it depends.
For example, if this is a Spring Controller method, then it is the framework itself that subscribes to the Mono or Flux.
If it is a method that is triggered from time to time by a Scheduler, then you must explicitly subscribe to the Mono or Flux, otherwise, no processing will take place.
This means that if your application only exposes a REST API and no processing need to be triggered in any other way, then it is very likely that you will never need to explicitly subscribe to a Mono or Flux because Spring will take care of that by you.

Workflow modeling problem in Spring Integration

I have a problem creating/modeling integration flow for the next global use case:
Input to the system is some kind of Message. That message goes
through Splitter and Transformer Endpoint and after that on
ServiceActivator where that transformed message is processed. This
use case is clear for me.
Confusion occurs because of the next part. After the ServiceActivator
finishes processing I need to took the base Message (message from the
beginning of first part) again and put it in other processing, for example again through Splitter and Transformer. How can
I model that use case? Can I return the message payload to that base
value? Is there some component that could help me?
Hope I describe it well.
Your use-case sounds more like a PublishSubscribeChannel: https://docs.spring.io/spring-integration/docs/current/reference/html/core.html#channel-implementations-publishsubscribechannel. So, you are going to have several subscribers (splitters) for that channel and the same input message is going to be processed in those independent sub-flows. You even can do that in parallel if you configure an Executor into that PublishSubscribeChannel.
Another way, if you can do that in parallel and you still need some result from that ServiceActivator to be available alongside with an original message for the next endpoint or so, then you can use a HeaderEnricher to store an original message in the headers ad get access to it whenever you need in your flow: https://docs.spring.io/spring-integration/docs/current/reference/html/message-transformation.html#header-enricher

Processing incoming payloads as batch not working as expected in spring-cloud-streams

I say 'not working as expected' but actually is more like 'I don't really know if I'm doing the proper work in here', I feel like I'm mixing stuff from different approaches and doesn't really correlate.
Right now I've been using Spring Cloud Streams to process String-type messages from a PubSub subscription and so far so good, message in message out without much of a hassle.
What I'm trying to achieve now is to gather, let's say, 1000 messages, process them and send them altogether to another PubSub Topic. Still unsure about sending them as a List or individually like now, but all at the same time (this shouldn't be related to this question though).
Now I just discovered the following property.
spring.cloud.stream.bindings.input.consumer.batch-mode=true
Together with the following ones more specific to the GCP stuff.
spring.cloud.gcp.pubsub.publisher.batching.enabled=true
spring.cloud.gcp.pubsub.publisher.batching.delay-threshold-seconds=300
spring.cloud.gcp.pubsub.publisher.batching.element-count-threshold=100
So first question is... Are they linked by any means? Must I have the first one together with the other three?
What happened after I added the previous properties to my application.properties file is actually no change at all. Messages keep arriving and leaving the application without any issue and with no batch approach whatsoever.
Currently using the functional features the following way.
#Bean
public Function<Message<String>, String> sampleFunction() {
... // Stream processing in here
return processedString;
}
I was expecting this to crash with some message since the method only receives a String, not a list of String. Since it didn't crash, I modified the method above to receive a list of String (maybe Spring does some magic behind the scenes to still receive messages as String but collect them in a list for the method to process afterwards?).
#Bean
public Function<Message<List<String>>, String> sampleFunction() {
... // Stream processing in here
return processedString;
}
But this just crashes since it's trying to parse a single String message as a List of String.
How could I prepare the code to batch all those String messages into a List? Is there any example on this?
...batch-mode only works with binders that support it (e.g. Kafka, RabbitMQ). It doesn't look like the GCP binder supports it (I see no references to the property).
https://github.com/spring-cloud/spring-cloud-gcp/blob/master/spring-cloud-gcp-pubsub-stream-binder/src/main/java/org/springframework/cloud/gcp/stream/binder/pubsub/PubSubMessageChannelBinder.java
https://docs.spring.io/spring-cloud-stream/docs/3.1.0/reference/html/spring-cloud-stream.html#_batch_consumers
Publisher batching is not related to consumer batching.

Spring 5 WebFlux Mono and Flux

In Spring 5 I just know Spring WebFlux Handler method handles the request and returns Mono or Flux as response.
#Component
public class HelloWorldHandler {
public Mono<ServerResponse> helloWorld(ServerRequest request) {
return ServerResponse.ok().contentType(MediaType.TEXT_PLAIN).body(BodyInserters.fromObject("Hello World"));
}
}
But I have no idea what means Mono and Flux and how it works with the WebFlux Handler.
Can any one simply explain
1.What means Mono and Flux.
2.How it works with the WebFlux Handler.
Thanks in advance.
Webflux is all about reactive programming, which in summary means that business logic is only executed as soon as the data to process it is available (reactive).
This means you no longer can return simple POJO's, but you have to return something else, something that can provide the result when it's available. Within the reactive streams initiative, this is called a Publisher. A Publisher has a subcribe() method that will allow the consumer to get the POJO when it's available.
A Publisher (for example Publisher<Foo>) can return zero or multiple, possibly infinite, results. To make it more clear how many results you can expect, Project Reactor (the reactive streams implementation of Pivotal) introduced two implementations of Publisher:
A Mono, which will complete after emitting a single result.
A Flux, which will emit zero or multiple, possibly infinite, results and then completes.
So, basically you can see Mono<Foo> as the reactive counterpart of returning Foo and Flux<Foo> as the reactive counterpart of Collection<Foo>.
For example:
Flux
.just(1, 2, 3, 4)
.map(nr -> nr * 2)
.subscribe(System.out::println);
Even though the numbers are already available (you can see them), you should realize that since it's a Flux, they're emitted one by one. In other cases, the numbers might come from an external API and in that case they won't be immediately available.
The next phase (the map operator), will multiply the number as soon as it retrieves one, this means that it also does this mapping one by one and then emit the new value.
Eventually, there's a subscriber (there should always be one, but it could be the Spring framework itself that's subscribing), and in this case it will print each value it obtains and print it to the console, also, one by one.
You should also realize that there's no particular order when processing these items. It could be that the first number has already been printed on the console, while the third item is hasn't been multiplied by two yet.
So, in your case, you have a Mono<ServerResponse>, which means that as soon as the ServerResponse is available, the WebFlux framework can utilize it. Since there is only one ServerResponse expected, it's a Mono and not a Flux.

Strategy for passing same payload between messages when optional outbound gateways fail

I have a workflow whose message payload (MasterObj) is being enriched several times. During the 2nd enrichment () an UnknownHostException was thrown by an outbound gateway. My error channel on the enricher is called but the message the error-channel receives is an exception, and the failed msg in that exception is no longer my MasterObj (original payload) but it is now the object gotten from request-payload-expression on the enricher.
The enricher calls an outbound-gateway and business-wise this is optional. I just want to continue my workflow with the payload that I've been enriching. The docs say that the error-channel on the enricher can be used to provide an alternate object (to what the enricher's request-channel would return) but even when I return an object from the enricher's error-channel, it still takes me to the workflow's overall error channel.
How do I trap errors from enricher's + outbound-gateways, and continue processing my workflow with the same payload I've been working on?
Is trying to maintain a single payload object for the entire workflow the right strategy? I need to be able to access it whenever I need.
I was thinking of using a bean scoped to the session where I store the payload but that seems to defeat the purpose of SI, no?
Thanks.
Well, if you worry about your MasterObj in the error-channel flow, don't use that request-payload-expression and let the original payload go to the enricher's sub-flow.
You always can use in that flow a simple <transformer expression="">.
On the other hand, you're right: it isn't good strategy to support single object through the flow. You carry messages via channel and it isn't good to be tied on each step. The Spring Integration purpose is to be able to switch from different MessageChannel types at any time with small effort for their producers and consumers. Also you can switch to the distributed mode when consumers and producers are on different machines.
If you still need to enrich the same object several times, consider to write some custom Java code. You can use a #MessagingGateway on the matter to still have a Spring Integration gain.
And right, scope is not good for integration flow, because you can simply switch there to a different channel type and lose a ThreadLocal context.

Resources