I am using Flux to build my reactive pipeline. In the pipeline, I need to call 3 different external system REST APIs, which are very strict with their Rate of access.
I will be throttled exponentially if i breach the rate-per-sec threshold. Each system will have their own thresholds.
I am using Spring WebClient to make the REST API calls; among 3 APIs, 2 of them are GET and 1 is POST.
In my reactor pipeline, the WebClient are wrapped within the flatMap to perform the API calls, like the below code:
WebClient getApiCall1 = WebClient.builder().build().get("api-system-1").retrieve().bodyToMono(String.class) //actual return DTO is different from string
WebClient getApiCall2 = WebClient.builder().build().get("api-system-2").retrieve().bodyToMono(String.class) //actual return DTO is different from string
WebClient getApiCall3 = WebClient.builder().build().get("api-system-3").retrieve().bodyToMono(String.class) //actual return DTO is different from string
Flux.generator(generator) // Generator pushes the elements from source 1 at a time
// make call to 1st API Service
.flatMap(data -> getApiCall1)
.map(api1Response -> api1ResponseModified)
// make call to 2nd API Service
.flatMap(api1ResponseModified -> getApiCall2)
.map(api2Response -> api2ResponseModified)
// make call to 3rd API Service
.flatMap(api2ResponseModified -> getApiCall3)
.map(api3Response -> api3ResponseModified)
// rest of the pipeline operators
//end
.subscriber();
The problem is, if I dont set the concurrency value to the flatMap, then I the pipeline execution breaching the threshold within few seconds of service startup.
If I set the value of concurrency to 1, 2, 5, 10, then the throughput is becoming very low.
The Question is, Without setting any value to the concurrency How could I achieve the Backpressure which should honour the rate limits of the external system?
Given you have a "rate per second" requirement, I would explicitly window the flux and limit each window to the chosen time period. This will give you the maximum throughput without being throttled.
I would use a helper function similar to:
public static <T> Flux<T> limitIntervalRate(Flux<T> flux, int ratePerInterval, Duration interval) {
return flux
.window(ratePerInterval)
.zipWith(Flux.interval(Duration.ZERO, interval))
.flatMap(Tuple2::getT1);
}
which allows you to do:
sourceFlux
.transform(f -> limitIntervalRate(f, 2, Duration.ofSeconds(1))) //Limit to a rate of 2 per second
You can then map this as necessary onto your WebClient calls while respecting the limit in place for each API:
sourceFlux
//...assume API 1 has a limit of 10 calls per second
.transform(f -> limitIntervalRate(f, 10, Duration.ofSeconds(1)))
.flatMap(data -> getApiCall1)
.map(api1Response -> api1ResponseModified)
//...assume API 2 has a limit of 20 calls per second
.transform(f -> limitIntervalRate(f, 20, Duration.ofSeconds(1)))
.flatMap(api1ResponseModified -> getApiCall2)
.map(api2Response -> api2ResponseModified)
...and so on.
Resilience4j has support for rate limiting with Reactor.
See:
https://resilience4j.readme.io/docs/ratelimiter
https://resilience4j.readme.io/docs/examples-1#section-decorate-mono-or-flux-with-a-ratelimiter
Related
How to write test case using StepVerifier for Flux interval and delayElements
I want to write StepVerifier for below scenarios.
Flux.interval(Duration.ofMillis(1000))
.onBackpressureDrop()
.flatMap(ignore -> doSomething())
Mono.just(1).repeat() // infinite Flux with backpressure
.delayElements(Duration.ofMillis(1000))
.concatMap(ignore -> doSomething())
TL;DR You can use StepVerifier.withVirtualTime to test time-based operators and avoid long delays. In addition, because stream is infinite, you would need to cancel subscription using thenCancel at some point.
Here are some examples
#Test
void testDelayElements() {
StepVerifier.withVirtualTime(() ->
Mono.just(1).repeat() // infinite Flux with backpressure
.delayElements(Duration.ofMillis(10000))
.concatMap(ignore -> doSomething())
)
.expectSubscription()
.expectNoEvent(Duration.ofMillis(10000))
.expectNextCount(1)
.expectNoEvent(Duration.ofMillis(10000))
.expectNextCount(1)
.expectNoEvent(Duration.ofMillis(10000))
.expectNextCount(1)
.thenCancel()
.verify();
}
For more information, check Manipulating Time section in the Reactor 3 Reference Guide.
Very important point from the documentation
Take extra care to ensure the Supplier<Publisher> can be used in a lazy fashion. Otherwise, virtual time is not guaranteed. Especially avoid instantiating the Flux earlier in the test code and having the Supplier return that variable. Instead, always instantiate the Flux inside the lambda.
Note that the publisher is created lazily using the Supplier<Publisher<T>>. For example, the following will not work as expected
#Test
void testDelayElements() {
var stream = Mono.just(1).repeat() // infinite Flux with backpressure
.delayElements(Duration.ofMillis(10000))
.concatMap(ignore -> doSomething());
StepVerifier.withVirtualTime(() -> stream)
....
}
How can I guarantee linearizability of requests in Reactor Netty?
Theory:
Given:
Request A wants to write x=2, y=0
Request B wants to read x, y and write x=x+2, y=y+1
Request C wants to read x and write y=x
All Requests are processed asynchronously and return to the client immediately with status ACCEPTED.
Example:
Send requests A, B, C in order.
Example Log Output: (request, thread name, x, y)
Request A, nioEventLoopGroup-2-0, x=2, y=0
Request C, nioEventLoopGroup-2-2, x=2, y=2
Request B, nioEventLoopGroup-2-1, x=4, y=3
Business logic requires all reads after A to see x=2 and y=0.
And request B to see x=2, y=0 and set y=1.
And request C to see x=4 and set y=4.
In short: The business logic makes every next write operation dependent on the previous write operation to be completed. Otherwise the operations are not reversible.
Example Code
Document:
#Document
#Data
#NoArgsConstructor
#AllArgsConstructor
public class Event {
#Id
private String id;
private int data;
public Event withNewId() {
setId(UUID.randomUUID().toString());
return this;
}
}
Repo:
public interface EventRepository extends ReactiveMongoRepository<Event, String> {}
Controller:
#RestController
#RequestMapping(value = "/api/event")
#RequiredArgsConstructor
public class EventHandler {
private final EventRepository repo;
#PostMapping
public Mono<String> create(Event event) {
return Mono.just(event.withNewId().getId())
.doOnNext(id ->
// do query based on some logic depending on event data
Mono.just(someQuery)
.flatMap(query ->
repo.find(query)
.map(e -> event.setData(event.getData() + e.getData())))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e))
.subscribeOn(Schedulers.single())
.subscribe());
}
}
It does not work, but with subscribeOn I try to guarantee linearizability. Meaning that concurrent requests A and B will always write their payload to the DB in the order in which they are received by the server. Therefore if another concurrent request C is a compound of first read than write, it will read changes from the DB that reflect those of request B, not A, and write its own changes based of B.
Is there a way in Reactor Netty to schedule executors with an unbound FIFO queue, so that I can process the requests asynchronously but in order?
I don't think that this is specific to Netty or Reactor in particular, but to a more broad topic - how to handle out-of-order message delivery and more-than-once message delivery. A few questions:
Does the client always sends the same number of requests in the same order? There's always a chance that, due to networking issues the requests may arrive out of order, or one or more may be lost.
Does the client make retries? What happens if the same request reaches the server twice?
If the order matters, why doesn't the client wait for the result of the nth-1 request, before issuing nth request? In other words, why there are many concurrent requests?
I'd try to redesign the operation in such a way that there's a single request executing the operations on the backend in the required order and using concurrency here if necessary to speed-up the process.
If it's not possible, for example, you don't control the client, or more generally the order in which the events (requests) arrive, you have to implement ordering on application-level logic using per-message semantics to do the ordering. You can, for example store or buffer the messages, waiting for all to arrive, and when they do, only then trigger the business logic using the data from the messages in the correct order. This requires some kind of a key (identity) which can attribute messages to the same entity, and a sorting-key, that you know how to sort the messages in the correct order.
EDIT:
After getting the answers, you can definitely implement it "the Reactor way".
Sinks.Many<Event> sink = Sinks.many() // you creat a 'sink' where the events will go
.multicast() // broads all messages to all subscribes of the stream
.directBestEffort(); // additional semantics - publishing will fail if no subscribers - doesn't really matter here
Flux<Event> eventFlux = sink.asFlux(); // the 'view' of the sink as a flux you can subscribe to
public void run() {
subscribeAndProcess();
sink.tryEmitNext(new Event("A", "A", "A"));
sink.tryEmitNext(new Event("A", "C", "C"));
sink.tryEmitNext(new Event("A", "B", "B"));
sink.tryEmitNext(new Event("B", "A", "A"));
sink.tryEmitNext(new Event("B", "C", "C"));
sink.tryEmitNext(new Event("B", "B", "B"));
}
void subscribeAndProcess() {
eventFlux.groupBy(Event::key)
.flatMap(
groupedEvents -> groupedEvents.distinct(Event::type) // distinct to avoid duplicates
.buffer(3) // there are three event types, so we buffer and wait for all to arrive
.flatMap(events -> // once all the events are there we can do the processing the way we need
Mono.just(events.stream()
.sorted(Comparator.comparing(Event::type))
.map(e -> e.key + e.value)
.reduce(String::concat)
.orElse(""))
)
)
.subscribe(System.out::println);
}
// prints values concatenated in order per key:
// - AAABAC
// - BABBBC
See Gist: https://gist.github.com/tarczynskitomek/d9442ea679e3eed64e5a8470217ad96a
There are a few caveats:
If all of the expected events for the given key don't arrive you waste memory buffering - unless you set a timeout
How will you ensure that all the events for a given key go to the same application instance?
How will you recover from failures encountered mid-processing?
Having all this in mind, I would go with a persistent storage - say saving the incoming events in the database, and doing the processing in background - for this you don't need to use Reactor. Most of the time a simple Servlets based Spring app will be far easier to maintain and develop, especially if you have no previous experience with Functional Reactive Programming.
Looking at the provided code I would not try to handle it on Reactor Netty level.
At first, several comments regarding controller implementation because it has multiple issues that violate reactive principles. I would recommend to spend some time learning reactive API but here are some hints
In reactive nothing happens until you subscribe. At the same time calling subscribe explicitly is an anti-pattern and should be avoided until you are creating framework similar to WebFlux.
parallel scheduler should be used to run non-blocking logic until you have some blocking code.
doOn... are so-called side-effect operators and should not be used for constructing reactive flows.
#PostMapping
public Mono<String> create(Event event) {
// do query based on some logic depending on event data
return repo.find(query)
.map(e -> event.setData(event.getData() + e.getData()))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e));
}
Now, processing requests in the predefined sequence could be tricky because of network failures, possible retries, etc. What if you never get Request B or Request C? Should you still persist Request A?
As #ttarczynski mentioned in his comment the best option is to redesign API and send single request.
In case it's not an option you would need to introduce some state to "postpone" request processing and then, depending on consistency semantic, process them as a "batch" when the last request is received or just defer Request C until you get Request A & B.
I am exploring micrometer and aws cloudwatch. I think there is some understanding gap -
I've create a gauge which is supposed to return the number of connections being used in a connection pool.
public MetricService(CloudWatchConfig config) {
this.cloudwatchMeterRegistry = new CloudWatchMeterRegistry(config, Clock.SYSTEM, CloudWatchAsyncClient.create());
gauge = Gauge.builder("ConnectionPoolGauge", this.connectionPool, value -> {
Double usedConnections = 0.0;
for (Map.Entry<String, Boolean> entry : value.entrySet()) {
if (entry.getValue().equals(Boolean.FALSE)) {
usedConnections++;
}
}
return usedConnections;
})
.tag("GaugeName", "Bhushan's Gauge")
.strongReference(true)
.baseUnit("UsedConnections")
.description("Gauge to monitor connection pool")
.register(Metrics.globalRegistry);
Metrics.addRegistry(cloudwatchMeterRegistry);
}
As you can see, I am currently initiating this gauge in a constructor. Passing the connectionPool instance from outside.
Following is a controller method which consumes the connection -
#GetMapping("/hello")
public String hello() {
// connectionPool.consumeConnection();
// finally { connectionPool.releaseConnection();}
}
Step interval is set to 10 seconds. My understanding is - Every 10 seconds, Micrometer should automatically execute the double function passed to the gauge.
Obviously, it is not happening. I've seen some code samples here which are explicitly setting the gauge value (in a separate thread or scheduled logic).
I also tried with a counter which is instantiated only once, but I explicitly invoke the increment method per call to hello method. My expectation was this counter would keep on incrementing, but after a while, it drops to 0 and starts counting again.
I am totally confused. Appreciate if someone can put light on this concept.
Edit:
Tried following approach for creating Gauge - still no luck.
cloudwatchMeterRegistry.gauge("ConnectionPoolGauge", this.connectionPool, value -> {
Double usedConnections = 0.0;
System.out.println("Inside Guage Value function." + value.entrySet());
for (Map.Entry<String, Boolean> entry : value.entrySet()) {
if (entry.getValue().equals(Boolean.FALSE)) {
usedConnections++;
}
}
return usedConnections;
});
This doesn't return the instance of Gauge, so I cannot call value() on it. Also the gauge is not visible in AWS Cloudwatch. I can see the counter in cloudwatch that I created in the same program.
Micrometer takes the stance that gauges should be sampled and not be set, so there is no information about what might have occurred between samples. After all, any intermediate values set on a gauge are lost by the time the gauge value is reported to a metrics backend anyway, so there seems to be little value in setting those intermediate values in the first place.
If it helps, think of a Gauge as a "heisen-gauge" - a meter that only changes when it is observed. Every other meter type provided out-of-the-box accumulates intermediate counts toward the point where the data is sent to the metrics backend.
So the gauge is updated when the metrics are published, here are a few tips to troubleshooting this:
Put a brake point in the publish method of your CloudWatchMeterRegistry and see if it is called or not.
You are using the Global registry (Metrics.addRegistry) as well as keeping the reference to CloudWatchMeterRegistry (this.cloudwatchMeterRegistry = new CloudWatchMeterRegistry). You don't need both, I would suggest to do not use the Global registry and inject the registry you have wherever you need it.
I'm not sure what you are doing with the connection pool (did you implement your own one?) but there is out-of-the-box support for HikariCP and DBCP is publishing JMX counters that you can bind to Micrometer.
I have the method below, where I am calling several ReactiveMongoRepositories in order to receive and process certain documents. Since I am kind of new to Webflux, I am learning as I go.
To my feeling the code below doesn't feel very efficient, as I am opening multiple streams at the same time. This non-blocking way of writing code makes it complicated somehow to get a value from a stream and re-use that value in the cascaded flatmaps down the line.
In the example below I have to call the userRepository twice, since I want the user at the beginning and than later as well. Is there a possibility to do this more efficiently with Webflux?
public Mono<Guideline> addGuideline(Guideline guideline, String keycloakUserId) {
Mono<Guideline> guidelineMono = userRepository.findByKeycloakUserId(keycloakUserId)
.flatMap(user -> {
return teamRepository.findUserInTeams(user.get_id());
}).zipWith(instructionRepository.findById(guideline.getInstructionId()))
.zipWith(userRepository.findByKeycloakUserId(keycloakUserId))
.flatMap(objects -> {
User user = objects.getT2();
Instruction instruction = objects.getT1().getT2();
Team team = objects.getT1().getT1();
if (instruction.getTeamId().equals(team.get_id())) {
guideline.setAddedByUser(user.get_id());
guideline.setTeamId(team.get_id());
guideline.setDateAdded(new Date());
guideline.setGuidelineStatus(GuidelineStatus.ACTIVE);
guideline.setGuidelineSteps(Arrays.asList());
return guidelineRepository.save(guideline);
} else {
return Mono.error(new InstructionDoesntBelongOrExistException("Unable to add, since this Instruction does not belong to you or doesn't exist anymore!"));
}
});
return guidelineMono;
}
i'll post my earlier comment as an answer. If anyone feels like writing the correct code for it then go ahead.
i don't have access to an IDE current so cant write an example but you could start by fetching the instruction from the database.
Keep that Mono<Instruction> then you fetch your User and flatMap the User and fetch the Team from the database. Then you flatMap the team and build a Mono<Tuple> consisting of Mono<Tuple<User, Team>>.
After that you take your 2 Monos and use zipWith with a Combinator function and build a Mono<Tuple<User, Team, Instruction>> that you can flatMap over.
So basically fetch 1 item, then fetch 2 items, then Combinate into 3 items. You can create Tuples using the Tuples.of(...) function.
I am using Akka in Play Controller and performing ask() to a actor by name publish , and internal publish actor performs ask to multiple actors and passes reference of sender. The controller actor needs to wait for response from multiple actors and create a list of response.
Please find the code below. but this code is only waiting for 1 response and latter terminating. Please suggest
// Performs ask to publish actor
Source<Object,NotUsed> inAsk = Source.fromFuture(ask(publishActor,service.getOfferVerifyRequest(request).getPayloadData(),1000));
final Sink<String, CompletionStage<String>> sink = Sink.head();
final Flow<Object, String, NotUsed> f3 = Flow.of(Object.class).map(elem -> {
log.info("Data in Graph is " +elem.toString());
return elem.toString();
});
RunnableGraph<CompletionStage<String>> result = RunnableGraph.fromGraph(
GraphDSL.create(
sink , (builder , out) ->{
final Outlet<Object> source = builder.add(inAsk).out();
builder
.from(source)
.via(builder.add(f3))
.to(out); // to() expects a SinkShape
return ClosedShape.getInstance();
}
));
ActorMaterializer mat = ActorMaterializer.create(aSystem);
CompletionStage<String> fin = result.run(mat);
fin.toCompletableFuture().thenApply(a->{
log.info("Data is "+a);
return true;
});
log.info("COMPLETED CONTROLLER ");
If you have several responses ask won't cut it, that is only for a single request-response where the response ends up in a Future/CompletionStage.
There are a few different strategies to wait for all answers:
One is to create an intermediate actor whose only job is to collect all answers and then when all partial responses has arrived respond to the original requestor, that way you could use ask to get a single aggregate response back.
Another option would be to use Source.actorRef to get an ActorRef that you could use as sender together with tell (and skip using ask). Inside the stream you would then take elements until some criteria is met (time has passed or elements have been seen). You may have to add an operator to mimic the ask response timeout to make sure the stream fails if the actor never responds.
There are some other issues with the code shared, one is creating a materializer on each request, these have a lifecycle and will fill up your heap over time, you should rather get a materializer injected from play.
With the given logic there is no need whatsoever to use the GraphDSL, that is only needed for complex streams with multiple inputs and outputs or cycles. You should be able to compose operators using the Flow API alone (see for example https://doc.akka.io/docs/akka/current/stream/stream-flows-and-basics.html#defining-and-running-streams )