When protecting a method using Retry, should I create a new instance each time or reuse the same instance from the RetryRegistry?
Which one should I use?
A)
for(int i: mylist){
Retry retry = Retry.ofDefaults("backendName");
Mono.fromCallable(backendService::doSomething).transformDeferred(RetryOperator.of(retry))
}
or
B)
RetryRegistry retryRegistry = RetryRegistry.ofDefaults();
for(int i: mylist){
Retry retry = registry.retry("backendName");
Mono.fromCallable(backendService::doSomething).transformDeferred(RetryOperator.of(retry))
}
Shouldn't a Retry always be a new instance to keep track of the states and the number of attempt or is the registry behind the scene returns a new instance when we invoke it?
You must use b).
You must not create a new Retry instace per call. The Retry instance internally creates a new Retry context per subscription.
You should even do the lookup only once in your constructor.
Related
How can I guarantee linearizability of requests in Reactor Netty?
Theory:
Given:
Request A wants to write x=2, y=0
Request B wants to read x, y and write x=x+2, y=y+1
Request C wants to read x and write y=x
All Requests are processed asynchronously and return to the client immediately with status ACCEPTED.
Example:
Send requests A, B, C in order.
Example Log Output: (request, thread name, x, y)
Request A, nioEventLoopGroup-2-0, x=2, y=0
Request C, nioEventLoopGroup-2-2, x=2, y=2
Request B, nioEventLoopGroup-2-1, x=4, y=3
Business logic requires all reads after A to see x=2 and y=0.
And request B to see x=2, y=0 and set y=1.
And request C to see x=4 and set y=4.
In short: The business logic makes every next write operation dependent on the previous write operation to be completed. Otherwise the operations are not reversible.
Example Code
Document:
#Document
#Data
#NoArgsConstructor
#AllArgsConstructor
public class Event {
#Id
private String id;
private int data;
public Event withNewId() {
setId(UUID.randomUUID().toString());
return this;
}
}
Repo:
public interface EventRepository extends ReactiveMongoRepository<Event, String> {}
Controller:
#RestController
#RequestMapping(value = "/api/event")
#RequiredArgsConstructor
public class EventHandler {
private final EventRepository repo;
#PostMapping
public Mono<String> create(Event event) {
return Mono.just(event.withNewId().getId())
.doOnNext(id ->
// do query based on some logic depending on event data
Mono.just(someQuery)
.flatMap(query ->
repo.find(query)
.map(e -> event.setData(event.getData() + e.getData())))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e))
.subscribeOn(Schedulers.single())
.subscribe());
}
}
It does not work, but with subscribeOn I try to guarantee linearizability. Meaning that concurrent requests A and B will always write their payload to the DB in the order in which they are received by the server. Therefore if another concurrent request C is a compound of first read than write, it will read changes from the DB that reflect those of request B, not A, and write its own changes based of B.
Is there a way in Reactor Netty to schedule executors with an unbound FIFO queue, so that I can process the requests asynchronously but in order?
I don't think that this is specific to Netty or Reactor in particular, but to a more broad topic - how to handle out-of-order message delivery and more-than-once message delivery. A few questions:
Does the client always sends the same number of requests in the same order? There's always a chance that, due to networking issues the requests may arrive out of order, or one or more may be lost.
Does the client make retries? What happens if the same request reaches the server twice?
If the order matters, why doesn't the client wait for the result of the nth-1 request, before issuing nth request? In other words, why there are many concurrent requests?
I'd try to redesign the operation in such a way that there's a single request executing the operations on the backend in the required order and using concurrency here if necessary to speed-up the process.
If it's not possible, for example, you don't control the client, or more generally the order in which the events (requests) arrive, you have to implement ordering on application-level logic using per-message semantics to do the ordering. You can, for example store or buffer the messages, waiting for all to arrive, and when they do, only then trigger the business logic using the data from the messages in the correct order. This requires some kind of a key (identity) which can attribute messages to the same entity, and a sorting-key, that you know how to sort the messages in the correct order.
EDIT:
After getting the answers, you can definitely implement it "the Reactor way".
Sinks.Many<Event> sink = Sinks.many() // you creat a 'sink' where the events will go
.multicast() // broads all messages to all subscribes of the stream
.directBestEffort(); // additional semantics - publishing will fail if no subscribers - doesn't really matter here
Flux<Event> eventFlux = sink.asFlux(); // the 'view' of the sink as a flux you can subscribe to
public void run() {
subscribeAndProcess();
sink.tryEmitNext(new Event("A", "A", "A"));
sink.tryEmitNext(new Event("A", "C", "C"));
sink.tryEmitNext(new Event("A", "B", "B"));
sink.tryEmitNext(new Event("B", "A", "A"));
sink.tryEmitNext(new Event("B", "C", "C"));
sink.tryEmitNext(new Event("B", "B", "B"));
}
void subscribeAndProcess() {
eventFlux.groupBy(Event::key)
.flatMap(
groupedEvents -> groupedEvents.distinct(Event::type) // distinct to avoid duplicates
.buffer(3) // there are three event types, so we buffer and wait for all to arrive
.flatMap(events -> // once all the events are there we can do the processing the way we need
Mono.just(events.stream()
.sorted(Comparator.comparing(Event::type))
.map(e -> e.key + e.value)
.reduce(String::concat)
.orElse(""))
)
)
.subscribe(System.out::println);
}
// prints values concatenated in order per key:
// - AAABAC
// - BABBBC
See Gist: https://gist.github.com/tarczynskitomek/d9442ea679e3eed64e5a8470217ad96a
There are a few caveats:
If all of the expected events for the given key don't arrive you waste memory buffering - unless you set a timeout
How will you ensure that all the events for a given key go to the same application instance?
How will you recover from failures encountered mid-processing?
Having all this in mind, I would go with a persistent storage - say saving the incoming events in the database, and doing the processing in background - for this you don't need to use Reactor. Most of the time a simple Servlets based Spring app will be far easier to maintain and develop, especially if you have no previous experience with Functional Reactive Programming.
Looking at the provided code I would not try to handle it on Reactor Netty level.
At first, several comments regarding controller implementation because it has multiple issues that violate reactive principles. I would recommend to spend some time learning reactive API but here are some hints
In reactive nothing happens until you subscribe. At the same time calling subscribe explicitly is an anti-pattern and should be avoided until you are creating framework similar to WebFlux.
parallel scheduler should be used to run non-blocking logic until you have some blocking code.
doOn... are so-called side-effect operators and should not be used for constructing reactive flows.
#PostMapping
public Mono<String> create(Event event) {
// do query based on some logic depending on event data
return repo.find(query)
.map(e -> event.setData(event.getData() + e.getData()))
.switchIfEmpty(Mono.just(event))
.flatMap(e -> repo.save(e));
}
Now, processing requests in the predefined sequence could be tricky because of network failures, possible retries, etc. What if you never get Request B or Request C? Should you still persist Request A?
As #ttarczynski mentioned in his comment the best option is to redesign API and send single request.
In case it's not an option you would need to introduce some state to "postpone" request processing and then, depending on consistency semantic, process them as a "batch" when the last request is received or just defer Request C until you get Request A & B.
I'm trying to cache http calls in the service so all subsequent calls returns same response. This is fairly easy with shareReplay:
data = this.http.get(url).pipe(
shareReplay(1)
);
But it doesn't work in case of backend / network errors. ShareReplay spams the backend with requests in case of any error when this Observable is bound to the view through async pipe.
I tried with retryWhen etc but the solution I got is untestable:
data = this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay(1)
);
fakeAsync tests fails with "1 timer(s) still in the queue" error because delay timer has no end condition. I also don't want to have some hanging endless timer in the background - it should stop with the last subscription.
The behavior I would like:
Multicast - make only one subscription to source even with many subscribers.
Do not count refs for successful queries - reuse same result when subscriber count goes to 0 and back to 1.
In case of error - retry every 10 seconds but only if there are any subscribers.
My 2 cents:
This code is for rxjs > 6.4 (here V6.6)
To use a shared observable, you need to return the same observable for all the subscribers (or you will create an observable which has nothing to share)
Multicasting can be done using shareReplay and you can replay the last emitted value (even after the last subscriber to have unsubscribed) using the {refCount: false} option.
As long as there is no subscription, the observable does nothing. You will not have any fetch on the server before the first subscriber.
beware:
If refCount is false, the source will not be
unsubscribed meaning that the inner ReplaySubject will still be
subscribed to the source (and potentially run for ever).
Also:
A successfully completed source will stay cached in the shareReplayed
observable forever, but an errored source can be retried.
The problem is using shareReplay, you have to choose between:
Always getting the last value even if the refCount went back to 0 and having possible never ending retries in case of error (remember shareReplay with refCount to false never unsubscribes)
Or keeping the default refCount:true which mean you won't have the second "first subscriber" cache benefit. Conversely the retry will also stop if no subscriber is there.
Here is a dummy example:
class MyServiceClass {
private data;
// assuming you are injecting the http service
constructor(private http: HttpService){
this.data = this.buildData("http://some/data")
}
// use this accessor to get the unique (shared) instance of data observable.
public getData(){
return this.data;
}
private buildData(url: string){
return this.http.get(url).pipe(
retryWhen(errors => errors.pipe(delay(10000))),
shareReplay({refCount: false})
);
}
}
Now in my opinion, to fix the flow you should prevent your retry to run forever, adding for instance a maximum number of retries
I am exploring micrometer and aws cloudwatch. I think there is some understanding gap -
I've create a gauge which is supposed to return the number of connections being used in a connection pool.
public MetricService(CloudWatchConfig config) {
this.cloudwatchMeterRegistry = new CloudWatchMeterRegistry(config, Clock.SYSTEM, CloudWatchAsyncClient.create());
gauge = Gauge.builder("ConnectionPoolGauge", this.connectionPool, value -> {
Double usedConnections = 0.0;
for (Map.Entry<String, Boolean> entry : value.entrySet()) {
if (entry.getValue().equals(Boolean.FALSE)) {
usedConnections++;
}
}
return usedConnections;
})
.tag("GaugeName", "Bhushan's Gauge")
.strongReference(true)
.baseUnit("UsedConnections")
.description("Gauge to monitor connection pool")
.register(Metrics.globalRegistry);
Metrics.addRegistry(cloudwatchMeterRegistry);
}
As you can see, I am currently initiating this gauge in a constructor. Passing the connectionPool instance from outside.
Following is a controller method which consumes the connection -
#GetMapping("/hello")
public String hello() {
// connectionPool.consumeConnection();
// finally { connectionPool.releaseConnection();}
}
Step interval is set to 10 seconds. My understanding is - Every 10 seconds, Micrometer should automatically execute the double function passed to the gauge.
Obviously, it is not happening. I've seen some code samples here which are explicitly setting the gauge value (in a separate thread or scheduled logic).
I also tried with a counter which is instantiated only once, but I explicitly invoke the increment method per call to hello method. My expectation was this counter would keep on incrementing, but after a while, it drops to 0 and starts counting again.
I am totally confused. Appreciate if someone can put light on this concept.
Edit:
Tried following approach for creating Gauge - still no luck.
cloudwatchMeterRegistry.gauge("ConnectionPoolGauge", this.connectionPool, value -> {
Double usedConnections = 0.0;
System.out.println("Inside Guage Value function." + value.entrySet());
for (Map.Entry<String, Boolean> entry : value.entrySet()) {
if (entry.getValue().equals(Boolean.FALSE)) {
usedConnections++;
}
}
return usedConnections;
});
This doesn't return the instance of Gauge, so I cannot call value() on it. Also the gauge is not visible in AWS Cloudwatch. I can see the counter in cloudwatch that I created in the same program.
Micrometer takes the stance that gauges should be sampled and not be set, so there is no information about what might have occurred between samples. After all, any intermediate values set on a gauge are lost by the time the gauge value is reported to a metrics backend anyway, so there seems to be little value in setting those intermediate values in the first place.
If it helps, think of a Gauge as a "heisen-gauge" - a meter that only changes when it is observed. Every other meter type provided out-of-the-box accumulates intermediate counts toward the point where the data is sent to the metrics backend.
So the gauge is updated when the metrics are published, here are a few tips to troubleshooting this:
Put a brake point in the publish method of your CloudWatchMeterRegistry and see if it is called or not.
You are using the Global registry (Metrics.addRegistry) as well as keeping the reference to CloudWatchMeterRegistry (this.cloudwatchMeterRegistry = new CloudWatchMeterRegistry). You don't need both, I would suggest to do not use the Global registry and inject the registry you have wherever you need it.
I'm not sure what you are doing with the connection pool (did you implement your own one?) but there is out-of-the-box support for HikariCP and DBCP is publishing JMX counters that you can bind to Micrometer.
In rxjs5, I'm trying to implement a Throttler class.
import Rx from 'rxjs/rx';
export default class Throttler {
constructor(interval) {
this.timeouts = [];
this.incomingActions = new Rx.Subject();
this.incomingActions
.concatMap(action => Rx.Observable.just(action).delay(interval / 2))
.subscribe(action => action());
}
clear() {
// How do I do this?
}
do(action) {
this.incomingActions.next(action);
}
}
The following invariants must hold:
every action passed to do gets added to an action queue
the action queue gets processed in order and at a fixed interval as determined by the constructor parameter
the action queue can be cleared using clear().
My current implementation, as seen above, handles the fixed interval, but I don't know how to clear the queue. It also has the problem that all actions are delayed by interval / 2ms even when the queue is empty.
P.S. The way I describe the invariants maps very easily to an implementation with setInterval and an array as a queue, but I'm wondering how I would do this with Rx.
This seems like not a good place for the default Subject class. Extending it with your own subclass would be better because of reasons you listed.
However, in your case I'd try to identify each action that comes to .do(action) method with some index and add .filter() operator before subscribe() to be able to cancel particular actions by checking some array for what indices are marked as canceled. Since you're using concatMap() you know that actions will be always called in the order they were added. Then clear() method that you want would just mark all actions to be canceled in the array.
You can also add .do() operator after concatMap() and keep track of how many action are scheduled at the moment with some accumulator. Adding action would cause scheduledAction++ while passing .do() right before .subscribe() would scheduledAction--. Then you can use this variable to decide whether you want to chain a new action with .delay(interval / 2) or not.
I am developing my customized debugger as an eclipse plugin. I am using the JPDA API to this end. I would like to retrieve the value of some object-reference variable. Therefore, I try to use ObjectReference.invokeMethod by invoking toString() method. My code is as follows:
if(thread.isSuspended()){
Method method = retriveToStringMethod(...);
Value messageValue = objValue.invokeMethod(thread, method, new ArrayList<Value>(), ObjectReference.INVOKE_SINGLE_THREADED);
stringValue = messageValue.toString();
}
However, it sometime does not work. For example, Given the following code:
1. public static void main(String[] args) {
2. InsertIntervalBug6 insert = new InsertIntervalBug6();
3.
4. Interval i1 = new Interval(1, 2);
5. Interval i2 = new Interval(3, 4);
6.
7. }
It works fine in line 4, I can successfully get the result by invoking toString() method of insert variable. However, when in line 5, a TimeOutException is reported. However, I have already set the timeout option when starting JVM at 10s, therefore I think such a period is long enough to retrieve the result of toString() method invocation. The trace stack is as follows. Do you have any idea about the problem? Thanks!
org.eclipse.jdi.TimeoutException: Timeout occurred while waiting for packet 586.
at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:186)
at org.eclipse.jdi.internal.connect.PacketReceiveManager.getReply(PacketReceiveManager.java:197)
at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:191)
at org.eclipse.jdi.internal.MirrorImpl.requestVM(MirrorImpl.java:226)
at org.eclipse.jdi.internal.ObjectReferenceImpl.invokeMethod(ObjectReferenceImpl.java:428)
at microbat.codeanalysis.runtime.variable.VariableValueExtractor.setMessageValue(VariableValueExtractor.java:518)
I have solved this problem by myself. I share the solution in this answer as follows:
The TimeoutException is caused by deadlock. When I am visiting the toString() method, it triggers a step request to JVM. However, my program is listening to any step request sent from the debugged program so that it is able to capture the stepping event and suspend the program for checking variable values. Therefore, a programmatical invocation of the toString() method suspend the program itself, the invokeMethod() wait the suspended program until the time output.
The solution is to disable the set step request. Afterward, the deadline lock problem disappears.