Shortcut chain of completablefuture based on a condition - java-8

I want to skip chain of completablefutures based a specific condition. I tried the solution proposed at chaining several completionstage, but it doesn't seem to work. Here it the code:
#RunWith(JUnit4.class)
public class ExceptionHandlingTests {
#Test
public void test1() {
CompletableFuture<Integer> result = new CompletableFuture<>();
CompletableFuture.runAsync(() -> {
System.out.println("Completing result1. Result: " + result.isDone());
result.complete(10);
}).thenCompose(x -> {
System.out.println("Completing result2. Result: " + result.isDone());
result.complete(10);
return CompletableFuture.completedFuture(5);
}).thenCompose(x -> {
System.out.println("Completing result3. Result: " + result.isDone());
result.complete(10);
return CompletableFuture.completedFuture(5);
}).applyToEither(result, Function.identity());
}
}
Output:
Completing result1. Result: false
Completing result2. Result: true
Completing result3. Result: true
Even though the "result" completablefuture is marked completed, subsequent completablefutures are still executed. How to skip Completablefuture 2 and 3?

You have created a dependency chain like this:
first
↓ (↘)
next result
↓ ↙
final
The (↘) is an explicit completion call result.complete(…), but all other completions happen automatically. The final stage, created via applyToEither will be completed with either of the prerequisites, whichever is completed first, but does not modify the behavior of them.
In principle, any code could invoke complete on it without affecting either of these stages that might complete the final stage otherwise. The same applies to cancellation. Calling cancel on a stage will complete the stage, your calling the method on, without affecting the stages you used to construct it.
The answer of the linked question creates stages like
first
(↙) (↘)
next 1 result
↓ |
next 2 |
↘ ↙
final
The key point is, the initial stage will complete either of two CompletableFutures explicitly, not triggering an automatic completion. The other chain of dependent stages will never get evaluated. Since the final stage is an applyToEither, the completion of only one of the chains is sufficient to evaluate the final function. It still does not affect the prerequisite stages.
Note that for your chain, consisting of thenCompose operations, you can implement a similar logic in a simpler fashion, due to the fact that your functions return a CompletableFuture anyway. So simply returning a new CompletableFuture that will never be completed when you want to shortcut, solves the issue. You can even rewrite your initial runAsync to use thenCompose instead:
for(int shortCutAt: IntStream.range(0, 4).toArray()) {
System.out.println("Example execution with "
+(shortCutAt==0? "no shortcut": "shortcut at "+shortCutAt));
CompletableFuture<Integer> result = new CompletableFuture<>();
CompletableFuture.completedFuture(null).thenCompose(justVoid -> { // runAsync
System.out.println("Completing result1. Result: " + result.isDone());
if(shortCutAt == 1) { result.complete(10); return new CompletableFuture<>(); }
return CompletableFuture.completedFuture(justVoid);
}).thenCompose(x -> {
System.out.println("Completing result2. Result: " + result.isDone());
if(shortCutAt == 2) { result.complete(10); return new CompletableFuture<>(); }
return CompletableFuture.completedFuture(5);
}).thenCompose(x -> {
System.out.println("Completing result3. Result: " + result.isDone());
if(shortCutAt == 3) { result.complete(10); return new CompletableFuture<>(); }
return CompletableFuture.completedFuture(5);
})
.applyToEither(result, Function.identity())
.thenAccept(fr -> System.out.println("final result: "+fr));
System.out.println();
}
Example execution with no shortcut
Completing result1. Result: false
Completing result2. Result: false
Completing result3. Result: false
final result: 5
Example execution with shortcut at 1
Completing result1. Result: false
final result: 10
Example execution with shortcut at 2
Completing result1. Result: false
Completing result2. Result: false
final result: 10
Example execution with shortcut at 3
Completing result1. Result: false
Completing result2. Result: false
Completing result3. Result: false
final result: 10

Related

What will be the output of the below java code in case of partial failure during API calls?

I have a list of total 115 inputs, which is divided into a packets size of 20 inputs each using the below code, which means xoneIndexListOfList has 6 elements with size [ [20], [20], [20], [20], [20], [15] ] respectively.
final AtomicInteger counter = new AtomicInteger();
final Collection<List<XoneIndexRequest>> xoneIndexListOfList = xoneIndexRequestList.stream()
.collect(Collectors.groupingBy(it -> counter.getAndIncrement() / xoneIndicesPacketSize))
.values();
In second step, I am iterating over this list and passing each packet as an input to some REST ENDPOINT, using the below code.
return xoneIndexListOfList.stream()
.map(e -> {
return getResponseBody(creditIndicesApiClient.creditIndicesSearch7(e));
}).collect(Collectors.toList());
From the above code, we could see that 6 calls will go to the above API(i.e.one for each packet), so here my question what will happen in case there is some partial failure?
Scenario :
Suppose packet 1 and 2 passed and packet 3 failed ( i.e. API not able to return data for 3rd packets)
**Question : **
In the above scenario will it proceed to 4 and 5 packets? If yes, then suppose packets 4 and 5 also passed, so in total packet 1,2,4,5 passed but 3 failed, so what will be the return output from the above code?
If no, they will be just returned the output for packets 1 and 2 because it was already passed and the request failed for packet 3?
or will it marked each packet to be failed if any one of it would fail at any time, of this packet list processing?
EDIT - Adding the full method
#Override
public List<XoneIndexResponse> creditIndicesSearch7(List<XoneIndexRequest> xoneIndexRequestList) {
try {
final AtomicInteger counter = new AtomicInteger();
final Collection<List<XoneIndexRequest>> xoneIndexListOfList = xoneIndexRequestList.stream()
.collect(Collectors.groupingBy(it -> counter.getAndIncrement() / xoneIndicesPacketSize))
.values();
return xoneIndexListOfList.stream()
.map(e -> {
return getResponseBody(creditIndicesApiClient.creditIndicesSearch7(e));
}).collect(Collectors.toList());
} catch (Exception e) {
log.error("error while retrieving data from x-one! for Index", e);
return emptyList();
}
}
When an unhandled exception occurs, the processing of the stream stops immediately.
So you won't get a list unless you handle the exception.
We can test it with the following code:
Stream.of(6, 0, 7)
.map(integer -> 5/integer + integer)
.forEach(System.out::println);
Output:
6
Exception in thread "main" java.lang.ArithmeticException: / by zero
at com.example.demo.DemoApplication.lambda$main$8(DemoApplication.java:244)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at com.example.demo.DemoApplication.main(DemoApplication.java:245)
It depends on what we are trying to do, but we can fix it by:
Stream.of(6, 0, 7)
.map(integer -> {
try{
return Optional.of(5/integer + integer);
}catch (ArithmeticException e){
return Optional.empty();
}
})
.filter(Optional::isPresent)
.map(Optional::get)
.forEach(System.out::println);
Output:
6
7
Edit:
When the whole thing is wrapped in a try scope, the exception that was thrown from the stream will propagate and be caught in the catch scope.
So in your case, an error will be printed to log and emptyList() will be called.

Spring Integration DSL how to route split messages to different concurrent flows?

I probably hate writing noob questions as much as other people hate answering them, but here goes.
I need to split a message retrieved from a JdbcPollingChannelAdapter into multiple messages based on the operation requested in each row of the resultset in the payload.
The split operation is simple enough. What is proving to be a challenge is conditionally routing the message to one flow or the other.
After much trial and error, I believe that this flow represents my intention
/- insertUpdateAdapter -\
Poll Table -> decorate headers -> split -> router -< >- aggregator -> cleanup
\---- deleteAdapter ----/
TO that end I have constructed this Java DSL:
final JdbcOutboundGateway inboundAdapter = createInboundAdapter();;
final JdbcOutboundGateway deleteAdapter = createDeleteAdapter();
final JdbcOutboundGateway insertUpdateAdapter = createInsertUpdateAdapter();
return IntegrationFlows
.from(setupAdapter,
c -> c.poller(Pollers.fixedRate(1000L, TimeUnit.MILLISECONDS).maxMessagesPerPoll(1)))
.enrichHeaders(h -> h.headerExpression("start", "payload[0].get(\"start\")")
.headerExpression("end", "payload[0].get(\"end\")"))
.handle(inboundAdapter)
.split(insertDeleteSplitter)
.enrichHeaders(h -> h.headerExpression("operation", "payload[0].get(\"operation\")"))
.channel(c -> c.executor("stepTaskExecutor"))
.routeToRecipients (r -> r
.recipientFlow("'I' == headers.operation or 'U' == headers.operation",
f -> f.handle(insertUpdateAdapter))
// This element is complaining "Syntax error on token ")", ElidedSemicolonAndRightBrace expected"
// Attempted to follow patterns from https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference#routers
.recipientFlow("'D' == headers.operation",
f -> f.handle(deleteAdapter))
.defaultOutputToParentFlow())
)
.aggregate()
.handle(cleanupAdapter)
.get();
Assumptions I have made, based on prior work include:
The necessary channels are auto-created as Direct Channels
Route To Recipients is the appropriate tool for this function (I have also considered expression router, but the examples of how to add sub-flows were less clear than the Route To Recipients)
Insert an ExecutorChannel somewhere between the splitter and router if you want to run the splits in parallel. You can limit the pool size of the executor to control the concurrency.
There is an extra parenthesis after .defaultOutputToParentFlow())
The corrected code is:
return IntegrationFlows
.from(setupAdapter,
c -> c.poller(Pollers.fixedRate(1000L, TimeUnit.MILLISECONDS).maxMessagesPerPoll(1)))
.enrichHeaders(h -> h.headerExpression("ALC_startTime", "payload[0].get(\"ALC_startTime\")")
.headerExpression("ALC_endTime", "payload[0].get(\"ALC_endTime\")"))
.handle(inboundAdapter)
.split(insertDeleteSplitter)
.enrichHeaders(h -> h.headerExpression("ALC_operation", "payload[0].get(\"ALC_operation\")"))
.channel(c -> c.executor(stepTaskExecutor))
.routeToRecipients (r -> r
.recipientFlow("'I' == headers.ALC_operation or 'U' == headers.ALC_operation",
f -> f.handle(insertUpdateAdapter))
// This element is complaining "Syntax error on token ")", ElidedSemicolonAndRightBrace expected"
// Attempted to follow patterns from https://github.com/spring-projects/spring-integration-java-dsl/wiki/Spring-Integration-Java-DSL-Reference#routers
.recipientFlow("'D' == headers.ALC_operation",
f -> f.handle(deleteAdapter))
.defaultOutputToParentFlow())
.aggregate()
.handle(cleanupAdapter)
.get();

SubscribeOn main thread

How can I set the async operator of Observable to run in the main thread instead in another thread. Or at least set to get the result in the main thread once we finish.
#Test
public void retryWhen() {
Scheduler scheduler = Schedulers.newThread();
Single.just("single")
.map(word -> null)
.map(Object::toString)
.retryWhen(ot ->
ot.doOnNext(t -> System.out.println("Retry mechanism:" + t))
.filter(t -> t instanceof NullPointerException && cont < 5)
.flatMap(t -> Observable.timer(100, TimeUnit.MILLISECONDS,scheduler))
.doOnNext(t -> cont++)
.switchIfEmpty(Observable.error(new NullPointerException())))
.subscribeOn(scheduler)
.subscribe(System.out::println, System.out::println);
// new TestSubscriber()
// .awaitTerminalEvent(1000, TimeUnit.MILLISECONDS);
}
I´m trying observerOn and subscribeOn but both are used to set in which thread you want the execution. But in my case I want the execution or the end of it in the same thread where I run the test
Right now the only way to see the prints are just blocking and waiting for the execution.
Regards.
You could use Observable.toBlocking() to get a BlockingObservable and use that to extract your results in your tests.
If you don't specify observeOn/subscribeOn and no operator or Observable changes the thread, then when you subscribe to the observable it will do all the processing during the subscription.

How to continue running a mocha `it` that has already failed?

The following mocha it statement:
it('should do truthy and falsey things', function() {
var val = true;
assert.equal(val, true, "true should be true!");
console.log('state 1:', val);
val != val;
assert.equal(true, false, "should continue running even after failing");
console.log('state 2:', val);
val != val;
assert.equal(false, false, "false should be false!");
});
results in the following log:
state 1: true
1) should do truthy and falsey things
And that's it. Once the second assert fails, the rest of the function doesn't run.
Is it possible to have the rest of the function (in this case, the last three lines), and if so, how?
The most direct way to prevent an assertion failure from ending the test right there would be to catch the exception thrown by the failure and rethrow it later. However, I do not recommend doing this because this creates complications. If more than one test fails, which exception are you going to rethrow? Also, you have to remember to rethrow the exception you caught, otherwise the test will seem to be passing.
Generally I just live with the fact that if I have a test with multiple assertions then the the test will stop on the first failure. In cases where I cannot live with it, I've generally adopted a strategy of recording checks into a structure and then comparing the structure to an expected value at the very end of the test. There are many ways a structure can be constructed. Here's an example:
it('should allow multiple tests', function () {
var expected = {
"something should be true": true,
"something should be false": false,
"forgot this one": 1
};
var actual = {};
var thing_to_test = true;
// First test
actual["something should be true"] = thing_to_test;
// Second test
actual["something should be false"] = thing_to_test;
// We forget the third test.
assert.equal(actual, expected);
});
When I run the test above, I get the following output:
1) should allow multiple tests
0 passing (12ms)
1 failing
1) should allow multiple tests:
AssertionError: { 'something should be true': true,
'something should be false': true } == { 'something should be true': true,
'something should be false': false,
'forgot this one': 1 }
+ expected - actual
{
- "something should be false": true
+ "forgot this one": 1
+ "something should be false": false
"something should be true": true
}
at Context.<anonymous> (test.js:22:12)

How to properly initialize Elasticsearch for queries?

I wrote a program that inserts an element, then it searches all elements in the store. Thus, it finds one more element every time the program runs. I’d expect to be able to comment out the insert and still run the program, just finding the stuff that was already inserted. Whenever I do though, I get an exception “Failed to execute phase [query_fetch], all shards failed”. Any ideas?
Hypothesis: inserting the element does some sort of implicit initialization on my node. However, I’m looking through the ES source, and I can’t figure out what that would be.
try (Node node = NodeBuilder.nodeBuilder().clusterName("tesssst").build().start()) {
try (Client client = node.client()) {
//insert an entry; if this part is removed, the program crashes
client.prepareIndex("movies", "movie", UUID.randomUUID().toString()).setSource(
"{\"title\": \"Lawrence of Arabia\",\"director\": \"David Lean\",\"year\": 1962,\"genres\":"
+ " [\"Adventure\", \"Biography\", \"Drama\"]}").execute().actionGet();
//search all entries
System.out.println("***************");
SearchResponse response = client.prepareSearch("movies")
.setTypes("movie")
.setSearchType(SearchType.QUERY_AND_FETCH)
.setFrom(0).setSize(60).setExplain(true)
.execute()
.actionGet();
SearchHit[] results = response.getHits().getHits();
System.out.println("Current results: " + results.length);
for (SearchHit hit : results) {
System.out.println("------------------------------");
Map<String, Object> result = hit.getSource();
System.out.println(result);
}
System.out.println("***************");
client.close();
}
node.close();
}
The problem was that Elasticsearch didn't have enough time to startup, but the initial insert gave it enough time. Simply adding an appropriate wait fixes it:
final ClusterHealthRequest clusterHealthRequest = new ClusterHealthRequest("movies")
.timeout(TimeValue.timeValueSeconds(60)).waitForGreenStatus();
final ClusterHealthResponse clusterHealth = client.admin().cluster()
.health(clusterHealthRequest).actionGet();
if (clusterHealth.isTimedOut()) {
System.out.println("ElasticSearch cluster health timed out");
} else {
System.out.println("ElasticSearch cluster health: Status "
+ clusterHealth.getStatus().name() + "; " + clusterHealth.getNumberOfNodes()
+ " nodes; " + clusterHealth.getActiveShards() + " active shards.");
}
(if your standards are lower, you can save time with waitForYellowStatus)

Resources