I'm completely new to reactor programming and I'm really struggling with migrating old integration tests since upgrading to the latest Spring Boot / State Machine.
Most Integration tests have the same basic steps :
Call a method that returns a Mono and starts a state Machine and returns an object containing a generated unique id as well as some other infos related to the initial request.
With the returned object call a method that verifies if a value has been updated in the database (using the information of the object retried in step 1)
Poll at a fixed interval the method that checks in the database if value has changed until either the value has changed or a predefined timeout occurs.
Check another table in the database if another object has been updated
Below an example:
#Test
void testEndToEnd() {
var instance = ServiceInstance.buildDefault();
var updateRequest = UpdateRequest.build(instance);
// retrieve an update Response related to the request
// since a unique id is generated when triggering the update request
// before starting a stateMachine that goes through different steps
var updateResponse = service.updateInstance(updateRequest).block();
await().alias("Check if operation was successful")
.atMost(Duration.ofSeconds(120))
.pollInterval(Duration.ofSeconds(2))
.until(() -> expectOperationState(updateResponse, OperationState.SUCCESS))
// check if values are updated in secondary table
assertValuesInTransaction(updateResponse);
}
This was working fine before but ever since the latest update where it fails with the exception :
java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-6
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:83)
at reactor.core.publisher.Mono.block(Mono.java:1710)
I saw that a good practice to test reactor methods using StepVerifier but I do not see how I can reproduce the part done with Awaitability to poll to see if the value has changed in the DB since the method that checks in the DB returns a Mono and not a flux that keeps sending values.
Any idea on how to accomplish this or to make the spring stack accept blocking operations?
Thanks
My current stack :
Spring Boot 3.0.1
Spring State Machine 3.0.1
Spring 6
Junit 5.9.2
So as discussed in comments here is an example with comments. I used flatMap to subscribe to what expectOperationState returns. Also there is Mono.fromCallable used which check the value from some method and if it fails to emit anything in 3 seconds - the timeout exception is thrown. Also we could try to get rid of this boolean value from expectOperationState and refactor the code to just return Mono<Void> with completed signal but this basically shows how you can achieve what you want.
class TestStateMachine {
#Test
void testUntilSomeOperationCompletes() {
final Service service = new Service();
final UpdateRequest updateRequest = new UpdateRequest();
StepVerifier.create(service.updateInstance(updateRequest)
.flatMap(updateResponse -> expectOperationState(updateResponse, OperationState.SUCCESS))
)
.consumeNextWith(Assertions::assertTrue)
.verifyComplete();
}
private Mono<Boolean> expectOperationState(final UpdateResponse updateResponse, final OperationState success) {
return Mono.fromCallable(() -> {
while (true) {
boolean isInDb = checkValueFromDb(updateResponse);
if (isInDb) {
return true;
}
}
})
.publishOn(Schedulers.single())
//timeout if we not receive any value from callable within 3 seconds so that we do not check forever
.timeout(Duration.ofSeconds(3));
}
private boolean checkValueFromDb(final UpdateResponse updateResponse) {
return true;
}
}
class Service {
Mono<UpdateResponse> updateInstance(final UpdateRequest updateRequest) {
return Mono.just(new UpdateResponse());
}
}
Here is an example without using Mono<Boolean> :
class TestStateMachine {
#Test
void test() {
final Service service = new Service();
final UpdateRequest updateRequest = new UpdateRequest();
StepVerifier.create(service.updateInstance(updateRequest)
.flatMap(updateResponse -> expectOperationState(updateResponse, OperationState.SUCCESS).timeout(Duration.ofSeconds(3)))
)
.verifyComplete();
}
private Mono<Void> expectOperationState(final UpdateResponse updateResponse, final OperationState success) {
return Mono.fromCallable(() -> {
while (true) {
boolean isInDb = checkValueFromDb(updateResponse);
if (isInDb) {
//return completed Mono
return Mono.<Void>empty();
}
}
})
.publishOn(Schedulers.single())
//timeout if we not receive any value from callable within 3 seconds so that we do not check forever
.timeout(Duration.ofSeconds(3))
.flatMap(objectMono -> objectMono);
}
private boolean checkValueFromDb(final UpdateResponse updateResponse) {
return true;
}
}
I have a Spring boot service with some code like below for parallel async call:
CompletableFuture future1 = accountManager.getResult(url1);
CompletableFuture future2 = accountManager.getResult(url2);
CompletableFuture.allOf(future1, future2).join();
String result1 = future1.get();
String result2 = future2.get();
It works fine when there is no exception. My question is how to handle exception? If getting future1 failed (let say url2 is an invalid url), I still want future2 back as partial result of allOf method. How should I do it?
Thanks!
CompletableFuture comes with a block called exceptionally() which can be used handle the exceptions happen inside the asynchronous code block. Snippet of getResult method for your reference,
public CompletableFuture<String> getGreeting(String url) {
return CompletableFuture.supplyAsync( () -> {
return // Business logic..
}, executor).exceptionally( ex -> {
log.error("Something went wrong : ", ex);
return null;
});
}
In this case the block would return null in case of exception and allOf method would lead to a completion where you can filter the one resulted in the exception when you fetch individual futures.
I have below code retuning Mono<Foo>:
try {
return userRepository.findById(id) // step 1
.flatMap(user -> barRepository.findByUserId( user.getId()) // step 2
.map(bar-> Foo.builder().msg("Already exists").build()) // step 3
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build()) // step 4
.map(bar-> Foo.builder().msg("Created").build()) // step 5
))
.doOnError(throwable -> Mono.just(handleError(throwable)));
} catch(Exception e) {
log.error("from catch block");
return Mono.just(handleError(e));
}
If error occurs in step 1 (e.g. user does not exist by the specified id), will it be caught by doOnError or by try catch block or none of these two?
Same question if error happens in step 2, step3, step 4.
What is the correct code so that error is always caught by doOnError and eliminate try catch?
I am using
public interface UserRepository extends ReactiveMongoRepository<User, String> same for barRepository.
handleError(throwable) simply does log.error(e.getMessage() and retuns Foo.
I think the first error is in the title: "Mono or Flux" is not related with the error handling.
Mono can only emit one item at the most (streams one element)
Flux can emit more complex stuff (i.e. List)
To handle errors you can follow this example:
return webClient.get()
.uri(url)
.retrieve()
.bodyToMono(ModelYouAreRetrieving.class)
.doOnError(throwable -> logger.error("Failed for some reason", throwable))
.onErrorReturn(new ModelYouAreRetrieving(...))
.block();
DoOnError will only perform side effects and assuming the findById are will return a Mono.Error() if it fails something like this should work.
return userRepository.findById(id)
.flatMap ( user ->
barRepository.findByUserId(user.getId())
.map((user,bar)-> Foo.builder().msg("Already exists").build())
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build())
.map(bar-> Foo.builder().msg("Created").build())
))
.onErrorReturn(throwable -> Mono.just(handleError(throwable)));
The try catch will only work if you either call a blocking operation of the chain, or a runtime error occurs before you enter the reactive chain. the doOn operations do not modify the chain, they are used for side effects only. Since flatMap expects a producer, you will need to return a Mono from the call, and in this case if an error occurs, then it will just propagate the error. In all reactive chains the error will propagate unless otherwise handled.
Use Exceptions.propagate(e) which wraps a checked exception into a special runtime exception that can be handled by onError
Below Code tries to covers User attributes in upper case. Now, when it encounters kyle the checked exception is throws and MIKE is returned from onErrorReturn
#Test
void Test19() {
Flux.fromIterable(Arrays.asList(new User("jhon", "10000"),
new User("kyle", "bot")))
.map(x -> {
try {
return toUpper(x);
} catch (TestException e) {
throw Exceptions.propagate(e);
}
})
.onErrorReturn(new User("MIKE", "BOT")).subscribe(x -> System.out.println(x));
}
protected final class TestException extends Exception {
private static final long serialVersionUID = -831485594512095557L;
}
private User toUpper(User user) throws TestException{
if (user.getName().equals("kyle")) {
throw new TestException();
}
return new User(user.getName().toUpperCase(), user.getProfession().toUpperCase());
}
Output
User [name=JHON, profession=10000]
User [name=MIKE, profession=BOT]
#Gianluca Pinto's last line of code is also incorrect. The code won't be compiled. onErrorReturn is not suitable for complicated error handling. What you should use is onErrorResume.
see: https://grokonez.com/reactive-programming/reactor/reactor-handle-error#21_By_falling_back_to_another_Flux
onErrorResume will fall back to another Flux and let you catch and manage the exception thrown by previous Flux. if look into the implementation of onErrorReturn, you will find onErrorReturn is actually using onErrorResume.
So here the code should be:
.onErrorResume(throwable -> Mono.just(handleError(throwable)));
The last line of the code of #James Ralston is wrong. The correct code should be:
return userRepository.findById(id)
.flatMap ( user ->
barRepository.findByUserId(user.getId())
.map((user,bar)-> Foo.builder().msg("Already exists").build())
.switchIfEmpty(barRepository.save(Bar.builder().userId(user.getId()).build())
.map(bar-> Foo.builder().msg("Created").build())
))
.onErrorReturn(Mono.just(handleError(throwable)));
While creating the reactive flow, we need to use onError* as it provides a fallback Mono/Flux while doOn* are side-effect operators.
NOTE: The examples are in Kotlin
Below is an example:
fun saveItems(item: Item) = testRepository.save(item)
.onErrorResume {
Mono.error(
onErrorResumeHandler(
it,
"APP-1002",
"Error occurred while saving the something :P, contact admin"
)
)
}
fun onErrorResumeHandler(exception: Throwable, errorCode: String, errorMessage: String) =
if (exception is TestRepositoryException) exception else
TestServiceException(errorCode, errorMessage)
There should be a central exception handler, we can create by extending AbstractErrorWebExceptionHandler. The order is -2 to supersede the default.
Below is an example:
#Component
#Order(-2)
class BaseControllerAdvice(
errorAttributes: ErrorAttributes,
resources: WebProperties.Resources,
applicationContext: ApplicationContext,
serverCodecConfigurer: ServerCodecConfigurer
) : AbstractErrorWebExceptionHandler(errorAttributes, resources, applicationContext) {
val log = logger()
init {
setMessageWriters(serverCodecConfigurer.writers)
}
override fun getRoutingFunction(errorAttributes: ErrorAttributes?) =
router {
RequestPredicates.all().invoke(this#BaseControllerAdvice::renderErrorResponse)
}
//RouterFunctions.route(RequestPredicates.all(),this::renderErrorResponse)
fun renderErrorResponse(
request: ServerRequest
): Mono<ServerResponse> {
val errorPropertiesMap = getErrorAttributes(
request,
ErrorAttributeOptions.defaults()
)
val ex: ApplicationException = getError(request) as ApplicationException
log.info("Error attributes:{}", request)
return ServerResponse.status(HttpStatus.BAD_REQUEST)
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromValue(ErrorResponseVO(ex.errorCode, ex.errorMessage)))
}
data class ErrorResponseVO(val errorMessage: String, val errorCode: String)
}
We created a custom item reader which extends AbstractPaginatedDataItemReader. Spring-batch allows to manage which exception stops or not a job (skipped exceptions).
In "classic" spring-batch readers, the doRead method throws any Exception. That means, if a skipped exception is thrown during the read, the item is skipped and the job continues running.
But in paginated readers, the doPageRead method, used to retrieve next data page, doesn't throw any exception:
protected abstract Iterator<T> doPageRead();
The doPageRead method is called by the doRead one:
protected T doRead() throws Exception {
synchronized (lock) {
if(results == null || !results.hasNext()) {
results = doPageRead();
page ++;
if(results == null || !results.hasNext()) {
return null;
}
}
if(results.hasNext()) {
return results.next();
}
else {
return null;
}
}
}
As doPageRead method doesn't declare any thrown exception, that means a configured skipped exception can only be a RuntimeException?
Thanks
A Spring Batch reader is eventually an ItemReader irrespective of it being a paging reader or a non - paging reader. Which eventually means that it will hand over single - single items to processor and read() method contract is all that matters.
Paging readers simply have an optimization about how they actually read an item but no different than than a regular non - paging reader.
So in my opinion, your look out for doReadPage() method seems unnecessary, what matters is read() method contract.
If you are facing any issue ( and that is not clear from your question ) , do let me know.
We have been using Elasticsearch Plugin in our project. While getting index name from alias getting below error
Error
{
"error": "AssertionError[Expected current thread[Thread[elasticsearch[Seth][http_server_worker][T#2]{New I/O worker #20},5,main]] to not be a transport thread. Reason: [Blocking operation]]", "status": 500
}
Code
String realIndex = client.admin().cluster().prepareState()
.execute().actionGet().getState().getMetaData()
.aliases().get(aliasName).iterator()
.next().key;
what causes this issue?? Googled it didn't get any help
From the look of the error, it seems like this operation is not allowed on the transport thread as it will block the thread until you get the result back. You need to execute this on a execute thread.
public String getIndexName() {
final IndexNameHolder result = new IndexNameHolder(); // holds the index Name. Needed a final instance here, hence created a holder.
getTransportClient().admin().cluster().prepareState().execute(new ActionListener<ClusterStateResponse>() {
#Override
public void onResponse(ClusterStateResponse response) {
result.indexName = response.getState().getMetaData().aliases().get("alias").iterator().next().key;
}
#Override
public void onFailure(Throwable e) {
//Handle failures
}
});
return result.value;
}
There is another method for execute(), one which takes a listener. You need to implement your own listener. In my answer, I have an anonymous implementation of Listener.
I hope it helps