Similar to Spring Reactor: How to throw an exception when publisher emit a value?
I have a finder method in my DAO java findSomePojo which returns result SomePojo . The finder calls amazon db apis and the javasoftware.amazon.awssdk.services.dynamodb.model.GetItemResponse has output of call.
So I am trying this hasElement() check in my service layer createSomePojo method. (Not sure if I am using it correctly- Iwas trying and debugging)
Basically :
I want to check if there is already element, it is illegal to save and I would not call DAOs save. So I need to throw exception.
Assuming that there is already a record of SomePojo in DB, I try to invoke create_SomePjo of service .But I see in logs that filter is not working and is get NPE when reactor invokes createModel_SomePojo making me believe that somehow even after check filter it throws NPE
///service SomePjoService it has create_SomePojo, find_SomePojo etc
Mono<Void> create_SomePojo(reqPojo){
// Before calling DAO 's save I call serivice find (which basically calls DAOs find (Shown befow after this methid)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
System.out.println("monoPresent="+monoPresent.toString());
if(monoPresent.toString().equals("MonoHasElement")){
//*************it comes here i see that***********//
System.out.println("hrereee monoPresent="+monoPresent);
// Mono<Error> monoCheck=
return monoPresent.handle((next, sink) -> sink.error(new SomeException(ITEM_ALREADY_EXISTS))).then();
} else {
return SomePojoRepo.save(reqPojo).then();
}
}
Mono<SomePojo> find_SomePojo(id){
return SomePojoRepo.find(id);
}
==============================================================
///DAO : SomePojoRepo.java : it has save,find,delete
Mono<SomePojo> find( String id) {
Mono<SomePojo> fallback = Mono.empty();
Mono<GetItemResponse> monoFilteredResponse = monoFuture
.filter(getItemResponse -> getItemResponse.item().size() > 0&& getItemResponse!=null);
Mono<SomePojo> result = monoFilteredResponse
.map(getItemResponse -> createModel_SomePojo(getItemResponse.item()));
Mono<SomePojo> deferedResult = Mono.defer(() -> result.switchIfEmpty(fallback));
return deferedResult;
}
I see there is hasElement() method on Mono . Not sure how to correctly use it.
I can achieve exception if I call DAO save in my service create_SomePojo(reqPojo) directly without doing all this findner check because primary key constraint will take care and throw excpetion and I cna rethrow and then catch in service but what If I want to check in service and throw exception with error codes . The idea is not to pass response error object to dao layer .
Try to use Hooks.onOperatorDebug() hook to get better debugging experience.
Correct way to use hasElement (assuming that find_SomePojo never returns null)
Mono<Boolean> monoPresent = find_SomePojo(accountId, contentIdExtn)
.filter(i -> i.getId() != null)
.hasElement();
return monoPresent.flatMap(isPresent -> {
if(isPresent){
Mono.error(new SomeException(ITEM_ALREADY_EXISTS)));
}else{
SomePojoRepo.save(reqPojo);
}
}).then();
Sidenote
There is a common misconception about what Mono actually is. It does not hold any data - it's just a fragment of pipeline, which transmits signals and data flowing through it. Therefore, line System.out.println("monoPresent="+monoPresent.toString()); makes no sense, because it just prints the hasElements() decorator around the existsing pipeline. Internal name of this decorator is MonoHasElement, no matter what is contained in it(true /false), MonoHasElement would be printed anyway.
Correct ways to print signal (and data transmitted along with them) are:
Mono.log(), Mono.doOnEach/next(System.out::println) or System.out.println("monoPresent="+monoPresent.block());. Beware of third one: it will block whole thread until data is emitted, so use it only if you know what you are doing.
Example with Monos printing to play with:
Mono<String> abc = Mono.just("abc").delayElement(Duration.ofSeconds(99999999));
System.out.println(abc); //this will print MonoDelayElement instantly
System.out.println(abc.block()); //this will print 'abc', if you are patient enough ;^)
abc.subscribe(System.out::println); //this will also print 'abc' after 99999999 seconds, but without blocking current thread
Related
I have a service that handles the insertion of a new record into a MongoDB collection:
public Mono<ProductDto> insertProduct(Mono<ProductDto> in) {
//TODO Must handle Duplicate key inserts --> Throw a ProductAlreadyExistsException
Mono<ProductDto> productDtoMono ;
try{
productDtoMono= in.map(ProductDto::toEntity)
.flatMap(productRepository::insert)
.map(ProductDto::new)
;
}
catch (DuplicateKeyException ex) {
throw new ProductAlreadyExistsException();
}
return productDtoMono;
}
When the ID given is already in use, the application throws a org.springframework.dao.DuplicateKeyException.
I am aware the above code with the try/catch block is incorrect, it is mostly there to demonstrate what I want to do. I am very new to Webflux, and reactive programming... I'd like to find out the correct way to handle this, but I have not been able to find much in the way of decent sample code for exception handling in the service layers for this, it is almost always in the router or request handler layer.
Hoping someone might be able to guide me on this.
The exception would be caught, and the application would throw the new, custom ProductAlreadyExistsException created for this purpose.
I have also tried to do this within the flatMap insert, but at this point I am kind of throwing poop at the wall to see if I can stumble into how it should be done:
public Mono<ProductDto> insertProduct(Mono<ProductDto> in) {
//TODO Must handle Duplicate key inserts --> Throw a ProductAlreadyExistsException
Mono<ProductDto> productDtoMono ;
productDtoMono= in.map(ProductDto::toEntity)
.flatMap(p -> {
try{
return productRepository.insert(p);
}
catch (DuplicateKeyException ex) {
return Mono.error(new ProductAlreadyExistsException());
}
})
.map(ProductDto::new)
;
return productDtoMono;
}
Since DuplicateKeyException is an unchecked exception and not a checked exception (which are quite annoying to use in Reactive code), you can use the onErrorMap()-method here:
public Mono<ProductDto> insertProduct(Mono<ProductDto> in) {
return in.map(ProductDto::toEntity)
.flatMap(productRepository::insert)
.onErrorMap(DuplicateKeyException.class, e -> new ProductAlreadyExistsException())
.map(ProductDto::new);
}
The intermediate productDtoMono variable here is redundant.
If however you need to work with checked exceptions, your last snippet of code is typically how you would do it.
Your first snippet of code does not do what you think it does, the catch-block will never run because Project Reactor catches it before your code does and transforms it into an error signal for downstream operators.
I am using web reactive in spring web flux. I have implemented a Handler function for POST request. I want the server to return immediately. So, I have implemeted the handler as below -:
public class Sample implements HandlerFunction<ServerResponse>{
public Mono<ServerResponse> handle(ServerRequest request) {
Mono bodyMono = request.bodyToMono(String.class);
bodyMono.map(str -> {
System.out.println("body got is " + str);
return str;
}).subscribe();
return ServerResponse.status(HttpStatus.CREATED).build();
}
}
But the print statement inside the map function is not getting called. It means the body is not getting extracted.
If I do not return the response immediately and use
return bodyMono.then(ServerResponse.status(HttpStatus.CREATED).build())
then the map function is getting called.
So, how can I do processing on my request body in the background?
Please help.
EDIT
I tried using flux.share() like below -:
Flux<String> bodyFlux = request.bodyToMono(String.class).flux().share();
Flux<String> processFlux = bodyFlux.map(str -> {
System.out.println("body got is");
try{
Thread.sleep(1000);
}catch (Exception ex){
}
return str;
});
processFlux.subscribeOn(Schedulers.elastic()).subscribe();
return bodyFlux.then(ServerResponse.status(HttpStatus.CREATED).build());
In the above code, sometimes the map function is getting called and sometimes not.
As you've found, you can't just arbitrarily subscribe() to the Mono returned by bodyToMono(), since in that case the body simply doesn't get passed into the Mono for processing. (You can verify this by putting a single() call in that Mono, it'll throw an exception since no element will be emitted.)
So, how can I do processing on my request body in the background?
If you really still want to just use reactor to do a long task in the background while returning immediately, you can do something like:
return request.bodyToMono(String.class).doOnNext(str -> {
Mono.just(str).publishOn(Schedulers.elastic()).subscribe(s -> {
System.out.println("proc start!");
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("proc end!");
});
}).then(ServerResponse.status(HttpStatus.CREATED).build());
This approach immediately publishes the emitted element to a new Mono, set to publish on an elastic scheduler, that is then subscribed in the background. However, it's kind of ugly, and it's not really what reactor is designed to do. You may be misunderstanding the idea behind reactor / reactive programming here:
It's not written with the idea of "returning a quick result and then doing stuff in the background" - that's generally the purpose of a work queue, often implemented with something like RabbitMQ or Kafka. It's "raison d'être" is instead to be non-blocking, so a single thread is never idly blocked, waiting for something else to complete.
The map() method isn't designed for side effects, it's designed to transform each object into another. For side effects, you want doOnNext() instead;
Reactor uses a single thread by default, so your "additional processing" in your map() method would still block that thread.
If your application is for anything more than quick demo purposes, and/or you need to make heavy use of this pattern, then I'd seriously consider setting up a proper work queue instead.
This is not possible.
Web servers (including Reactor Netty, Tomcat, etc) clean up and recycle resources when request processing is done. This means that when your controller handler is done, the HTTP resources, the request itself, reusable buffers, etc are recycled or closed. At that point, you cannot read from the request body anymore.
In your case, you need to read and buffer the whole request body first, then return a response and kick off a task for processing that request in a separate execution.
I have the following method in an #Service class which has #Transactional defined:
#Override
public Result add(#NonNull final UserSaveRequest request) {
final Result<Email> emailResult = Email.create(request.getEmail());
final Result<UserFirstName> userFirstNameResult = UserFirstName.create(request.getFirstName());
final Result<UserLastName> userLastNameResult = UserLastName.create(request.getLastName());
final Result combinedResult = Result.combine(emailResult, userFirstNameResult, userLastNameResult);
if (combinedResult.isFailure()) {
return Result.fail(combinedResult.getErrorMessage());
}
final Result<User> userResult = User.create(emailResult.getValue(), userFirstNameResult.getValue(), userLastNameResult.getValue());
if (userResult.isFailure()) {
return Result.fail(userResult.getErrorMessage());
}
this.userRepository.save(userResult.getValue());
return Result.ok();
}
Now as you can see I utilize a Result class which can contain a return value or an error message as I don't think using exceptions for flow control is very clean.
The problem I now have is; the complete method is bound in one transaction and if one database call should fail the whole transaction will be rolled back. In my model however, after the this.userRepository.save(userResult.getValue()); call, if something would happen that would force me to return a failed result, I can't undo that save(userResult.getVlaue()); call seeing as I don't use exceptions for flow control.
Is this a problem that has an elegant solution, or is this a place where I need to make a trade-off between using exceptions as flow control and having to mentally keep track of the ordering of my statements in these kind of situations?
Yes, you can trigger rollback manually. Try this:
TransactionAspectSupport.currentTransactionStatus().setRollbackOnly();
More information: https://docs.spring.io/spring/docs/5.0.7.RELEASE/spring-framework-reference/data-access.html#transaction-declarative-rolling-back
I am using Spring projectreactor reactor-core 3.1.8.RELEASE. I am implementing a logging framework for my microservice to have JSON Audit logs, so used context to store certain fields such as userID, collaboration ID, component Name and few other fields that are common across request life-cycle. Since Threadlocal cannot be used in reactive services to stores these elements, I have to use the context. But getting a reference to the context is apparently very difficult. I can get a reference to the context from the Signal through the doOnEach function call and that's it. If I use doOnEach, it gets called for all signals types and I am unable to isolate on Error, success and so on. Moreover, if an error occurs in between, then all the subsequent doOnEach gets called anyway, so the logs get repeated with several onError log types.
There is very limited documentation regarding how to get a reference to the context object in Spring reactor. Any help regarding a better way to generate audit logs that has collaboration IDs and other request specific IDs stored and propagated across function calls and external invocations is appreciated.
Code Snippets -
In the WebFilter, I am setting few key-value pairs as follows -
override fun filter(exchange: ServerWebExchange, filterChain: WebFilterChain): Mono<Void> {
// add the context variables at the end of the chain as the context moves from
// downstream to upstream.
return filterChain.filter(exchange)
.subscriberContext { context ->
var ctx = context.put(RestRequestInfo::class.java, restRequestInfo(exchange))
ctx = ctx.put(COLLABORATION_ID, UUID.randomUUID().toString())
ctx=ctx.put(COMPONENT_NAME, "sample-component-name")
ctx=ctx.put(USER_NAME, "POSTMAN")
ctx
}
}
Then I want to use the key-value pairs added above in all the subsequent logs so that log aggregators like Splunk can get all the JSON logs associated with this particular request, based on collaboration ID. Right now, the only way to get values out of context is through doOnEach function call, where we get a handle to the SIgnal through which we get handle to context. But all doOnEach gets called during each and every events, irrespective of whether each function call was success or failure
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::validateRequest)
.doOnEach(**Code to log with context data**)
.map(RequestValidations::buildRequest)
.map(RequestValidations::validateQueryParameters)
.doOnEach(**Code to log with context data**)
.flatMap(coverageSummariesGateway::getCoverageSummaries)
.doOnEach(**Code to log with context data**)
.map({ coverageSummaries ->
getCoverageSummariesResponse(coverageSummaries, serviceReferenceId) })
.doOnEach(**Code to log with context data**)
.flatMap(this::renderSuccess)
.doOnEach(**Code to log with context data**)
.doOnError { logger.info("ERROR OCCURRED") }
Thank you!
You could do something like following:
return Mono.just(request)
.doOnEach(**Code to log with context data**)
.flatMap( r -> withMDC(r, RequestValidations::validateRequest))
following method will populate mapping diagnostic context (MDC) so you have it automatically in your logs (depends on you logging pattern). E.g. logback has %X{traceId} where traceId is a key in the tracingContext map.
public static <T, R> Mono<R> withMDC(T value, Function<T, Mono<R>> f) {
return Mono.subscriberContext()
.flatMap( ctx -> {
Optional<Map> tracingContext = ctx.getOrEmpty("tracing-context-key");
if (tracingContext.isPresent()) {
try {
MDC.setContextMap(tracingContext.get());
return f.apply(value);
} finally {
MDC.clear();
}
} else{
return f.apply(value);
}
});
}
It is not quite nice, hope it will be eventually improved by Logging frameworks and context will be injected automatically.
In a microservice environment I see two main benefits from tracing requests through all microservice instances over an entire business process.
Finding latency gaps between or in service instances
Finding roots of failures, whether technical or regarding the business case
With Zipkin there is a tool, which addresses the first issue. But how can tracing be used to unveil failures in your microservice landscape? I definitely want to trace all error afflicted spans, but not each request, where nothing went wrong.
As mentioned here a custom Sampler could be used.
Alternatively, you may register your own Sampler bean definition and programmatically make the decision which requests should be sampled. You can make more intelligent choices about which things to trace, for example, by ignoring successful requests, perhaps checking whether some component is in an error state, or really anything else.
So I tried to implement that, but it doesn't work or I used it wrong.
So, as the blog post suggested I registered my own Sampler:
#Bean
Sampler customSampler() {
return new Sampler() {
#Override
public boolean isSampled(Span span) {
boolean isErrorSpan = false;
for(String tagKey : span.tags().keySet()){
if(tagKey.startsWith("error_")){
isErrorSpan = true;
}
}
return isErrorSpan ;
}
};
}
And in my controller I create a new Span, which is being tagged as an error if an exception raises
private final Tracer tracer;
#Autowired
public DemoController(Tracer tracer) {
this.tracer = tracer;
}
#RequestMapping(value = "/calc/{i}")
public String calc(#PathVariable String i){
Span span = null;
try {
span = this.tracer.createSpan("my_business_logic");
return "1 / " + i + " = " + new Float(1.0 / Integer.parseInt(i)).toString();
}catch(Exception ex){
log.error(ex.getMessage(), ex);
span.logEvent("ERROR: " + ex.getMessage());
this.tracer.addTag("error_" + ex.hashCode(), ex.getMessage());
throw ex;
}
finally{
this.tracer.close(span);
}
}
Now, this doesn't work. If I request /calc/a the method Sampler.isSampled(Span) is being called before the Controller method throws a NumberFormatException. This means, when isSampled() checks the Span, it has no tags yet. And the Sampler method is not being called again later in the process. Only if I open the Sampler and allow every span to be sampled, I see my tagged error-span later on in Zipkin. In this case Sampler.isSampled(Span) was called only 1 time but HttpZipkinSpanReporter.report(Span) was executed 3 times.
So what would the use case look like, to transmit only traces, which have error spans ? Is this even a correct way to tag a span with an arbitrary "error_" tag ?
The sampling decision is taken for a trace. That means that when the first request comes in and the span is created you have to take a decision. You don't have any tags / baggage at that point so you must not depend on the contents of tags to take this decision. That's a wrong approach.
You are taking a very custom approach. If you want to go that way (which is not recommended) you can create a custom implementation of a SpanReporter - https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/SpanReporter.java#L30 . SpanReporter is the one that is sending spans to zipkin. You can create an implementation that will wrap an existing SpanReporter implementation and will delegate the execution to it only when some values of tags match. But from my perspective it doesn't sound right.