When trying to use Quarkus (version 2.9.2.Final) EventBus requestAndForget with a #ConsumeEvent method that returns void, the following exception occurs in the logs, even though the processing occurs without any problem.
OK
2022-06-07 09:44:04,064 ERROR [io.qua.mut.run.MutinyInfrastructure]
(vert.x-eventloop-thread-1) Mutiny had to drop the following
exception: (TIMEOUT,-1) Timed out after waiting 30000(ms) for a reply.
address: __vertx.reply.3, repliedAddress: receivedSomeEvent
The consumer code:
#ApplicationScoped
public class ConsumerManiac{
#ConsumeEvent(value = "receivedSomeEvent")
public void consume(SomeEvent someEvent ) {
System.out.println("OK");
}
}
The Producer code (a REST Endpoint):
public class SomeResource {
private final EventBus eventBus;
#Inject
public SomeResource (EventBus eventBus) {
this.eventBus = eventBus;
}
#POST
public Response send(#Valid SomeEvent someEvent) {
eventBus.requestAndForget("receivedSomeEvent", someEvent);
return Response.accepted().build();
}
}
If the consumer method is changed to return some value, then the exception in logs does not occur.
#ApplicationScoped
public class ConsumerManiac{
#ConsumeEvent(value = "receivedSomeEvent")
public String consume(SomeEvent someEvent ) {
System.out.println("OK");
return "ok";
}
}
Is there any missing piece of code that is missing so the exception does not occur (even though processing concludes without any problem)?
Reference: https://quarkus.io/guides/reactive-event-bus#implementing-fire-and-forget-interactions
Full stacktrace:
2022-06-07 09:44:04,064 ERROR [io.qua.mut.run.MutinyInfrastructure]
(vert.x-eventloop-thread-1) Mutiny had to drop the following
exception: (TIMEOUT,-1) Timed out after waiting 30000(ms) for a reply.
address: __vertx.reply.3, repliedAddress: receivedSomeEvent at
io.vertx.core.eventbus.impl.ReplyHandler.handle(ReplyHandler.java:76)
at
io.vertx.core.eventbus.impl.ReplyHandler.handle(ReplyHandler.java:24)
at
io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:893)
at
io.vertx.core.impl.VertxImpl$InternalTimerHandler.handle(VertxImpl.java:860)
at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:50)
at
io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:168)
at io.vertx.core.impl.AbstractContext.emit(AbstractContext.java:53)
at
io.vertx.core.impl.VertxImpl$InternalTimerHandler.run(VertxImpl.java:883)
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)
at
io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170)
at
io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at
io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) at
io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
at
io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:833)
I had to return arbitrary value to avoid this exception.
Related
I am trying to implement spring-retry(version - 1.3.1) in my spring boot application. I have to retry webservice operation to read the record if not found in first request.
sample code:
#Retryable(include = {IllegalArgumentException.class}, backoff = #Backoff(500), maxAttempts = 3, recover ="readFallback")
Object read(String Id);
#Recover
Object readFallback(RuntimeException e, String Id);
void deletePayment(String paymentId);
Problem :
I am getting correct response from read method(annotated with #Retryable) in exception scenario but I am getting RetryExhaustedException with nested original exception when I am getting exception on my delete method. As you see, delete method doesn't annotated with #Retryable . Delete method is in different package.
**Sample exception response ** : "Retry exhausted after last attempt with no recovery path; nested exception is exception.NotFoundException: Not found"
Expected : Delete method should not be impacted by #Retryable. Can someone help me to find what am i missing or doing wrong. I have tried but unable to not found the solution of this problem on internet.
Thanks in advance !
Works as expected for me:
#SpringBootApplication
#EnableRetry
public class So71546747Application {
public static void main(String[] args) {
SpringApplication.run(So71546747Application.class, args);
}
#Bean
ApplicationRunner runner(SomeRetryables retrier) {
return args -> {
retrier.foo("testFoo");
try {
Thread.sleep(1000);
retrier.bar("testBar");
}
catch (Exception e) {
e.printStackTrace();
}
};
}
}
#Component
class SomeRetryables {
#Retryable
void foo(String in) {
System.out.println(in);
throw new RuntimeException(in);
}
#Recover
void recover(String in, Exception ex) {
System.out.println("recovered");
}
void bar(String in) {
System.out.println(in);
throw new RuntimeException(in);
}
}
testFoo
testFoo
testFoo
recovered
testBar
java.lang.RuntimeException: testBar
at com.example.demo.SomeRetryables.bar(So71546747Application.java:52)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:789)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.retry.annotation.AnnotationAwareRetryOperationsInterceptor.invoke(AnnotationAwareRetryOperationsInterceptor.java:166)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:753)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:698)
at com.example.demo.SomeRetryables$$EnhancerBySpringCGLIB$$e61dd199.bar(<generated>)
at com.example.demo.So71546747Application.lambda$0(So71546747Application.java:26)
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:768)
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:758)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:310)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1312)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1301)
at com.example.demo.So71546747Application.main(So71546747Application.java:17)
Please provide an MCRE that exhibits the behavior you see so we can see what's wrong.
I've been trying to figure out how route errors to my own error handler with the following, seemingly simple configuration, but Camel is swallowing the exception without routing it to any error handler I configure. I've run out of ideas. Any help would be much appreciated.
I've got a seda route that supports multiple consumers:
#Component
public class MessageGenerator {
public static final String ERROR_GENERATOR_CHANNEL = "seda:my-error-generator?multipleConsumers=true&concurrentConsumers=3";
private final FluentProducerTemplate producerTemplate;
public MessageGenerator(FluentProducerTemplate producerTemplate) {
this.producerTemplate = producerTemplate;
}
public void generateMessage() {
producerTemplate
.to(ERROR_GENERATOR_CHANNEL)
.withBody("Hello World")
.asyncSend();
}
}
I've got two separate POJO consumers:
#Configuration
public class MessageConsumer1 {
#Consume(ERROR_GENERATOR_CHANNEL)
void receiveMessage(String message) {
System.out.println("Received message 1: " + message);
throw new NullPointerException("Error generated");
}
}
#Configuration
public class MessageConsumer2 {
#Consume(ERROR_GENERATOR_CHANNEL)
void receiveMessage(String message) {
System.out.println("Received message 2: " + message);
}
}
When I run the following example, the NullPointerException gets swallowed by the underlying Camel MulticastProcessor as we can see in the logs:
Received message 2: Hello World
Received message 1: Hello World
2022-01-15 13:40:23.711 DEBUG 32945 --- [error-generator] o.a.camel.processor.MulticastProcessor : Message exchange has failed: Multicast processing failed for number 0 for exchange: Exchange[] Exception: java.lang.NullPointerException: Error generated
2022-01-15 13:40:23.711 DEBUG 32945 --- [error-generator] o.a.camel.processor.MulticastProcessor : Message exchange has failed: Multicast processing failed for number 0 for exchange: Exchange[] Exception: java.lang.NullPointerException: Error generated
The exception only gets logged as debug and never gets propagated to any error handler I set up.
Any thoughts on how I could receive the error in my own error handler rather than Camel swallowing the exception as a debug statement?
Note1: I've attempted many variations on both default error handling and default dead letter handling to no avail. I could just be doing it wrong...
Note2: that I'm using Spring[Boot] here too, hence the #Configuration annotation.
Note1: I've attempted many variations on both default error handling and default dead letter handling to no avail. I could just be doing it wrong...
Haven't used #Consume annotations but generally if you want Camel route not to handle any errors you can use .errorHandler(noErrorHandler()). This can be used to pass the error back to parent route or all the way to the code calling ProducerTemplate.sendBody.
Example:
public class ExampleTest extends CamelTestSupport {
#Test
public void noErrorHandlerTest() {
try {
template.sendBody("direct:noErrorHandler", null);
fail();
} catch (Exception e) {
System.out.println("Caught Exception: " + e.getMessage());
}
}
#Override
protected RoutesBuilder createRouteBuilder() throws Exception {
return new RouteBuilder(){
#Override
public void configure() throws Exception {
from("direct:noErrorHandler")
.errorHandler(noErrorHandler())
.log("Throwing error")
.throwException(Exception.class, "Test Exception");
}
};
}
}
I have service A that calls downstream service B.
Service A code
#RestController
#RequestMapping(value = "", produces = MediaType.APPLICATION_JSON_VALUE)
public class GreetingController {
private final GreetingService greetingService;
public GreetingController(GreetingService greetingService){
this.greetingService = greetingService;
}
#GetMapping(value = "/greetings")
public Mono<String> getGreetings() {
return greetingService.callServiceB();
}
}
#Component
#RequiredArgsConstructor
public class GreetingService {
CircuitBreaker circuitBreaker = CircuitBreaker.ofDefaults("greetingService");
Callable<Mono<String>> callable = CircuitBreaker.decorateCallable(circuitBreaker, this::clientCall);
Future<Mono<String>> future = Executors.newSingleThreadExecutor().submit(callable);
public Mono<String> callServiceB() {
try {
return future.get();
} catch (CircuitBreakerOpenException | InterruptedException | ExecutionException ex){
return Mono.just("Service is down!");
}
}
private final String url = "/v1/holidaysgreetings";
private Mono<String> clientCall(){
WebClient client = WebClient.builder().baseUrl("http://localhost:8080").build();
return client
.get()
.uri(url)
.retrieve()
.bodyToMono(String.class);
}
when i shut down downstream service B(running on localhost:8080) and hit /greetings endpoint in GreetingsController class to see if my circuit breaker is working properly or not, i get very this nasty error
2021-06-28 21:27:31.431 ERROR 10285 --- [nio-8081-exec-7] o.a.c.c.C.[.[.[.[dispatcherServlet]: Servlet.service() for servlet [dispatcherServlet] in context with path [/v1/holidaysgreetings]
threw exception [Request processing failed; nested exception is org.springframework.web.reactive.function.client.WebClientRequestException: Connection refused: localhost/127.0.0.1:8080;
nested exception is io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: localhost/127.0.0.1:8080] with root cause
java.net.ConnectException: Connection refused
Anyone knows why i am getting this? What i am missing here? Am i implementing circuit breaker correctly?
You are mixing reactive libraries with regular non-reactive libraries. If you aim to use spring-webflux it is better to use the reactor-resilience4j together with the regular reactor-adapter library.
Use these imports:
implementation 'org.springframework.boot:spring-boot-starter-webflux'
implementation 'org.springframework.cloud:spring-cloud-starter-circuitbreaker-reactor-resilience4j'
implementation "io.projectreactor.addons:reactor-adapter:${reactorVersion}"
You are also not creating the circuit-breaker service that you can rely on. After creating it you can call the " Mono run(Mono toRun, Function<Throwable, Mono> fallback)" (to the one that return a Flux if you want) to execute your service and provide a fallback.
Here is one example from a demo code.
#RestController
public class CompletableFutureDemoController {
Logger LOG = LoggerFactory.getLogger(CompletableFutureDemoController.class);
private CompletableFutureHttpBinService httpBin;
private ReactiveCircuitBreakerFactory reactiveCircuitBreakerFactory;
public CompletableFutureDemoController(CompletableFutureHttpBinService httpBin, ReactiveCircuitBreakerFactory reactiveCircuitBreakerFactory) {
this.httpBin = httpBin;
this.reactiveCircuitBreakerFactory = reactiveCircuitBreakerFactory;
}
#GetMapping("/completablefuture/delay/{seconds}")
public Mono<Map> delay(#PathVariable int seconds) {
return reactiveCircuitBreakerFactory.create("completablefuturedelay")
.run(Mono.fromFuture(httpBin.delay(seconds)), t -> {
LOG.warn("delay call failed error", t);
Map<String, String> fallback = new HashMap();
fallback.put("hello", "world");
return Mono.just(fallback);
}
);
}
}
I implemented feign client and hystrix to my spring boot microservice application.
I first tried to test to communicate users service to albums service with feign client,
so I threw an exception at albums service to check if users service Error Decoder can catch the exception and then make the fallback method triggered.
It worked, but the cause is always null only at the first time, and after that I can see the error message that I wanted to see.
Can anyone tell me if something is wrong or not.
This is my code.
Users Service Feign Client
#FeignClient(name = "albums-ws", fallbackFactory = AlbumsFallbackFactory.class)
public interface AlbumServiceClient {
#GetMapping(path = "users/{userId}/albums")
List<AlbumDetailResponse> getAlbums(#PathVariable("userId") String userId);
}
Fallback Factory
#Component
public class AlbumsFallbackFactory implements FallbackFactory<AlbumServiceClient> {
#Override
public AlbumServiceClient create(Throwable cause) {
return new AlbumServiceClientFallback(cause);
}
}
public class AlbumServiceClientFallback implements AlbumServiceClient {
private final Throwable cause;
private final Logger logger = LoggerFactory.getLogger(this.getClass());
public AlbumServiceClientFallback(Throwable cause) {
this.cause = cause;
}
#Override
public List<AlbumDetailResponse> getAlbums(String userId) {
logger.error("An exception took place: " + cause.getMessage());
return new ArrayList<>();
}
}
Feign Error Decoder
#Component
public class FeignErrorDecoder implements ErrorDecoder {
#Override
public Exception decode(String methodKey, Response response) {
switch(response.status()) {
case 400:
break;
case 404:
if(methodKey.contains("getAlbums")) {
return new ResponseStatusException(HttpStatus.valueOf(response.status()), response.reason());
}
break;
default:
return new Exception(response.reason());
}
return null;
}
}
First fallback triggered
2020-08-02 12:42:27.836 ERROR 24772 --- [ HystrixTimer-1] c.a.p.a.u.P.f.AlbumServiceClientFallback : An exception took place: null
After
2020-08-02 12:43:07.672 DEBUG 24772 --- [rix-albums-ws-2] c.a.p.a.u.P.feign.AlbumServiceClient : [AlbumServiceClient#getAlbums] User not found with id: f5b313e2-411f-4fc3-95e7-9aa5c43c286c
Hystrix has class org.springframework.cloud.netflix.feign.HystrixTargeter. There is a comment in targetWithFallbackFactory method:
We take a sample fallback from the fallback factory to check if it
returns a fallback that is compatible with the annotated feign
interface.
and code after:
Object exampleFallback = fallbackFactory.create(new RuntimeException());
It is why you don't have cause in exception.
I have a case where I might have Java NPE into the listener which accepts queue payload. I get multiple attempts and errors:
18:41:50.549 [processingeContainer-1] WARN o.s.a.r.l.ConditionalRejectingErrorHandler - Execution of Rabbit message listener failed.
2019-09-24 18:41:50,551 INFO [stdout] (processingContainer-1) org.springframework.amqp.rabbit.listener.exception.ListenerExecutionFailedException: Listener method 'transactionProcess' threw exception
Is there some way to limit the AMQP client attempts?
You should really fix the NPE but you can configure the listener container error handler.
The default ConditionalRejectingErrorHandler treats certain exceptions as fatal.
It uses a DefaultExceptionStrategy which has the following code:
private boolean isCauseFatal(Throwable cause) {
return cause instanceof MessageConversionException // NOSONAR boolean complexity
|| cause instanceof org.springframework.messaging.converter.MessageConversionException
|| cause instanceof MethodArgumentResolutionException
|| cause instanceof NoSuchMethodException
|| cause instanceof ClassCastException
|| isUserCauseFatal(cause);
}
/**
* Subclasses can override this to add custom exceptions.
* #param cause the cause
* #return true if the cause is fatal.
*/
protected boolean isUserCauseFatal(Throwable cause) {
return false;
}
So, configure your own ConditionalRejectingErrorHandler with a subclass of DefaultExceptionStrategy which overrides isUserCauseFatal() return true for NullPointerException.
You would then inject your error hander into the listener container or listener container factory.
Another technique would be to add a retry interceptor; by default, the error is just logged after the retries are exhausted. With spring boot, the default recoverer is a RejectAndDontRequeueRecoverer.
EDIT
I just tested it and it worked fine...
#SpringBootApplication
public class So58087354Application {
public static void main(String[] args) {
SpringApplication.run(So58087354Application.class, args);
}
#RabbitListener(queues = "foo")
public void listen(String in) {
System.out.println("here");
throw new NullPointerException("Test");
}
}
spring.rabbitmq.listener.simple.retry.enabled=true
spring.rabbitmq.listener.simple.retry.initial-interval=1000ms
spring.rabbitmq.listener.simple.retry.max-attempts=2
and
here
here
2019-10-01 09:07:11.936 WARN 75435 --- [ntContainer#0-1] o.s.a.r.r.RejectAndDontRequeueRecoverer : Retries exhausted for message (Body:'[B#6d890bbc(byte[3])' MessageProperties [headers={}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=, receivedRoutingKey=foo, deliveryTag=1, consumerTag=amq.ctag-mwYtmPtBplrefsOa05hG0w, consumerQueue=foo])
...
2019-10-01 09:07:11.937 WARN 75435 --- [ntContainer#0-1] s.a.r.l.ConditionalRejectingErrorHandler : Execution of Rabbit message listener failed.
...
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: null
... 19 common frames omitted
EDIT2
To add a retry advice to the container factory manually...
#Component
class ContainerRetryConfigurer {
ContainerRetryConfigurer(AbstractRabbitListenerContainerFactory<?> factory) {
factory.setAdviceChain(RetryInterceptorBuilder.stateless()
.maxAttempts(2)
.backOffOptions(1000, 1.0, 1000)
.build());
}
}