So I'm trying to retry for specific exceptions and created a bean which has shouldRetry(Throwable t) function. The function returns true if exception has to be retried, otherwise false.
But What I'm observing is shouldRetry(Throwable t) is executing twice(log is printing twice) for one retry attempt, however serviceImpl from where exception is being thrown is executing only once for one retry attempt.
Could someone please let me know if I'm doing something wrong here, or is it the default behavior/bug with spring retry itself.
#Component("dbRecoverableExceptionHandler")
#RequiredArgsConstructor(onConstructor = #__(#Autowired))
#Slf4j
public class DBRecoverableExceptionHandler {
private final Environment environment;
private final MultiTaggedCounter exceptionRetryCounter;
public Boolean isRetryable(Throwable t) {
String[] recoverableExceptionClasses = environment
.getRequiredProperty("db-recoverable-exception-classes", String[].class);
for (String s1 : recoverableExceptionClasses) {
if (t.getClass().getSimpleName().contains(s1)) {
exceptionRetryCounter.increment(1, s1);
log.warn("Retrying for exception " + t.toString());
return true;
}
}
return false;
}
}
#Retryable(exceptionExpression = "#{#dbRecoverableExceptionHandler.isRetryable(#root)}",
maxAttemptsExpression = "#{${max-attempts}}",
backoff = #Backoff(delayExpression = "#{${retry-backoff-delay-time}}",
multiplierExpression = "#{${retry-backoff-multiplier}}"))
It is as expected.
The method will be called by the RetryTemplate twice for each execution...
while (canRetry(retryPolicy, context) && !context.isExhaustedOnly()) {
try {
if (this.logger.isDebugEnabled()) {
this.logger.debug("Retry: count=" + context.getRetryCount());
}
// Reset the last exception, so if we are successful
// the close interceptors will not think we failed...
lastException = null;
return retryCallback.doWithRetry(context);
}
catch (Throwable e) {
lastException = e;
try {
registerThrowable(retryPolicy, state, context, e);
}
catch (Exception ex) {
throw new TerminatedRetryException("Could not register throwable",
ex);
}
finally {
doOnErrorInterceptors(retryCallback, context, e);
}
if (canRetry(retryPolicy, context) && !context.isExhaustedOnly()) {
...
The first call to canRetry() (in the while loop) is skipped on the very first call since there is no exception yet, on subsequent iterations, when the method throws an exception, it is called twice.
Related
I have a springboot Kafka Consumer & Producer. The consumer is expected to read data from topic 1 by 1, process(time consuming) it & write it to another topic and then manually commit the offset.
In order to avoid rebalancing, I have tried to call pause() and resume() on KafkaContainer but the consumer is always running & never responds to pause() call, tried it even with a while loop and faced no success(unable to pause the consumer). KafkaListenerEndpointRegistry is Autowired.
Springboot version = 2.6.9, spring-kafka version = 2.8.7
#KafkaListener(id = "c1", topics = "${app.topics.topic1}", containerFactory = "listenerContainerFactory1")
public void poll(ConsumerRecord<String, String> record, Acknowledgment ack) {
log.info("Received Message by consumer of topic1: " + value);
String result = process(record.value());
producer.sendMessage(result + " topic2");
log.info("Message sent from " + topicIn + " to " + topicOut);
ack.acknowledge();
log.info("Offset committed by consumer 1");
}
private String process(String value) {
try {
pauseConsumer();
// Perform time intensive network IO operations
resumeConsumer();
} catch (InterruptedException e) {
log.error(e.getMessage());
}
return value;
}
private void pauseConsumer() throws InterruptedException {
if (registry.getListenerContainer("c1").isRunning()) {
log.info("Attempting to pause consumer");
Objects.requireNonNull(registry.getListenerContainer("c1")).pause();
Thread.sleep(5000);
log.info("kafkalistener container state - " + registry.getListenerContainer("c1").isRunning());
}
}
private void resumeConsumer() throws InterruptedException {
if (registry.getListenerContainer("c1").isContainerPaused() || registry.getListenerContainer("c1").isPauseRequested()) {
log.info("Attempting to resume consumer");
Objects.requireNonNull(registry.getListenerContainer("c1")).resume();
Thread.sleep(5000);
log.info("kafkalistener container state - " + registry.getListenerContainer("c1").isRunning());
}
}
Am I missing something? Could someone please guide me with the right way of achieving the required behaviour?
You are running the process() method on the listener thread so pause/resume will not have any effect; the pause only takes place when the listener thread exits the listener method (and after it has processed all the records received by the previous poll).
The next version (2.9), due later this month, has a new property pauseImmediate, which causes the pause to take effect after the current record is processed.
You can try like this. This work for me
public class kafkaConsumer {
public void run(String topicName) {
try {
Consumer<String, String> consumer = new KafkaConsumer<>(config);
consumer.subscribe(Collections.singleton(topicName));
while (true) {
try {
ConsumerRecords<String, String> consumerRecords = consumer.poll(Duration.ofMillis(80000));
for (TopicPartition partition : consumerRecords.partitions()) {
List<ConsumerRecord<String, String>> partitionRecords = consumerRecords.records(partition);
for (ConsumerRecord<String, String> record : partitionRecords) {
kafkaEvent = record.value();
consumer.pause(consumer.assignment());
/** Implement Your Business Logic Here **/
Once your processing done
consumer.resume(consumer.assignment());
try {
consumer.commitSync();
} catch (CommitFailedException e) {
}
}
}
} catch (Exception e) {
continue;
}
}
} catch (Exception e) {
}
}
I am making parallel call using completablefuture like below,
public Response getResponse() {
Response resultClass = new Response();
try {
CompletableFuture<Optional<ClassA>> classAFuture
= CompletableFuture.supplyAsync(() -> service.getClassA() );
CompletableFuture<ClassB> classBFuture
= CompletableFuture.supplyAsync(() -> {
try {
return service.getClassB();
}
catch (Exception e) {
throw new CompletionException(e);
}
});
CompletableFuture<Response> responseFuture =
CompletableFuture.allOf(classAFuture, classBFuture)
.thenApplyAsync(dummy -> {
if (classAFuture.join().isPresent() {
ClassA classA = classAFuture.join();
classA.setClassB(classBFuture.join());
response.setClassA(classA)
}
return response;
});
responseFuture.join();
} catch (CompletionExecution e) {
throw e;
}
return response;
}
I need to add try catch for return service.getClassB() as it throwing an exception inside the method getClassB.
Now problem I am facing is if service.getClassB() throws error it always comes wrapped inside java.util.concurrent.ExecutionException. There is a scenario where this method throws UserNameNotFoundException but this is wrapped inside ExecutionException and it is not getting caught in the right place #ControllerAdvice Exception handler class. I tried different option using throwable but it didn't help.
Is there a good way to handle the exception and unwrap and send it to #ControllerAdvice class?
Your code has several errors, like referring to a variable response that is not declared in this code and most probably supposed to be the resultClass declared at the beginning. The line
ClassA classA = classAFuture.join(); suddenly ignores that this future encapsulates an Optional and there are missing ) and ; separators.
Further, you should avoid accessing variables from the surrounding code when there’s a clean alternative. Also, using allOf to combine two futures is an unnecessary complication.
If I understood your intention correctly, you want to do something like
public Response getResponse() {
return CompletableFuture.supplyAsync(() -> service.getClassA())
.thenCombine(CompletableFuture.supplyAsync(() -> {
try {
return service.getClassB();
}
catch(ExecutionException e) {
throw new CompletionException(e.getCause());
}
}), (optA, b) -> {
Response response = new Response();
optA.ifPresent(a -> {
a.setClassB(b);
response.setClassA(a);
});
return response;
})
.join();
}
The key point to solve your described problem is to catch the most specific exception type. When you catch ExecutionException, you know that it will wrap the actual exception and can extract it unconditionally. When the getClassB() declares other checked exceptions which you have to catch, add another catch clause, but be specific instead of catching Exception, e.g.
try {
return service.getClassB();
}
catch(ExecutionException e) {
throw new CompletionException(e.getCause());
}
catch(IOException | SQLException e) { // all of getClassB()’s declared exceptions
throw new CompletionException(e); // except ExecutionException, of course
}
method I am testing (the method setEventHubDataPayload throws JSONException and JsonProcessingException):
public class EventHubMapper {
//inits
public byte[] toEventDataJsonByteArray(UserRecord inbound) {
EventHubDto ehDto = new EventHubDto();
ehDto.setEventTypeVersion(inbound.getVersion());
ehDto.setEventId(inbound.getNotificationId());
JSONObject eventJson = new JSONObject(ehDto);
try {
eventJson.put("data", setEventHubDataPayload(ehDto, inbound));
} catch (JSONException e) {
analytics.trackError(AnalyticsConstants.EventHub.JSON_MAPPING_ERROR, e.toString());
} catch (JsonProcessingException e) {
analytics.trackError(AnalyticsConstants.EventHub.JSON_PROCESSING_ERROR, e.toString());
}
return eventJson.toString().getBytes();
}
}
unit test code:
#Test
public void toEventDataByteArray_JsonException() throws JSONException, JsonProcessingException {
EventHubMapper ehmMock = Mockito.spy(eventHubMapper);
doThrow(new JSONException("blah")).when(ehmMock).setEventHubDataPayload(any(), any());
eventHubMapper.toEventDataJsonByteArray(setUpMockUserRecord());
verify(analytics, times(1)).trackError( AnalyticsConstants.EventHub.JSON_MAPPING_ERROR, new JSONException("blah").toString());
}
I've tried using more specific matchers ... ex: any(EventHubDto.class) or any(UserRecord.class) and got the same result:
Wanted but not invoked:
analytics.trackError(
"EventHub_Publish_Error",
""
;
and also
Actually, there were zero interactions with this mock.
what is going on here?
I think you need to call like below while testing.
ehmMock.toEventDataJsonByteArray(setUpMockUserRecord());
Hi I m trying to use httpcomponents5 beta to make persistent connection, I have tried the example given in their site, the code is as follows,
final IOReactorConfig ioReactorConfig = IOReactorConfig.custom().setSoTimeout(Timeout.ofSeconds(45)).setSelectInterval(10000).setSoReuseAddress(true).setSoKeepAlive(true).build();
final SSLContext sslContext = SSLContexts.custom().loadTrustMaterial(new TrustAllStrategy()).build();
final PoolingAsyncClientConnectionManager connectionManager = PoolingAsyncClientConnectionManagerBuilder.create().setConnectionTimeToLive(TimeValue.of(1, TimeUnit.DAYS)).setTlsStrategy(new H2TlsStrategy(sslContext, NoopHostnameVerifier.INSTANCE)).build();
client = HttpAsyncClients.createMinimal(protocol, H2Config.DEFAULT, null, ioReactorConfig, connectionManager);
client.start();
final org.apache.hc.core5.http.HttpHost target = new org.apache.hc.core5.http.HttpHost("localhost", 8000, "https");
Future<AsyncClientEndpoint> leaseFuture = client.lease(target, null);
AsyncClientEndpoint asyncClientEndpoint = leaseFuture.get(60, TimeUnit.SECONDS);
final CountDownLatch latch = new CountDownLatch(1);
final AsyncRequestProducer requestProducer = AsyncRequestBuilder.post(target.getSchemeName()+"://"+target.getHostName()+":"+target.getPort()+locationposturl).addParameter(new BasicNameValuePair("info", requestData)).setEntity(new StringAsyncEntityProducer("json post data will go here", ContentType.APPLICATION_JSON)).setHeader("Pragma", "no-cache").setHeader("from", "http5").setHeader("Custom", customheaderName).setHeader("Secure", secureHeader).build();
locEndPoint.execute(requestProducer, SimpleResponseConsumer.create(), new FutureCallback<SimpleHttpResponse>() {
#Override
public void completed(final SimpleHttpResponse response) {
if (response != null) {
if (response.getCode() > -1) {
try {
System.out.println("http5:: COMPLETED : RESPONSE "+response.getBodyText());
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
latch.countDown();
}
#Override
public void failed(final Exception ex) {
System.out.println("http5:: FAILED : "+target+locationposturl);
LoggerUtil.printStackTrace(ex);
System.out.println("http5::Exception Request failed "+LoggerUtil.getStackTrace(ex));
latch.countDown();
}
#Override
public void cancelled() {
System.out.println("http5:: CANCELLED : "+target+locationposturl);
System.out.println(http5::Exception Request cancelled");
latch.countDown();
}
});
latch.await();
This code works without a problem for the first time,but when I send a subsequent requests it throws an exception as follows,
http5:: Exception occured java.lang.IllegalStateException: Endpoint is
not connected at
org.apache.hc.core5.util.Asserts.check(Asserts.java:38) at
org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager$InternalConnectionEndpoint.getValidatedPoolEntry(PoolingAsyncClientConnectionManager.java:497)
at
org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManager$InternalConnectionEndpoint.execute(PoolingAsyncClientConnectionManager.java:552)
at
org.apache.hc.client5.http.impl.async.MinimalHttpAsyncClient$InternalAsyncClientEndpoint.execute(MinimalHttpAsyncClient.java:405)
at
org.apache.hc.core5.http.nio.AsyncClientEndpoint.execute(AsyncClientEndpoint.java:81)
at
org.apache.hc.core5.http.nio.AsyncClientEndpoint.execute(AsyncClientEndpoint.java:114)
What may be the problem with endpoint, I m forcing endpoint to keep alive for a day, kindly shed some light on this
I am processing three transaction inside a single method in stateless container managed bean .i want to persist three transaction while if one throws exception other two should complete their respective transaction ,error is that if first or any one is throwing exception other two are to executing please give some helpful suggestion
public void allocateSubjectToStudent(SubjectAllocatedToStudentDto dto)throws Exception {
logger.info("allocateSubjectToStudent method entry :");
List<Subject> coreList=dto.getCoreList();
Iterator<Subject> iterator=coreList.iterator();
while(iterator.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("Core subject already allocated to student");
} //end of if
}//end of catch
}//end of while
List<Subject> departmentallist=dto.getDepartmentList();
Iterator<Subject> iterator1=departmentallist.iterator();
while(iterator1.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator1.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("InterDepartmental subject already allocated to student");
} //end of if
}//end of catch
}//end of while
List<Subject> electiveList=dto.getElectiveList();
Iterator<Subject> iterator2=electiveList.iterator();
while(iterator2.hasNext()){
logger.info("inside while :");
SubjectAllocatedToStudentBo bo=new SubjectAllocatedToStudentBo();
bo.setBacthId(dto.getBacthId());
bo.setSemester(dto.getSemester());
bo.setStudentId(dto.getStudentId());
Subject subject=iterator2.next();
bo.setSubjectName(subject.getSubjectName());
bo.setSubjectType(subject.getAbbreviation());
try{
manager.persist(bo);
}
catch(javax.persistence.PersistenceException e){
Throwable t = e.getCause();
while ((t != null) && !(t instanceof org.hibernate.exception.ConstraintViolationException)) {
t = t.getCause();
}//while
if (t instanceof org.hibernate.exception.ConstraintViolationException) {
throw new Exception("Elective subject already allocated to student");
} //end of if
}//end of catch
}//end of while
logger.info("allocateSubjectToStudent method exit :");
} //end of method
create three different method all with TranscationAttributeType REQUIRES_NEW
Please find below code snippet for EJB3 Bean
public void doYourWork()
{
a();
b();
c();
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void a()
{
try
{
//Do the first transaction here
}catch(Exception e)
{
}
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void b()
{
try
{
//Do the second transaction here
}catch(Exception e)
{
}
}
#TransactionAttribute(TransactionAttributeType.REQUIRES_NEW)
public void c()
{
try
{
//Do the third transaction here
}catch(Exception e)
{
}
}
Within a single method invocation there is only a single transaction active. To achieve what you want, you must perform the three operations in different transactions. This would require one more level of abstracttion.
public class MyFreshTransaction {
#TransactionAttribute(REQUIRES_NEW)
public void updateO() {
//do something
}
#TransactionAttribute(REQUIRES_NEW)
public void update1() {
//do something
}
#TransactionAttribute(REQUIRES_NEW)
public void update2() {
//do something
}
}
#Stateless
public class MyTransactionProcessor {
#EJB
private MyFreshTransaction freshTransaction;
public void processTransaction() {
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update0();
} catch (Exception ex ) {//handle}
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update1();
} catch (Exception ex ) {//handle}
try {
//The current transaction context will be suspended, and a new one invoked
//if the new one fails and is rollback, the current one is not affected.
//you can then handle the exception, by rethrowing the exception,in which case
//the current transaction will also be rolled back, or continue based on your logic.
freshTransaction.update2();
} catch (Exception ex ) {//handle}
}
}
Note that if any of the update transaction was successful, and the the parent transaction is rolled back, it will not affect the status of the 'child' transactions, as they had already been committed and their effects (if DB effects) will be committed too.
Read on Java EE Transactions Java EE Transactions