I am receiving the following error when trying to ouptut a aggregate graph within jmeter:
2018-10-11 09:15:00,731 ERROR o.a.j.JMeter: Uncaught exception:
java.lang.IndexOutOfBoundsException: Index -1 out-of-bounds for length 13
at jdk.internal.util.Preconditions.outOfBounds(Preconditions.java:64) ~[?:?]
at jdk.internal.util.Preconditions.outOfBoundsCheckIndex(Preconditions.java:70) ~[?:?]
at jdk.internal.util.Preconditions.checkIndex(Preconditions.java:248) ~[?:?]
at java.util.Objects.checkIndex(Objects.java:372) ~[?:?]
at java.util.ArrayList.get(ArrayList.java:440) ~[?:?]
at org.apache.jorphan.gui.ObjectTableModel.getValueAt(ObjectTableModel.java:187) ~[jorphan.jar:5.0 r1840935]
at org.apache.jmeter.visualizers.StatGraphVisualizer.getData(StatGraphVisualizer.java:638) ~[ApacheJMeter_components.jar:5.0 r1840935]
at org.apache.jmeter.visualizers.StatGraphVisualizer.makeGraph(StatGraphVisualizer.java:594) ~[ApacheJMeter_components.jar:5.0 r1840935]
at org.apache.jmeter.visualizers.StatGraphVisualizer.actionMakeGraph(StatGraphVisualizer.java:790) ~[ApacheJMeter_components.jar:5.0 r1840935]
at org.apache.jmeter.visualizers.StatGraphVisualizer.actionPerformed(StatGraphVisualizer.java:702) ~[ApacheJMeter_components.jar:5.0 r1840935]
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1967) ~[?:?]
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2308) ~[?:?]
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:405) ~[?:?]
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:262) ~[?:?]
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:270) ~[?:?]
at java.awt.Component.processMouseEvent(Component.java:6589) ~[?:?]
at javax.swing.JComponent.processMouseEvent(JComponent.java:3342) ~[?:?]
at java.awt.Component.processEvent(Component.java:6354) ~[?:?]
at java.awt.Container.processEvent(Container.java:2261) ~[?:?]
at java.awt.Component.dispatchEventImpl(Component.java:4966) ~[?:?]
at java.awt.Container.dispatchEventImpl(Container.java:2319) ~[?:?]
at java.awt.Component.dispatchEvent(Component.java:4798) ~[?:?]
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4914) ~[?:?]
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4543) ~[?:?]
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4484) ~[?:?]
at java.awt.Container.dispatchEventImpl(Container.java:2305) ~[?:?]
at java.awt.Window.dispatchEventImpl(Window.java:2772) ~[?:?]
at java.awt.Component.dispatchEvent(Component.java:4798) ~[?:?]
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:772) ~[?:?]
at java.awt.EventQueue.access$600(EventQueue.java:97) ~[?:?]
at java.awt.EventQueue$4.run(EventQueue.java:721) ~[?:?]
at java.awt.EventQueue$4.run(EventQueue.java:715) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:97) ~[?:?]
at java.awt.EventQueue$5.run(EventQueue.java:745) ~[?:?]
at java.awt.EventQueue$5.run(EventQueue.java:743) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:?]
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:87) ~[?:?]
at java.awt.EventQueue.dispatchEvent(EventQueue.java:742) ~[?:?]
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:203) [?:?]
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:124) [?:?]
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:113) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:109) [?:?]
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101) [?:?]
at java.awt.EventDispatchThread.run(EventDispatchThread.java:90) [?:?]
2018-10-11 09:15:04,438 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [#_samples]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [median]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [90%_line]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [95%_line]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [99%_line]
2018-10-11 09:15:04,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [min]
2018-10-11 09:15:04,522 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [error_%]
2018-10-11 09:15:04,522 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [throughput]
2018-10-11 09:15:04,524 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [received_kb/sec]
2018-10-11 09:15:04,524 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [sent_kb/sec]
2018-10-11 09:15:04,551 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
2018-10-11 09:15:04,552 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
2018-10-11 09:15:04,552 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
2018-10-11 09:15:04,552 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [label]
This means the graph cannot be generated and I am unsure how to fix the issue.
These are the results and settings of my aggregate graph, can anybody help me with this issue please?
Thanks
This is a known bug of JMeter 5.0:
https://bz.apache.org/bugzilla/show_bug.cgi?id=62770
It is fixed in nightly build:
https://builds.apache.org/job/JMeter-trunk/lastSuccessfulBuild/artifact/trunk/dist/
And will be available in 5.1
Anyway, the best way to analyze your test is to use the Web Report which exists since 3.0 version:
https://jmeter.apache.org/usermanual/generating-dashboard.html
Related
I have a simple Spring Boot application where I have the following settings for RabbitMQ (spring-boot-starter-amqp version is 2.7.0):
spring:
rabbitmq:
host: localhost
port: 5672
virtual-host: my_host
username: admin
password: password
template:
retry:
enabled: true
initial-interval: 3s
max-interval: 10s
multiplier: 2
max-attempts: 3
listener:
simple:
retry:
enabled: true
initial-interval: 3s
max-interval: 10s
multiplier: 2
max-attempts: 3
A SimpleRabbitListenerContainerFactory is configured with a MessagePostProcessor:
#Bean
public SimpleRabbitListenerContainerFactory myInterceptContainerFactory(final SimpleRabbitListenerContainerFactoryConfigurer configurer,
final ConnectionFactory connectionFactory,
final MyMessagePostProcessor myMessagePostProcessor) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory() {
#Override
protected void initializeContainer(SimpleMessageListenerContainer instance, RabbitListenerEndpoint endpoint) {
instance.setAfterReceivePostProcessors(myMessagePostProcessor);
super.initializeContainer(instance, endpoint);
}
};
((CachingConnectionFactory) connectionFactory).setRequestedHeartBeat(60);
configurer.configure(factory, connectionFactory);
return factory;
}
MessagePostProcessor is very simple, if the version is wrong in the header, it throws an AmqpRejectAndDontRequeueException:
#Component
public class MyMessagePostProcessor implements MessagePostProcessor {
private static final Logger LOGGER = LogManager.getLogger(MyMessagePostProcessor.class);
#Override
public Message postProcessMessage(Message message) throws AmqpException {
if (!"1.0".equalsIgnoreCase(message.getMessageProperties().getHeader("version"))) {
LOGGER.error("wrong version");
throw new AmqpRejectAndDontRequeueException("wrong version");
}
return message;
}
}
When I send a message with a wrong version, it works properly based on the logs:
MyApp: 2022-07-05 15:49:33,675 ERROR [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] c.g.r.a.q.p.MyMessagePostProcessor:22 - wrong version
MyApp: 2022-07-05 15:49:33,676 WARN [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] o.s.a.r.l.ConditionalRejectingErrorHandler:169 - Execution of Rabbit message listener failed.
org.springframework.amqp.AmqpRejectAndDontRequeueException: wrong version
at com.my.app.queue.postprocess.MyMessagePostProcessor.postProcessMessage(MyMessagePostProcessor.java:23) ~[classes/:1.0.0]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1544) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1499) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:992) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:939) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:84) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1316) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1222) ~[spring-rabbit-2.4.5.jar:2.4.5]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
My consumer is only doing one error log and immediately throws AmqpRejectAndDontRequeueException to see if the message goes into the deadletter queue without retry:
#Component
public class MyMessageConsumer {
private static final Logger LOGGER = LogManager.getLogger(MyMessageConsumer.class);
#RabbitListener(queues = "my.queue", containerFactory = "myInterceptContainerFactory")
public void consumerMessage() {
LOGGER.error("error in consuming message");
throw new AmqpRejectAndDontRequeueException("over");
}
}
But based on the logs it is clearly visible that it retries 3 times (because my error log is printed 3 times) and only after then the message is delivered into deadletter queue:
MyApp: 2022-07-05 15:50:32,182 ERROR [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] c.g.r.a.q.c.MyMessageConsumer:23 - error in consuming message
MyApp: 2022-07-05 15:50:46,603 ERROR [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] c.g.r.a.q.c.MyMessageConsumer:23 - error in consuming message
MyApp: 2022-07-05 15:50:52,617 ERROR [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] c.g.r.a.q.c.MyMessageConsumer:23 - error in consuming message
MyApp: 2022-07-05 15:50:52,618 WARN [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] o.s.a.r.r.RejectAndDontRequeueRecoverer:74 - Retries exhausted for message (Body:'[B#4060893b(byte[75])' MessageProperties [headers={version=1.0, requestor=MyApp}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=false, receivedExchange=, receivedRoutingKey=my.queue, deliveryTag=2, consumerTag=amq.ctag-5Xx6EwwjVokRV6wD9xL8Og, consumerQueue=my.queue])
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener method 'public void com.my.app.queue.consumer.MyMessageConsumer.retrieveNcid()' threw exception
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:271) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:208) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:147) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1657) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1576) ~[spring-rabbit-2.4.5.jar:2.4.5]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:97) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:225) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:122) ~[spring-retry-1.3.3.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.amqp.rabbit.listener.$Proxy187.invokeListener(Unknown Source) ~[?:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1564) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1555) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1499) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:992) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:939) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:84) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1316) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1222) ~[spring-rabbit-2.4.5.jar:2.4.5]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: over
at com.my.app.queue.consumer.MyMessageConsumer.retrieveNcid(MyMessageConsumer.java:24) ~[classes/:1.0.0]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169) ~[spring-messaging-5.3.20.jar:5.3.20]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119) ~[spring-messaging-5.3.20.jar:5.3.20]
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:75) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:262) ~[spring-rabbit-2.4.5.jar:2.4.5]
... 27 more
MyApp: 2022-07-05 15:50:52,620 WARN [org.springframework.amqp.rabbit.RabbitListenerEndpointContainer#0-1] o.s.a.r.l.ConditionalRejectingErrorHandler:169 - Execution of Rabbit message listener failed.
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Retry Policy Exhausted
at org.springframework.amqp.rabbit.retry.RejectAndDontRequeueRecoverer.recover(RejectAndDontRequeueRecoverer.java:76) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.config.StatelessRetryOperationsInterceptorFactoryBean.recover(StatelessRetryOperationsInterceptorFactoryBean.java:77) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.retry.interceptor.RetryOperationsInterceptor$ItemRecovererCallback.recover(RetryOperationsInterceptor.java:157) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.handleRetryExhausted(RetryTemplate.java:539) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:387) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:225) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.interceptor.RetryOperationsInterceptor.invoke(RetryOperationsInterceptor.java:122) ~[spring-retry-1.3.3.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.amqp.rabbit.listener.$Proxy187.invokeListener(Unknown Source) ~[?:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:1564) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:1555) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.executeListener(AbstractMessageListenerContainer.java:1499) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.doReceiveAndExecute(SimpleMessageListenerContainer.java:992) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.receiveAndExecute(SimpleMessageListenerContainer.java:939) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer.access$1600(SimpleMessageListenerContainer.java:84) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.mainLoop(SimpleMessageListenerContainer.java:1316) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1222) ~[spring-rabbit-2.4.5.jar:2.4.5]
at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException
... 19 more
Caused by: org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener method 'public void com.my.app.queue.consumer.MyMessageConsumer.retrieveNcid()' threw exception
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:271) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:208) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:147) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1657) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1576) ~[spring-rabbit-2.4.5.jar:2.4.5]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:97) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329) ~[spring-retry-1.3.3.jar:?]
... 14 more
Caused by: org.springframework.amqp.AmqpRejectAndDontRequeueException: over
at com.my.app.queue.consumer.MyMessageConsumer.retrieveNcid(MyMessageConsumer.java:24) ~[classes/:1.0.0]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169) ~[spring-messaging-5.3.20.jar:5.3.20]
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119) ~[spring-messaging-5.3.20.jar:5.3.20]
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:75) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:262) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandlerAndProcessResult(MessagingMessageListenerAdapter.java:208) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.onMessage(MessagingMessageListenerAdapter.java:147) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:1657) ~[spring-rabbit-2.4.5.jar:2.4.5]
at org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer.actualInvokeListener(AbstractMessageListenerContainer.java:1576) ~[spring-rabbit-2.4.5.jar:2.4.5]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?]
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
at java.lang.reflect.Method.invoke(Method.java:568) ~[?:?]
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.20.jar:5.3.20]
at org.springframework.retry.interceptor.RetryOperationsInterceptor$1.doWithRetry(RetryOperationsInterceptor.java:97) ~[spring-retry-1.3.3.jar:?]
at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:329) ~[spring-retry-1.3.3.jar:?]
... 14 more
But I want to deliver the message into the deadletter queue without retrying it 3 times.
I guess there is something with ListenerExecutionFailedException: Retry Policy Exhausted, but I don't understand why my AmqpRejectAndDontRequeueException is wrapped in this exception, because my MessagePostProcessor doesn't throw ListenerExecutionFailedException, only the one which I throw in code.
What am I missing here?
As #Gary Russel suggested, I modified the retry policy as below:
#Bean
public RabbitRetryTemplateCustomizer customizeRetryPolicy(#Value("${spring.rabbitmq.listener.simple.retry.max-attempts}") int maxAttempts) {
SimpleRetryPolicy policy = new SimpleRetryPolicy(maxAttempts, Map.of(AmqpRejectAndDontRequeueException.class, false), true, true);
return (target, retryTemplate) -> retryTemplate.setRetryPolicy(policy);
}
And only configuring this new bean, everything works as expected!
The container knows nothing about the exception until retries are exhausted.
You would need to customize the retry policy to not retry for that exception.
This is currently not possible using application properties; you would need to configure the retry interceptor as a bean and inject it into the container factory.
If you need help with that, I can add an example.
Getting "alluxio.exception.status.UnimplementedException: Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists" while trying to check the existence of directory inside alluxio. What wrong i am doing here.
Code:
public void createAlluxioDirectories(String partitionPath) {
// initialize the alluxio system
InstancedConfiguration conf = initAlluxioCnf();
// create the directory
try (FileSystem fs = FileSystem.Factory.create(conf)) {
AlluxioURI alluxioPath = new AlluxioURI(partitionPath);
if (!fs.exists(alluxioPath)) {
fs.createDirectory(alluxioPath, CreateDirectoryPOptions.newBuilder().setRecursive(true).build());
setAclPermission(fs, alluxioPath);
}
WbsLogger.trace(AlluxioUtility.class, "NA", "Successfully created alluxio path {}", alluxioPath.getPath());
} catch (Exception e) {
throw e;
}
}
Using implementation group: 'org.alluxio', name: 'alluxio-shaded-client', version: '2.8.0-2'
Exception Trace:
org.allupoc.gbs.cps.exp.type.CpsServiceExecutionException: *Root Exception Clause: [alluxio.exception.AlluxioException: Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists][Custom Exception Clause: JEPS_SYSTEM_FAILURE: {System is unable to process your request at this moment due to exception. Cause of failure: Directory creation in aluxio failed. Cause : Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists}]
at com.allupoc.r2r.generic.cpm.alluxio.AlluxioUtility.createAlluxioDirectories(AlluxioUtility.java:60) ~[main/:?]
at com.allupoc.r2r.generic.cpm.hive.HiveIndexTypeMapper.createHiveTables(HiveIndexTypeMapper.java:66) ~[main/:?]
at com.allupoc.r2r.generic.cpm.hive.HiveCnfRegisterar.lambda$0(HiveCnfRegisterar.java:49) ~[main/:?]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384) ~[?:1.8.0_301]
at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580) ~[?:1.8.0_301]
at com.allupoc.r2r.generic.cpm.hive.HiveCnfRegisterar.reloadHiveSchemasIfAny(HiveCnfRegisterar.java:47) ~[main/:?]
at com.allupoc.r2r.generic.cpm.hive.HiveCnf.onApplicationEvent(HiveCnf.java:64) ~[main/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_301]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_301]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_301]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_301]
at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:344) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:229) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:166) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:176) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:169) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:143) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:421) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:378) ~[spring-context-5.3.20.jar:5.3.20]
at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:114) ~[spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:82) ~[spring-boot-2.7.0.jar:2.7.0]
at java.util.ArrayList.forEach(ArrayList.java:1259) ~[?:1.8.0_301]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:120) ~[spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:114) ~[spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:82) ~[spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:323) [spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306) [spring-boot-2.7.0.jar:2.7.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295) [spring-boot-2.7.0.jar:2.7.0]
at com.allupoc.cps.jeps.wbs.IbmJepsWbsApplication.main(IbmJepsWbsApplication.java:90) [main/:?]
Caused by: alluxio.exception.AlluxioException: Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists
at alluxio.exception.status.AlluxioStatusException.toAlluxioException(AlluxioStatusException.java:105) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.rpc(BaseFileSystem.java:581) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.exists(BaseFileSystem.java:200) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.FileSystem.exists(FileSystem.java:275) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at com.allupoc.r2r.generic.cpm.alluxio.AlluxioUtility.createAlluxioDirectories(AlluxioUtility.java:54) ~[main/:?]
... 28 more
Caused by: alluxio.exception.status.UnimplementedException: Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists
at alluxio.exception.status.AlluxioStatusException.from(AlluxioStatusException.java:152) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.exception.status.AlluxioStatusException.fromStatusRuntimeException(AlluxioStatusException.java:224) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPCInternal(AbstractClient.java:403) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPC(AbstractClient.java:367) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPC(AbstractClient.java:356) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.RetryHandlingFileSystemMasterClient.exists(RetryHandlingFileSystemMasterClient.java:189) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.lambda$exists$4(BaseFileSystem.java:203) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.rpc(BaseFileSystem.java:566) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.exists(BaseFileSystem.java:200) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.FileSystem.exists(FileSystem.java:275) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at com.allupoc.r2r.generic.cpm.alluxio.AlluxioUtility.createAlluxioDirectories(AlluxioUtility.java:54) ~[main/:?]
... 28 more
Caused by: alluxio.shaded.client.io.grpc.StatusRuntimeException: UNIMPLEMENTED: Method not found: alluxio.grpc.file.FileSystemMasterClientService/Exists
at alluxio.shaded.client.io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:262) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.shaded.client.io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:243) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.shaded.client.io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:156) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.grpc.FileSystemMasterClientServiceGrpc$FileSystemMasterClientServiceBlockingStub.exists(FileSystemMasterClientServiceGrpc.java:1733) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.RetryHandlingFileSystemMasterClient.lambda$exists$6(RetryHandlingFileSystemMasterClient.java:189) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPCInternal(AbstractClient.java:401) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPC(AbstractClient.java:367) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.AbstractClient.retryRPC(AbstractClient.java:356) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.RetryHandlingFileSystemMasterClient.exists(RetryHandlingFileSystemMasterClient.java:189) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.lambda$exists$4(BaseFileSystem.java:203) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.rpc(BaseFileSystem.java:566) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.BaseFileSystem.exists(BaseFileSystem.java:200) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at alluxio.client.file.FileSystem.exists(FileSystem.java:275) ~[alluxio-shaded-client-2.8.0-2.jar:?]
at com.allupoc.r2r.generic.cpm.alluxio.AlluxioUtility.createAlluxioDirectories(AlluxioUtility.java:54) ~[main/:?]
When I start my ES nodes, I've been seeing the message client did not trust this server's certificate, closing connection and the following exception.
Anybody knows what it is about? After some seconds the exceptions stop and the clsuter seems to work ok.
Thanks.
[2020-06-18T22:11:09,608][INFO ][o.e.t.TransportService ] [my-es-hostname] publish_address {172.55.55.55:9300}, bound_addresses {[::]:9300}
[2020-06-18T22:11:09,623][INFO ][o.e.b.BootstrapChecks ] [my-es-hostname] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2020-06-18T22:11:10,769][WARN ][o.e.t.TcpTransport ] [my-es-hostname] exception caught on transport layer [Netty4TcpChannel{localAddress=/0:0:0:0:0:0:0:1:51060, remoteAddress=/0:0:0:0:0:0:0:1:9300}], closing connection
io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:473) ~[netty-codec-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:281) ~[netty-codec-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:352) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1422) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:374) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:360) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:931) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:700) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:600) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:554) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514) [netty-transport-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050) [netty-common-4.1.43.Final.jar:4.1.43.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.43.Final.jar:4.1.43.Final]
at java.lang.Thread.run(Thread.java:830) [?:?]
Caused by: javax.net.ssl.SSLHandshakeException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found
at sun.security.ssl.Alert.createSSLException(Alert.java:131) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267) ~[?:?]
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262) ~[?:?]
at sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1331) ~[?:?]
at sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1206) ~[?:?]
at sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1153) ~[?:?]
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:396) ~[?:?]
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1260) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1247) ~[?:?]
at java.security.AccessController.doPrivileged(AccessController.java:691) ~[?:?]
at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1192) ~[?:?]
at io.netty.handler.ssl.SslHandler.runAllDelegatedTasks(SslHandler.java:1502) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1516) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1400) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1227) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1274) ~[netty-handler-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:503) ~[netty-codec-4.1.43.Final.jar:4.1.43.Final]
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442) ~[netty-codec-4.1.43.Final.jar:4.1.43.Final]
... 16 more
Caused by: java.security.cert.CertificateException: No subject alternative names matching IP address 0:0:0:0:0:0:0:1 found
[2020-06-18T22:11:10,837][WARN ][o.e.x.c.s.t.n.SecurityNetty4Transport] [my-es-hostname] client did not trust this server's certificate, closing connection Netty4TcpChannel{localAddress=/127.0.0.1:9300, remoteAddress=/127.0.0.1:43784}
[2020-06-18T22:11:10,848][WARN ][o.e.t.TcpTransport ] [my-es-hostname] exception caught on transport layer [Netty4TcpChannel{localAddress=/127.0.0.1:43784, remoteAddress=/127.0.0.1:9300}], closing connection
I am using JMeter 3.2 r1790748 and getting the below messages in the log.
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_equation_healthy]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_equation_active]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_equation_warning]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_equation_dead]
2017-08-25 11:41:52,208 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_equation_load]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_tab_title]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_legend_health]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_legend_load]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_legend_memory_per]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_legend_thread_per]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_left_top]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_left_middle]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_left_bottom]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_right_healthy]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_right_dead]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_performance_title]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_performance_servers]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_performance_tab_title]
2017-08-25 11:41:52,224 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_label_prefix]
2017-08-25 11:41:52,239 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,239 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,427 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,427 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,427 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,427 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,427 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,443 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
2017-08-25 11:41:52,521 WARN o.a.j.u.JMeterUtils: ERROR! Resource string not found: [monitor_health_title]
Does anyone know what are these errors are about and how to resolve them?
You seem to be having a component of a 3rd Party plugin that relies on resources of a removed component called Monitor Results.
See:
https://bz.apache.org/bugzilla/show_bug.cgi?id=60423
And release notes of JMeter 3.2 in section Removed elements or functions:
http://jmeter.apache.org/changes.html#Incompatible%20changes
Bug 60423 - Drop Monitor Results listener
we are using connection pooling which is provided by mule in Database component. we are getting below error when we applied huge load.
configured parameters as : maxConnection=100, minConn=0, autoIncrement =15 waitMillisec=10000 and prepared statement cache size kept it as default(i.e did not configure anything).
Could you please assit on this to reslove below.
{"timestamp":"2016-10-22T09:28:13,611","level":"WARN","thread":"ActiveMQ Session Task-25","loggerName":"com.mchange.v2.c3p0.impl.NewPooledConnection","message":"[c3p0] A PooledConnection that has already signalled a Connection error is still in use!"}
{"timestamp":"2016-10-22T09:28:13,611","level":"WARN","thread":"ActiveMQ Session Task-25","loggerName":"com.mchange.v2.c3p0.impl.NewPooledConnection","message":"[c3p0] Another error has occurred [ java.sql.SQLException: A problem occurred while trying to acquire a cached PreparedStatement in a background thread. ] which will not be reported to listeners!"}
java.sql.SQLException: A problem occurred while trying to acquire a cached PreparedStatement in a background thread.
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.acquireStatement(GooGooStatementCache.java:571) ~[?:?]
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkoutStatement(GooGooStatementCache.java:204) ~[?:?]
at com.mchange.v2.c3p0.impl.NewPooledConnection.checkoutStatement(NewPooledConnection.java:321) ~[?:?]
at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:455) ~[?:?]
at org.mule.module.db.internal.domain.connection.DefaultDbConnection.prepareStatement(DefaultDbConnection.java:52) ~[?:?]
at org.mule.module.db.internal.domain.autogeneratedkey.NoAutoGeneratedKeyStrategy.prepareStatement(NoAutoGeneratedKeyStrategy.java:34) ~[?:?]
at org.mule.module.db.internal.domain.statement.QueryStatementFactory.create(QueryStatementFactory.java:59) ~[?:?]
at org.mule.module.db.internal.domain.executor.AbstractSingleQueryExecutor.execute(AbstractSingleQueryExecutor.java:44) ~[?:?]
at org.mule.module.db.internal.processor.UpdateMessageProcessor.doExecuteQuery(UpdateMessageProcessor.java:59) ~[?:?]
at org.mule.module.db.internal.processor.AbstractSingleQueryDbMessageProcessor.executeQuery(AbstractSingleQueryDbMessageProcessor.java:42) ~[?:?]
at org.mule.module.db.internal.processor.AbstractDbMessageProcessor.process(AbstractDbMessageProcessor.java:66) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.chain.DefaultMessageProcessorChain.doProcess(DefaultMessageProcessorChain.java:80) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.doProcess(InterceptingChainLifecycleWrapper.java:50) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.access$001(InterceptingChainLifecycleWrapper.java:22) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper$1.process(InterceptingChainLifecycleWrapper.java:66) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.process(InterceptingChainLifecycleWrapper.java:61) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.chain.DefaultMessageProcessorChain.doProcess(DefaultMessageProcessorChain.java:80) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.AbstractInterceptingMessageProcessorBase.processNext(AbstractInterceptingMessageProcessorBase.java:98) ~[?:?]
at org.mule.construct.DynamicPipelineMessageProcessor.process(DynamicPipelineMessageProcessor.java:55) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.chain.DefaultMessageProcessorChain.doProcess(DefaultMessageProcessorChain.java:80) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.AbstractInterceptingMessageProcessorBase.processNext(AbstractInterceptingMessageProcessorBase.java:98) ~[?:?]
at org.mule.interceptor.AbstractEnvelopeInterceptor.processBlocking(AbstractEnvelopeInterceptor.java:58) ~[?:?]
at org.mule.processor.AbstractRequestResponseMessageProcessor.process(AbstractRequestResponseMessageProcessor.java:47) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.AbstractInterceptingMessageProcessorBase.processNext(AbstractInterceptingMessageProcessorBase.java:98) ~[?:?]
at org.mule.processor.AbstractFilteringMessageProcessor.process(AbstractFilteringMessageProcessor.java:41) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:98) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.AbstractInterceptingMessageProcessorBase.processNext(AbstractInterceptingMessageProcessorBase.java:98) ~[?:?]
at org.mule.processor.AbstractRequestResponseMessageProcessor.processBlocking(AbstractRequestResponseMessageProcessor.java:56) ~[?:?]
at org.mule.processor.AbstractRequestResponseMessageProcessor.process(AbstractRequestResponseMessageProcessor.java:47) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.executeNext(BlockingProcessorExecutor.java:88) ~[?:?]
at org.mule.processor.BlockingProcessorExecutor.execute(BlockingProcessorExecutor.java:59) ~[?:?]
at org.mule.processor.chain.DefaultMessageProcessorChain.doProcess(DefaultMessageProcessorChain.java:80) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.doProcess(InterceptingChainLifecycleWrapper.java:50) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.access$001(InterceptingChainLifecycleWrapper.java:22) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper$1.process(InterceptingChainLifecycleWrapper.java:66) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.process(InterceptingChainLifecycleWrapper.java:61) ~[?:?]
at org.mule.construct.AbstractPipeline$3.process(AbstractPipeline.java:231) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.chain.SimpleMessageProcessorChain.doProcess(SimpleMessageProcessorChain.java:43) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.doProcess(InterceptingChainLifecycleWrapper.java:50) ~[?:?]
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:76) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.access$001(InterceptingChainLifecycleWrapper.java:22) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper$1.process(InterceptingChainLifecycleWrapper.java:66) ~[?:?]
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24) ~[?:?]
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:107) ~[?:?]
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44) ~[?:?]
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.process(InterceptingChainLifecycleWrapper.java:61) ~[?:?]
at org.mule.transport.AbstractMessageReceiver.routeEvent(AbstractMessageReceiver.java:511) ~[?:?]
at org.mule.transport.AbstractMessageReceiver.routeMessage(AbstractMessageReceiver.java:226) ~[?:?]
at org.mule.transport.AbstractMessageReceiver.routeMessage(AbstractMessageReceiver.java:208) ~[?:?]
at org.mule.transport.AbstractReceiverWorker$1$1.process(AbstractReceiverWorker.java:145) ~[?:?]
at org.mule.transport.AbstractReceiverWorker$1$1.process(AbstractReceiverWorker.java:113) ~[?:?]
at org.mule.execution.ExecuteCallbackInterceptor.execute(ExecuteCallbackInterceptor.java:16) ~[?:?]
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:35) ~[?:?]
at org.mule.execution.CommitTransactionInterceptor.execute(CommitTransactionInterceptor.java:22) ~[?:?]
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:30) ~[?:?]
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:14) ~[?:?]
at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:67) ~[?:?]
at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:44) ~[?:?]
at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:50) ~[?:?]
at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:40) ~[?:?]
at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:41) ~[?:?]
at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:48) ~[?:?]
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:28) ~[?:?]
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:13) ~[?:?]
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:110) ~[?:?]
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:30) ~[?:?]
at org.mule.transport.AbstractReceiverWorker$1.process(AbstractReceiverWorker.java:112) ~[?:?]
at org.mule.transport.AbstractReceiverWorker$1.process(AbstractReceiverWorker.java:96) ~[?:?]
at org.mule.execution.ExecuteCallbackInterceptor.execute(ExecuteCallbackInterceptor.java:16) ~[?:?]
at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:67) ~[?:?]
at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:44) ~[?:?]
at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:50) ~[?:?]
at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:40) ~[?:?]
at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:41) ~[?:?]
at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:48) ~[?:?]
at org.mule.execution.TransactionalExecutionTemplate.execute(TransactionalExecutionTemplate.java:65) ~[?:?]
at org.mule.transport.AbstractReceiverWorker.processMessages(AbstractReceiverWorker.java:179) ~[?:?]
at org.mule.transport.jms.MultiConsumerJmsMessageReceiver$SubReceiver.onMessage(MultiConsumerJmsMessageReceiver.java:439) ~[?:?]
at org.apache.activemq.ActiveMQMessageConsumer.dispatch(ActiveMQMessageConsumer.java:1394) ~[?:?]
at org.apache.activemq.ActiveMQSessionExecutor.dispatch(ActiveMQSessionExecutor.java:131) ~[?:?]
at org.apache.activemq.ActiveMQSessionExecutor.iterate(ActiveMQSessionExecutor.java:202) ~[?:?]
at org.apache.activemq.thread.PooledTaskRunner.runTask(PooledTaskRunner.java:133) ~[?:?]
at org.apache.activemq.thread.PooledTaskRunner$1.run(PooledTaskRunner.java:48) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_73]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_73]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_73]
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:170) ~[?:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:304) ~[?:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.prepareStatement(SQLServerConnection.java:1863) ~[?:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.prepareStatement(SQLServerConnection.java:1566) ~[?:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_73]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_73]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_73]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_73]
at com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask.run(GooGooStatementCache.java:546) ~[?:?]
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696) ~[?:?]
OK.
If you are using Statement caching, c3p0 asynchronously acquires Statements for the cache if there is not an appropriate cached Statement already available for the Connection. Under heavy load and without careful configuration, this asynchronous acquire could take a long time due to congestion at the server or congestion of c3p0's thread pool. Clients trying to acquire a Statement wait indefinitely until the Statement is successfully acquired or an Exception has occurred.
From the root cause Exception (way down towards the bottom of your stack trace), it's clear that, while the potentially long Statement acquisition was occurring, the Connection from which the Statement was to be acquired was close()ed or died. That's what you need to debug to really understand this. Possible causes include:
Under these conditions, load/network problems/mysterious gremlins occasionally corrupt the Connections leaving them invalid and close()ed
The DBMS or some administrative watcher close()es these (perhaps long-time hung) Connections, because too many Connections are open or they have been open too long. (Perhaps this is the work of a deadlock detector.)
Your application (ill-advisedly) uses the Connections in multiple Threads, one of which calls close() while another is still wait()ing on a Statement. The pool then (for any of a number of reasons) tries close()es the checked-in Connection leading to this error.
If (3) is the case you should stop doing that, but I think that's the least likely scenario. If (1) or (2), the main way you can mitigate the issue is to make sure that Statement acquisition is fast, so that bad things are unlikely happen while we're wait()ing for a Statement.
The first thing I'd look at is c3p0's Thread pool under the load conditions you are subjecting your application to. Is it keeping up? (You can check this via JMX, or via calls to PooledDataSource methods like getThreadPoolNumTasksPending() Or, the very easiest thing to do would be simply to scale up c3p0 config param numHelperThreads, so it's less likely that tasks like Statement acquisition are backlogged.
Another configuration change that could really help would be to ensure that you are allowing a sufficient number of cached Statements to prevent churning (constantly close()ing and reacquiring Connections in the Statement cache). A very frequent configuration mistake is to set the global maxStatements config param to a too low value. The easy way to get this right is to use maxStatementsPerConnection, rather than a global limit, and set this to approximately the number of PreparedStatements frequently used by your application.
To really understand what's going on, you'll want to see what sort of tasks are getting backlogged. Use PooledDataSource.sampleThreadPoolStatus(), again directly in code or via JMX.
Update: In the original post, you state "prepared statement cache size kept it as default". That may be a mule default, but c3p0's default is not to use Statement caching at all, maxStatements=0 and maxStatementsPerConnection=0. Your configuration may not be what you think it is. Disabling Statement caching might be a very easy way to remedy this issue.
PROBLEM :
If you are using ojdbc8 on Oracle 12c 12.2.0.0.0 and later and NLS_CHARACTERSET WE8ISO8859P15 with a prepared statement that returns auto-generated keys, it will generated the following error :
Caused by: java.sql.SQLException: A problem occurred while trying to acquire a cached PreparedStatement in a background thread.
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.acquireStatement(GooGooStatementCache.java:571)
at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkoutStatement(GooGooStatementCache.java:204)
at com.mchange.v2.c3p0.impl.NewPooledConnection.checkoutStatement(NewPooledConnection.java:321)
at com.mchange.v2.c3p0.impl.NewProxyConnection.prepareStatement(NewProxyConnection.java:431)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$3.doPrepare(StatementPreparerImpl.java:117)
at org.hibernate.engine.jdbc.internal.StatementPreparerImpl$StatementPreparationTemplate.prepareStatement(StatementPreparerImpl.java:172)
... 29 more
Caused by: java.sql.SQLException: Invalid argument(s) in call
at oracle.jdbc.driver.AutoKeyInfo.getNewSql(AutoKeyInfo.java:187)
at oracle.jdbc.driver.PhysicalConnection.prepareStatement(PhysicalConnection.java:4342)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.mchange.v2.c3p0.stmt.GooGooStatementCache$1StmtAcquireTask.run(GooGooStatementCache.java:547)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
SOLUTION :
A database patch is available to fix the issue.
Otherwhise you can change the caracther set to WE8ISO8859P1
REFERENCE
https://support.oracle.com/knowledge/Middleware/2278611_1.html
I had this problem with hsqldb (2.4.1) and hibernate (5.3.1) while saving (inserting) the first object.
Creating the tables (DDL) worked but the save (SQL prepared statement) failed.
I had not applied a naming strategy and hibernate apparently created SQL where the case of the table name did not match what hsqldb expected.
Hsqldb is case sensitive, and while one would expect a clear error message, the error above was reported.
The issue was fixed by adding the JPA annotation #Table(name="your_table").