servicemix 4.3.2 camel cxf producer to cxf-nmr consumer Not Working - apache-servicemix

I've been trying to get two separate bundles to communicate via a nmr endpoint using a camel cxf producer. I have been working on this all week and so far I can't get it to work.
I can however all the nmr endpoint just as in example cxf-camel-nmr without issue so I know that bundle is ok. Seriously this is a super simple hello world prototype and I can't get it to work.
Any help or suggestions would be awesome!!
Env: Java 1.5
Apache Servicemix 4.3.2 (virgin installation)
In servicemix I have two separate osgi bundles. I am unable to get my camel cxf producer to successfully call the nmr endpoint and have zero clue on how to fix.
Bundle #1 cxf-nmr //consumer
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:jaxws="http://cxf.apache.org/jaxws"
xsi:schemaLocation="
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://cxf.apache.org/jaxws http://cxf.apache.org/schemas/jaxws.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
<import resource="classpath:META-INF/cxf/transport/nmr/cxf-transport-nmr.xml" />
<jaxws:endpoint id="helloWorld1"
implementor="org.apache.servicemix.examples.cxf.HelloWorldImpl"
address="nmr:HelloWorld1" />
</beans>
Bundle #2 //producer
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:osgi="http://www.springframework.org/schema/osgi"
xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:cxf="http://camel.apache.org/schema/cxf"
xsi:schemaLocation="
http://camel.apache.org/schema/cxf http://camel.apache.org/schema/cxf/camel-cxf.xsd
http://servicemix.apache.org/http/1.0 http://servicemix.apache.org/http/1.0/servicemix-http.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://www.springframework.org/schema/osgi http://www.springframework.org/schema/osgi/spring-osgi.xsd
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-extension-soap.xml" />
<import resource="classpath:META-INF/cxf/transport/nmr/cxf-transport-nmr.xml" />
<import resource="classpath:org/apache/servicemix/camel/nmr/camel-nmr.xml" />
<import resource="classpath:META-INF/cxf/cxf-extension-http.xml"/>
<import resource="classpath:META-INF/cxf/cxf-extension-http-jetty.xml" />
<camel:camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="cxf:bean:testEndpoint"/>
<to uri="nmr:HelloWorld1"/>
</route>
</camel:camelContext>
<cxf:cxfEndpoint
id="testEndpoint"
address="http://localhost:9090/test"
serviceClass="org.apache.servicemix.examples.cxf.HelloWorld">
<cxf:properties>
<entry key="dataFormat" value="PAYLOAD" />
</cxf:properties>
</cxf:cxfEndpoint>
</beans>
--
#################
soap request response
#################
<?xml version="1.0"?>
<SOAP-ENV:Envelope
xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/">
<SOAP-ENV:Body>
<m:sayHi xmlns:m="http://cxf.examples.servicemix.apache.org/">
<arg0>foobar</arg0>
</m:sayHi>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>
STATUS: 500
<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"><soap:Body><soap:Fault><faultcode>soap:Server</faultcode><faultstring>java.lang.NullPointerException</faultstring></soap:Fault></soap:Body></soap:Envelope>
--
####################
Console Output - No Idea How To Fix
####################
18:56:04,544 | ERROR | oWorld1-thread-1 | NMRDestination | - - | error preparing message
java.lang.NullPointerException
at org.apache.cxf.staxutils.StaxUtils.createXMLStreamReader(StaxUtils.java:1183)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:427)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:380)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.servicemix.cxf.transport.nmr.NMRMessageHelper.convertMessageToInputStream(NMRMessageHelper.java:43)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:127)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.nmr.core.InternalEndpointWrapper.process(InternalEndpointWrapper.java:86)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl.process(ChannelImpl.java:255)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl$1.run(ChannelImpl.java:215)[83:org.apache.servicemix.nmr.core:1.4.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)[:1.5.0_22]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)[:1.5.0_22]
at java.lang.Thread.run(Thread.java:595)[:1.5.0_22]
18:56:04,547 | WARN | oWorld1-thread-1 | NMR | 83 - org.apache.servicemix.nmr.core - 1.4.0 | Error processing exchange [
id: a778d16c-2955-41de-89e8-7775f4f976bf
mep: InOut
status: Active
role: Provider
target: PropertyMatchingReference[{NAME=HelloWorld1}]
properties: [
jaxwsContext = {javax.xml.ws.wsdl.operation={http://cxf.examples.servicemix.apache.org/}sayHi, javax.xml.ws.wsdl.description=http://localhost:9090/test?wsdl, javax.xml.ws.wsdl.service={http://cxf.examples.servicemix.apache.org/}HelloWorldService, javax.xml.ws.wsdl.interface={http://cxf.examples.servicemix.apache.org/}HelloWorld, javax.xml.ws.wsdl.port={http://cxf.examples.servicemix.apache.org/}HelloWorldPort}
CamelToEndpoint = nmr://HelloWorld1
org.apache.cxf.service.model.BindingOperationInfo = [BindingOperationInfo: {http://cxf.examples.servicemix.apache.org/}sayHi]
mtom-enabled = false
CamelCreatedTimestamp = Thu Jun 09 18:56:04 PDT 2011
CamelCXFDataFormat = PAYLOAD
]
]
org.apache.servicemix.nmr.api.ServiceMixException: java.lang.NullPointerException
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:151)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.nmr.core.InternalEndpointWrapper.process(InternalEndpointWrapper.java:86)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl.process(ChannelImpl.java:255)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl$1.run(ChannelImpl.java:215)[83:org.apache.servicemix.nmr.core:1.4.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)[:1.5.0_22]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)[:1.5.0_22]
at java.lang.Thread.run(Thread.java:595)[:1.5.0_22]
Caused by: java.lang.NullPointerException
at org.apache.cxf.staxutils.StaxUtils.createXMLStreamReader(StaxUtils.java:1183)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:427)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:380)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.servicemix.cxf.transport.nmr.NMRMessageHelper.convertMessageToInputStream(NMRMessageHelper.java:43)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:127)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
... 6 more
18:56:04,552 | ERROR | f2d4607-thread-1 | DefaultErrorHandler | 68 - org.apache.camel.camel-core - 2.6.0 | Failed delivery for exchangeId: ID-Dana-PC-45880-1307670949817-0-2. Exhausted after delivery attempt: 1 caught: org.apache.servicemix.nmr.api.ServiceMixException: java.lang.NullPointerException
org.apache.servicemix.nmr.api.ServiceMixException: java.lang.NullPointerException
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:151)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.nmr.core.InternalEndpointWrapper.process(InternalEndpointWrapper.java:86)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl.process(ChannelImpl.java:255)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl$1.run(ChannelImpl.java:215)[83:org.apache.servicemix.nmr.core:1.4.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)[:1.5.0_22]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)[:1.5.0_22]
at java.lang.Thread.run(Thread.java:595)[:1.5.0_22]
Caused by: java.lang.NullPointerException
at org.apache.cxf.staxutils.StaxUtils.createXMLStreamReader(StaxUtils.java:1183)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:427)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:380)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.servicemix.cxf.transport.nmr.NMRMessageHelper.convertMessageToInputStream(NMRMessageHelper.java:43)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:127)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
... 6 more
18:56:04,553 | WARN | qtp407691725-242 | PhaseInterceptorChain | - - | Interceptor for {http://cxf.examples.servicemix.apache.org/}HelloWorldService#{http://cxf.examples.servicemix.apache.org/}sayHi has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: java.lang.NullPointerException
at org.apache.camel.component.cxf.CxfConsumer$1.checkFailure(CxfConsumer.java:219)[118:org.apache.camel.camel-cxf:2.6.0]
at org.apache.camel.component.cxf.CxfConsumer$1.setResponseBack(CxfConsumer.java:196)[118:org.apache.camel.camel-cxf:2.6.0]
at org.apache.camel.component.cxf.CxfConsumer$1.asyncInvoke(CxfConsumer.java:116)[118:org.apache.camel.camel-cxf:2.6.0]
at org.apache.camel.component.cxf.CxfConsumer$1.invoke(CxfConsumer.java:73)[118:org.apache.camel.camel-cxf:2.6.0]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:58)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:93)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:255)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.phase.PhaseInterceptorChain.resume(PhaseInterceptorChain.java:224)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:70)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.serviceRequest(JettyHTTPDestination.java:311)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.transport.http_jetty.JettyHTTPDestination.doService(JettyHTTPDestination.java:280)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.transport.http_jetty.JettyHTTPHandler.handle(JettyHTTPHandler.java:72)[116:org.apache.cxf.bundle:2.3.2]
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:868)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:185)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:114)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.Server.handleAsync(Server.java:397)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:601)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:410)[149:org.eclipse.jetty.server:7.2.2.v20101205]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:508)[144:org.eclipse.jetty.io:7.2.2.v20101205]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.access$000(SelectChannelEndPoint.java:34)[144:org.eclipse.jetty.io:7.2.2.v20101205]
at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:40)[144:org.eclipse.jetty.io:7.2.2.v20101205]
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:451)[143:org.eclipse.jetty.util:7.2.2.v20101205]
at java.lang.Thread.run(Thread.java:595)[:1.5.0_22]
Caused by: org.apache.servicemix.nmr.api.ServiceMixException: java.lang.NullPointerException
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:151)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.nmr.core.InternalEndpointWrapper.process(InternalEndpointWrapper.java:86)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl.process(ChannelImpl.java:255)[83:org.apache.servicemix.nmr.core:1.4.0]
at org.apache.servicemix.nmr.core.ChannelImpl$1.run(ChannelImpl.java:215)[83:org.apache.servicemix.nmr.core:1.4.0]
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:651)[:1.5.0_22]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:676)[:1.5.0_22]
... 1 more
Caused by: java.lang.NullPointerException
at org.apache.cxf.staxutils.StaxUtils.createXMLStreamReader(StaxUtils.java:1183)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:427)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:380)[116:org.apache.cxf.bundle:2.3.2]
at org.apache.servicemix.cxf.transport.nmr.NMRMessageHelper.convertMessageToInputStream(NMRMessageHelper.java:43)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
at org.apache.servicemix.cxf.transport.nmr.NMRDestination.process(NMRDestination.java:127)[203:org.apache.servicemix.cxf.transport.nmr:4.3.0]
... 6 more

It looks like I have been dealing with exactly same issue for the last two days. I believe I have by now solved it. Here's what I found out:
The NPE in StaxUtils seems to be due to a source (javax.xml.transform.Source) being null. it can be null when exchange.getIn().getBody(Source.class) returns null in the NMRDestination class. Here's where things get a little ugly. Let's make a step back.
To see whether your message had a body when it came in you would need to turn on TRACE log level. You can do that in servicemix console with log:set TRACE command. ExchangeUtils#display(Exchange) only gets called with the option to print out body in TRACE, not even in DEBUG. With TRACE on you will see things like:
id: 1520804564-51998-1331328769981-0-3
mep: InOut
status: Active
role: Consumer
target: ...
properties: [ ]
In: [
content:
]
the In.content section is essentially the body of the message. I am sure your exchange had a body just like mine did. Which means it was "lost" somewhere within the NMR/Camel routing schema.
The exchange.getIn().getBody(Source.class) goes through a special conversion logic that may silently (!) return null if something doesn't click. Take a look at org.apache.camel.impl.converter.BaseTypeConverterRegistry#doConvertTo().
When I was reading TRACE logs I noticed another subtle (almost an FYI) trace message about org.apache.xml.serializer.ToXMLStream not being visible to my bundle. I figured NMR was failing to convert the message to the Source object and was telling me nothing about it.
SOLUTION: I ended up adding org.apache.xml.serializer to my Import-Package maven instruction along with org.apache.xalan (once I added the serializer import the next thing that blew up was ClassNotFountException complaining about org.apache.xalan.xsltc.trax.DOM2SAX). Everything is there in runtime, it's all in lib/endorsed, it's just wasn't visible to my service bundle until now.
It now works and my message is at the destination endpoint. One more thing to pay attention to is the streamCache attribute of the Camel content. There's something about stream resources not being cached by default in latest versions of Camel and your code won't be able to read from a stream multiple times unless it's cached.
The question that remains is why do I have to make the service "aware" of those dependencies NMR/Camel needs to send my message down the route. Doesn't really make sense.

Related

Websphere + Spring DMLC + MQ -J2CA0045E: Connection not available while invoking method createOrWaitForConnection for resource JMS

We are using Spring in our Web application.
I want to concurrently listen the messages from MQ using using Spring's DefaultMessageListenerContainer(DMLC).
In websphere I didnt change any connection configurations.
Issue:
When DMLC listner tried to receive few(6 messages) messages concurrently from Queue, Websphere Application Server logged FFDC incident logs.
Can anyone help me If I missing the connection configurations?
Version:
Server: Websphere Application Server 8.5
MessageListener:
<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="connectionFactory"/>
<property name="destinationName" ref="queue"/>
<property name="messageListener" ref="myMessageListener"/>
<property name ="concurrentConsumers" value ="5"/>
<property name ="maxConcurrentConsumers" value ="20"/>
<property name="idleTaskExecutionLimit" value="10" />
<property name="idleConsumerLimit" value="5" />
<property name="recoveryInterval" value="60000" />
</bean>
QueueConnectionfactory - Connection pools:
Logs in Server logs:
[8/1/17 12:47:54:071 ] FreePool E J2CA0045E: Connection not available while invoking method createOrWaitForConnection for resource JMS$ConnectionFactory$JMSManagedConnection#0.
[8/1/17 12:43:44:131 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt Max connections reached 869
[8/1/17 12:43:44:131 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt com.ibm.ejs.jms.JMSConnectionHandle.createSession 1051
[8/1/17 12:44:44:132 ] SystemOut O 12:44:44.132 [jmsContainer-11] WARN o.s.j.l.DefaultMessageListenerContainer - Setup of JMS message listener invoker failed for destination 'queue:' - trying to recover. Cause: Failed to create session; nested exception is com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: CWTE_NORMAL_J2CA1009
[8/1/17 12:44:45:153 ] FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on WebSphere/path/path/logFileName.txt com.ibm.ejs.jms.JMSMessageConsumerHandle.receive 264
[8/1/17 12:44:55:304 ] SystemOut O 12:44:55.304 [jmsContainer-11] INFO o.s.j.l.DefaultMessageListenerContainer - Successfully refreshed JMS Connection
[8/1/17 12:47:55:310 ] FreePool E J2CA0045E: Connection not available while invoking method createOrWaitForConnection for resource JMS$ConnectionFactory$JMSManagedConnection#0
FFDC log incident:
[8/1/17 13:41:03:046] FFDC Exception:com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException SourceId:Max connections reached ProbeId:869 Reporter:com.ibm.ejs.j2c.PoolManager#aad4897
com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: CWTE_NORMAL_J2CA1009
at com.ibm.ejs.j2c.FreePool.createOrWaitForConnection(FreePool.java:1782)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3816)
at com.ibm.ejs.j2c.PoolManager.reserve(PoolManager.java:3092)
at com.ibm.ejs.j2c.ConnectionManager.allocateMCWrapper(ConnectionManager.java:1548)
at com.ibm.ejs.j2c.ConnectionManager.allocateConnection(ConnectionManager.java:1031)
at com.ibm.ejs.jms.JMSConnectionHandle.createSession(JMSConnectionHandle.java:1434)
at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
at org.springframework.jms.listener.DefaultMessageListenerContainer.access$12(DefaultMessageListenerContainer.java:1)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.initResourcesIfNecessary(DefaultMessageListenerContainer.java:1091)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1070)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
==> Performing default dump from com.ibm.ejs.j2c.DiagnosticModuleForJ2C :Tue Aug 01 13:41:03 2017
Maximum number of connections has been reached, and the connection request has been waiting longer than:ConnectionWaitTime. Two possible solutions : increase the max number of connections, or increase the:ConnectionWaitTime.:
Maximum Connections = :10
Current number of connections = :10
Connection Wait Timout = :180
If the current number of connections is not greater than or equal to max connections, :there has been a WebSphere internal error.:
This = BEGIN:com.ibm.ejs.j2c.PoolManager#aad4897
com.ibm.ejs.j2c.PoolManager::tc BEGIN:com.ibm.ejs.ras.TraceComponent#330b9221
com.ibm.ejs.ras.TraceElement::ivLevel:10
com.ibm.ejs.ras.TraceElement::ivName:com.ibm.ejs.j2c.PoolManager
com.ibm.ejs.ras.TraceElement::ivDebugEnabled:false
com.ibm.ejs.ras.TraceElement::ivEventEnabled:false
com.ibm.ejs.ras.TraceElement::ivEntryEnabled:false
com.ibm.ejs.ras.TraceElement::ivDetailEnabled:false
com.ibm.ejs.ras.TraceElement::ivConfigEnabled:false
com.ibm.ejs.ras.TraceElement::ivInfoEnabled:true
com.ibm.ejs.ras.TraceElement::ivServiceEnabled:true
com.ibm.ejs.ras.TraceElement::ivWarningEnabled:true
com.ibm.ejs.ras.TraceElement::ivErrorEnabled:true
com.ibm.ejs.ras.TraceElement::ivFatalEnabled:true
com.ibm.ejs.ras.TraceComponent::defaultMessageFile:com.ibm.ejs.resources.seriousMessages
com.ibm.ejs.ras.TraceComponent::EXTENSION_NAME_DPID:DiagnosticProvider
com.ibm.ejs.ras.TraceComponent::ivDumpEnabled:false
com.ibm.ejs.ras.TraceComponent::ivResourceBundleName:com.ibm.ws.j2c.resources.J2CAMessages
com.ibm.ejs.ras.TraceComponent::ivLogger:com.ibm.ws.logging.WsLogger#8273a12c
com.ibm.ejs.ras.TraceComponent::ivDiagnosticProviderID:null
com.ibm.ejs.ras.TraceComponent::anyTracingEnabled:null
END:com.ibm.ejs.ras.TraceComponent#330b9221
[8/1/17 13:42:03:108 ] FFDC Exception:com.ibm.msg.client.jms.DetailedIllegalStateException SourceId:com.ibm.ejs.jms.JMSMessageConsumerHandle.receive ProbeId:264 Reporter:com.ibm.ejs.jms.JMSMessageConsumerHandle#ab2dfd8d
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSCC0032: This message consumer is closed.
An application called a method that must not be used after the message consumer is closed.
Ensure that the message consumer is not closed before calling the method.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:86)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:58)
at java.lang.reflect.Constructor.newInstance(Constructor.java:542)
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:314)
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:228)
at com.ibm.msg.client.jms.internal.JmsErrorUtils.createException(JmsErrorUtils.java:104)
at com.ibm.msg.client.jms.internal.State.checkNotClosed(State.java:145)
at com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.receive(JmsMessageConsumerImpl.java:413)
at com.ibm.mq.jms.MQMessageConsumer.receive(MQMessageConsumer.java:212)
at com.ibm.ejs.jms.JMSMessageConsumerHandle.receive(JMSMessageConsumerHandle.java:553)
at com.ibm.ejs.jms.JMSMessageConsumerHandle.receive(JMSMessageConsumerHandle.java:501)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveMessage(AbstractPollingMessageListenerContainer.java:431)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:311)
at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:264)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1071)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
==> Performing default dump from com.ibm.ejs.jms.JMSDiagnosticModule :Tue Aug 01 13:42:03 2017
+Data for directive [defaultdefault] obtained.:
==> Dump complete for com.ibm.ejs.jms.JMSDiagnosticModule :Tue Aug 01 13:42:03 CEST 201
[8/1/17 13:50:13:445] FFDC Exception:javax.jms.IllegalStateException SourceId:com.ibm.ejs.jms.JMSConnectionHandle.createSession ProbeId:1044 Reporter:com.ibm.ejs.jms.JMSConnectionHandle#e8e11802
javax.jms.IllegalStateException: Connection closed
at com.ibm.ejs.jms.JMSConnectionHandle.checkOpen(JMSConnectionHandle.java:837)
at com.ibm.ejs.jms.JMSConnectionHandle.createSession(JMSConnectionHandle.java:1430)
at org.springframework.jms.support.JmsAccessor.createSession(JmsAccessor.java:196)
at org.springframework.jms.listener.DefaultMessageListenerContainer.access$12(DefaultMessageListenerContainer.java:1)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.initResourcesIfNecessary(DefaultMessageListenerContainer.java:1091)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1070)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1063)
at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:960)
at java.lang.Thread.run(Thread.java:863)
8/2/17 15:30:31:874 CEST] FFDC Exception:javax.jms.IllegalStateException SourceId:com.ibm.ejs.jms.JMSConnectionHandle.stop ProbeId:477 Reporter:com.ibm.ejs.jms.JMSConnectionHandle#5c5db824
javax.jms.IllegalStateException: Method stop not permitted
at com.ibm.ejs.jms.JMSCMUtils.methodNotPermittedException(JMSCMUtils.java:244)
at com.ibm.ejs.jms.JMSConnectionHandle.checkRestrictedMethod(JMSConnectionHandle.java:854)
at com.ibm.ejs.jms.JMSConnectionHandle.stop(JMSConnectionHandle.java:768)
at org.springframework.jms.listener.AbstractJmsListeningContainer.stopSharedConnection(AbstractJmsListeningContainer.java:460)
at org.springframework.jms.listener.DefaultMessageListenerContainer.stopSharedConnection(DefaultMessageListenerContainer.java:791)
at org.springframework.jms.listener.AbstractJmsListeningContainer.doStop(AbstractJmsListeningContainer.java:326)
at org.springframework.jms.listener.AbstractJmsListeningContainer.stop(AbstractJmsListeningContainer.java:302)
at org.springframework.jms.listener.DefaultMessageListenerContainer.stop(DefaultMessageListenerContainer.java:582)
at org.springframework.context.support.DefaultLifecycleProcessor.doStop(DefaultLifecycleProcessor.java:223)
at org.springframework.context.support.DefaultLifecycleProcessor.access$2(DefaultLifecycleProcessor.java:207)
at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.stop(DefaultLifecycleProcessor.java:356)
at org.springframework.context.support.DefaultLifecycleProcessor.stopBeans(DefaultLifecycleProcessor.java:196)
at org.springframework.context.support.DefaultLifecycleProcessor.onClose(DefaultLifecycleProcessor.java:113)
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1045)
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:1000)
at com.bnppf.ssc.imb.sf.fwk.mia.context.CloseableGenericWebApplicationContext.close(CloseableGenericWebApplicationContext.java:34)
at org.springframework.web.context.ContextLoader.closeWebApplicationContext(ContextLoader.java:548)
at org.springframework.web.context.ContextLoaderListener.contextDestroyed(ContextLoaderListener.java:142)
at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextDestroyed(WebApp.java:1765)
at com.ibm.ws.webcontainer.webapp.WebApp.destroy(WebApp.java:3131)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.destroy(WebAppImpl.java:1551)
at com.ibm.ws.container.AbstractContainer.destroy(AbstractContainer.java:75)
at com.ibm.ws.webcontainer.webapp.WebGroup.destroy(WebGroup.java:228)
at com.ibm.ws.webcontainer.webapp.WebGroup.removeWebApplication(WebGroup.java:269)
at com.ibm.ws.webcontainer.VirtualHost.removeWebApplication(VirtualHost.java:296)
at com.ibm.ws.webcontainer.VirtualHostImpl.removeWebApplication(VirtualHostImpl.java:211)
at com.ibm.ws.webcontainer.WSWebContainer.removeWebApplication(WSWebContainer.java:820)
at com.ibm.ws.webcontainer.component.WebContainerImpl.uninstall(WebContainerImpl.java:458)
at com.ibm.ws.webcontainer.component.WebContainerImpl.stop(WebContainerImpl.java:729)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:1218)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStop(DeployedApplicationImpl.java:1375)
at com.ibm.ws.runtime.component.DeployedModuleImpl.stop(DeployedModuleImpl.java:671)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.stop(DeployedApplicationImpl.java:1149)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplication(ApplicationMgrImpl.java:952)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplicationImpl(ApplicationMgrImpl.java:1488)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stop(ApplicationMgrImpl.java:2249)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stop(CompositionUnitMgrImpl.java:492)
at com.ibm.ws.runtime.component.CompositionUnitImpl.stop(CompositionUnitImpl.java:141)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$2.run(CompositionUnitMgrImpl.java:795)
at com.ibm.ws.security.auth.ContextManagerImpl.runAs(ContextManagerImpl.java:5387)
at com.ibm.ws.security.auth.ContextManagerImpl.runAsSystem(ContextManagerImpl.java:5603)
at com.ibm.ws.security.core.SecurityContext.runAsSystem(SecurityContext.java:255)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stopCompositionUnit(CompositionUnitMgrImpl.java:830)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.stopCompositionUnit(CompositionUnitMgrImpl.java:736)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.stopApplication(ApplicationMgrImpl.java:1465)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:95)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:88)
at sun.reflect.GeneratedMethodAccessor133.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:56)
at java.lang.reflect.Method.invoke(Method.java:620)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:292)
at javax.management.modelmbean.RequiredModelMBean$4.run(RequiredModelMBean.java:1261)
at java.security.AccessController.doPrivileged(AccessController.java:311)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:88)
at javax.management.modelmbean.RequiredModelMBean.invokeMethod(RequiredModelMBean.java:1255)
at javax.management.modelmbean.RequiredModelMBean.invoke(RequiredModelMBean.java:1093)
at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:832)
at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:814)
at com.ibm.ws.management.AdminServiceImpl$1.run(AdminServiceImpl.java:1335)
at com.ibm.ws.security.util.AccessController.doPrivileged(AccessController.java:118)
at com.ibm.ws.management.AdminServiceImpl.invoke(AdminServiceImpl.java:1228)
at com.ibm.ws.management.connector.AdminServiceDelegator.invoke(AdminServiceDelegator.java:181)
at com.ibm.ws.management.connector.ipc.CallRouter.route(CallRouter.java:247)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink.doWork(IPCConnectorInboundLink.java:360)
at com.ibm.ws.management.connector.ipc.IPCConnectorInboundLink$IPCConnectorReadCallback.complete(IPCConnectorInboundLink.java:602)
at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1881)
Looks like your current Spring configuration is draining the whole connection pool. So either configuration is not correct or you are leaking connections somewhere.
==> Performing default dump from com.ibm.ejs.j2c.DiagnosticModuleForJ2C :Tue Aug 01 13:41:03 2017
Maximum number of connections has been reached, and the connection request has been waiting longer than:ConnectionWaitTime. Two possible solutions : increase the max number of connections, or increase the:ConnectionWaitTime.:
Maximum Connections = :10
Current number of connections = :10
Connection Wait Timout = :180
so either fix the configuration - maybe start with the simpler one and remove:
<property name ="maxConcurrentConsumers" value ="20"/>
<property name="idleTaskExecutionLimit" value="10" />
<property name="idleConsumerLimit" value="5" />
or just use normal Java EE MDB for reciving messages instead of this Spring.

Exception ORA-31011: XML parsing failed

I am calling a SOAP based webservice using PL/SQL block. The response returned is in the following format:
<env:Envelope xmlns:env="http://www.w3.org/2003/05/soap-envelope">
<env:Header/>
<env:Body>
<ns0:getDocumentDataResponse xmlns:ns0="http://xmlns.oracle.com/oxp/service/ScheduleReportService">
<ns3:getDocumentData ="http://xmlns.oracle.com/oxp/service/ScheduleReportService">JVBERi0xLjYNCjUgMCB vYmoNCjw8DQovVHlwZSAvWE9iamVjdA0KL1N1YnR5cGUgL0ltYWdlDQov
RmlsdGVyIC9GbGF0ZURlY29kZQ0KL0xlbmd0aCA1MzUNCi9XaWR0aCA5Mg0KL0hlaWdodCAxNQ0K
L0JpdHNQZXJDb21wb25lbnQgOA0KL0NvbG9yU3BhY2UgL0RldmljZVJHQg0KPj4NCnN0cmVhbQ0K
eJztVzuOAjEM5U65yJwip6DgGnOhrai5AhLNLh0SRfZ5nDiexPFktS1PFgqMYz//kiGlD47xeDzS
9WoKPdJ4/4w0s/g40nm9Xo5xeUqLGbhsszWT5LKkU0inky0hpBjrXixGmqI/ivp8yTqmAvivK20f
WT5fqECOBYXn80nuHGvwxe4aU/hRx2JKE6nkxNfvOetkhkCcFahkVJqxZTx9/9CuiZxQ6jTzXmLM
....
OWExMDllPjxhMWE2NTE2ZDNmYTcwY2Y2NzhkY2EwMWIzZjlhMTA5ZT5dDQo+Pg0Kc3RhcnR4cmVm
DQoxMDg0MTIwDQolJUVPRg0K</ns3:getDocumentData>
</ns0:getDocumentDataResponse>
</env:Body>
</env:Envelope>
This response is stored in the variable : l_xml_response_text(varchar2) and when I execute the following code:
l_xml_response_xmltype := xmltype(l_xml_response_text);
it Is throwing the error:
Exception ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00007: unexpected end-of-file encountered
Is the issue due to data returned in decoded format? If yes..how can this be handled?

Coherence Exception in NamedCache.putall after updating coherence to 12.1.3

I use coherence in my project with following configuration.
Everything is Ok with coherence 12.1.2, but after updating coherence to version 12.1.3, i have a problem. When i put for example 1000 item on cache one by one using NamedCache.put() there is no error, but when i put 1000 items on cache with calling NamedCache.putall once, coherence raises an exception.
My project is in java and is deployed on jboss.
cache configuration:
<distributed-scheme>
<scheme-name>my-map</scheme-name>
<service-name>MyMap</service-name>
<serializer>
<instance>
<class-name>com.tangosol.io.pof.ConfigurablePofContext</class-name>
<init-params>
<init-param>
<param-type>String</param-type>
<param-value>pof-config.xml</param-value>
</init-param>
</init-params>
</instance>
</serializer>
<thread-count>100</thread-count>
<local-storage>true</local-storage>
<backup-count>0</backup-count>
<backing-map-scheme>
<read-write-backing-map-scheme>
<internal-cache-scheme>
<ramjournal-scheme/>
</internal-cache-scheme>
<cachestore-scheme>
<class-scheme>
<class-name>myPackage.loader</class-name>
</class-scheme>
</cachestore-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
<autostart>true</autostart>
</distributed-scheme>
Raised exception details:
Exception in thread "pool-7-thread-1" com.tangosol.net.RequestIncompleteException: Partial failure
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$BinaryMap.validatePartialResponse(PartitionedCache.CDB:56)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$BinaryMap.putAll(PartitionedCache.CDB:123)
at com.oracle.common.collections.ConverterCollections$ConverterMap.putAll(ConverterCollections.java:1553)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$ViewMap.putAll(PartitionedCache.CDB:5)
at com.tangosol.coherence.component.util.SafeNamedCache.putAll(SafeNamedCache.CDB:1)
at mypackage.CacheEnabledDataProvider.putOnCache(CacheEnabledDataProvider.java:66)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: Portable(com.tangosol.util.WrapperException): (Wrapped: Failed request execution for myMap service on Member(Id=1, Timestamp=2014-10-20 18:00:25.737, Address=192.168.70.101:8088, MachineId=5642, Location=site:,machine:PS,process:9876,member:Mem_1, Role=CoherenceServer) (Wrapped: Failed to store keys="9112791815, 9165497418, 9199193873, 9192139970, 9392020128, ") Assertion failed:) Assertion failed:
at com.tangosol.util.Base.ensureRuntimeException(Base.java:289)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.tagException(Grid.CDB:50)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onPartialCommit(PartitionedCache.CDB:7)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onPutAllRequest(PartitionedCache.CDB:85)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PutAllRequest$PutJob.run(PartitionedCache.CDB:1)
at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:65)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:51)
at java.lang.Thread.run(Unknown Source)
at <process boundary>
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.ConfigurablePofContext.deserialize(ConfigurablePofContext.java:376)
at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.readObject(Service.CDB:1)
at com.tangosol.coherence.component.net.Message.readObject(Message.CDB:1)
at com.tangosol.coherence.component.net.message.responseMessage.DistributedPartialResponse.read(DistributedPartialResponse.CDB:12)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PartialValueResponse.read(PartitionedCache.CDB:4)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.deserializeMessage(Grid.CDB:20)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:21)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:51)
at java.lang.Thread.run(Thread.java:745)
Caused by: Portable(com.tangosol.util.AssertionException): Assertion failed:
at com.tangosol.util.Base.azzertFailed(Base.java:209)
at com.tangosol.util.Base.azzert(Base.java:166)
at com.tangosol.net.cache.ReadWriteBackingMap$CacheStoreWrapper.storeAllInternal(ReadWriteBackingMap.java:5943)
at com.tangosol.net.cache.ReadWriteBackingMap$StoreWrapper.storeAll(ReadWriteBackingMap.java:5067)
at com.tangosol.net.cache.ReadWriteBackingMap.putAll(ReadWriteBackingMap.java:840)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.putAllPrimaryResource(PartitionedCache.CDB:7)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.postPutAll(PartitionedCache.CDB:27)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$Storage.putAll(PartitionedCache.CDB:14)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onPutAllRequest(PartitionedCache.CDB:62)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PutAllRequest$PutJob.run(PartitionedCache.CDB:1)
at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:1)
at com.tangosol.coherence.component.util.DaemonPool$WrapperTask.run(DaemonPool.CDB:32)
at com.tangosol.coherence.component.util.DaemonPool$Daemon.onNotify(DaemonPool.CDB:65)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:51)
at java.lang.Thread.run(Unknown Source)
at <process boundary>
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:57)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.PortableException.readExternal(PortableException.java:150)
at com.tangosol.io.pof.ThrowablePofSerializer.deserialize(ThrowablePofSerializer.java:59)
at com.tangosol.io.pof.PofBufferReader.readAsObject(PofBufferReader.java:3316)
at com.tangosol.io.pof.PofBufferReader.readObject(PofBufferReader.java:2604)
at com.tangosol.io.pof.ConfigurablePofContext.deserialize(ConfigurablePofContext.java:376)
at com.tangosol.coherence.component.util.daemon.queueProcessor.Service.readObject(Service.CDB:1)
at com.tangosol.coherence.component.net.Message.readObject(Message.CDB:1)
at com.tangosol.coherence.component.net.message.responseMessage.DistributedPartialResponse.read(DistributedPartialResponse.CDB:12)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache$PartialValueResponse.read(PartitionedCache.CDB:4)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.deserializeMessage(Grid.CDB:20)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.Grid.onNotify(Grid.CDB:21)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.PartitionedService.onNotify(PartitionedService.CDB:3)
at com.tangosol.coherence.component.util.daemon.queueProcessor.service.grid.partitionedService.PartitionedCache.onNotify(PartitionedCache.CDB:3)
at com.tangosol.coherence.component.util.Daemon.run(Daemon.CDB:51)
at java.lang.Thread.run(Thread.java:745)
I found that if loader class defined in cachestore-scheme element, implements CacheStore instead of CacheLoader and leave store() and storeAll() empty, no exception is raised. According to oracle documents, implementing CacheStore is not necessary for read-only cache-store and implementing CacheLoader is sufficient. This might be a bug.
I am not sure implementing CacheStore with empty Store() and storeAll() methods when we need read-only cache-store has performance penalty or not?
I would only implement CacheLoader (not CacheStore) if you are not writing back to the database.

Unable to deploy category from BCC ATG Oracle Web Commerce

I'm trying to deploy an existing category and I'm getting the below error. I understand the problem, but I can't understand what could be causing this error. Any pointers ?
11:36:40,646 ERROR [ProductCatalog_production] SQL Statement Failed: [++SQLInsert++]
INSERT INTO dcs_cat_chldcat(child_cat_id,category_id)
VALUES(?,?)
-- Parameters --
p[1] = {pd} 1215046631963 (java.lang.String)
p[2] = {pd: fixedParentCategories} 1215045135527 (atg.adapter.gsa.SingleValueGSAId)
[--SQLInsert--]
11:36:40,663 ERROR [DeploymentManager] item = repositoryMarker:mark17800005 cause = CONTAINER:atg.deployment.DistributedDeploymentException; SOURCE:CONTAINER:atg.repository.RepositoryException; SOURCE:java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("ATGCATALOGA"."DCS_CAT_CHLDCAT"."SEQUENCE_NUM")
at atg.deployment.repository.RepositoryWorkerThread.processMarkerForReferenceUpdatePhase(RepositoryWorkerThread.java:374)
at atg.deployment.DeploymentWorkerThread.processMarkerPhase(DeploymentWorkerThread.java:568)
at atg.deployment.DeploymentWorkerThread.run(DeploymentWorkerThread.java:335)
message = Deployment Failed time = Thu Sep 25 11:36:40 IST 2014 atg.deployment.DeploymentFailure#21f7e76a
CAUGHT AT:
CONTAINER:atg.deployment.DistributedDeploymentException; SOURCE:CONTAINER:atg.repository.RepositoryException; SOURCE:java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("ATGCATALOGA"."DCS_CAT_CHLDCAT"."SEQUENCE_NUM")
at atg.deployment.repository.RepositoryWorkerThread.processMarkerForReferenceUpdatePhase(RepositoryWorkerThread.java:374)
at atg.deployment.DeploymentWorkerThread.processMarkerPhase(DeploymentWorkerThread.java:568)
at atg.deployment.DeploymentWorkerThread.run(DeploymentWorkerThread.java:335)
Caused by: CONTAINER:atg.repository.RepositoryException; SOURCE:java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("ATGCATALOGA"."DCS_CAT_CHLDCAT"."SEQUENCE_NUM")
at atg.adapter.gsa.GSAItemDescriptor.updateItem(GSAItemDescriptor.java:7644)
at atg.adapter.gsa.GSARepository.updateItem(GSARepository.java:1075)
at atg.deployment.repository.RepositoryWorkerThread.deployItem(RepositoryWorkerThread.java:1135)
at atg.deployment.repository.RepositoryWorkerThread.processMarkerForReferenceUpdatePhase(RepositoryWorkerThread.java:362)
... 2 more
Caused by: java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("ATGCATALOGA"."DCS_CAT_CHLDCAT"."SEQUENCE_NUM")
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:440)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:837)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:445)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:191)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:523)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1010)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1315)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3576)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3657)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1350)
at org.jboss.resource.adapter.jdbc.WrappedPreparedStatement.executeUpdate(WrappedPreparedStatement.java:365)
at atg.adapter.gsa.SQLStatement.executeUpdate(SQLStatement.java:857)
at atg.adapter.gsa.SetTable.insertRows(SetTable.java:463)
at atg.adapter.gsa.SetTable.update(SetTable.java:531)
at atg.adapter.gsa.GSAItemDescriptor.updateItem(GSAItemDescriptor.java:7633)
... 5 more
You cannot re-insert the same category, but you can update it.
Are you deploying through publishing? I mean through project publishing from BCC?
As the SQL Insert statement doesn't contain the sequence number, it would appear that the definition of the Versioned catalog repository and the Production catalog repository don't match. i.e. The versioned catalog repository doesn't have the multi-column-name defined, but the Production definition does.
Both definitions should look something like this.
<table xml:id="catChildCats" name="dcs_cat_chldcat" type="multi" id-column-name="category_id"
multi-column-name="sequence_num" shared-table-sequence="1">
<property category-resource="categoryCategorization" name="fixedChildCategories" data-type="list"
component-item-type="category"
column-name="child_cat_id" queryable="true" display-name-resource="fixedChildCategories">
<attribute name="propertySortPriority" value="-9"/>
<attribute name="references" value="true"/>
<attribute name="prohibitDuplicates" value="true"/>
</property>
</table>

c3p0 deadlock on RefurbishCheckinResourceTask, getConnection()

I've a webApp deployed on tomcat6, there's an import message function, which will import a large amount of messages persisting them into an Oracle 11g database.
I'm getting the deadlock occasionally/unpredictably, after running import for a while.
The trace:
7 août 2014 13:04:38 com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
ATTENTION: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#637d7dd6 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
7 août 2014 13:04:38 com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
ATTENTION: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#637d7dd6 -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 8
Active Threads: 8
Active Tasks:
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#55e4ef4e (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#45460ed4 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#3)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#49da1926 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#7)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#14bc659d (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#78597c06 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#4)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#6667f88e (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#5)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#2a5d8dbb (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#6)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#441364ad (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
Pending Tasks:
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#7ca35537
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#1370b3e5
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#9d7dc72
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#14bfd04d
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#3d0479df
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#27310c3a
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#519c7f0
Pool thread stack traces:
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#3,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#7,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#4,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#5,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#6,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:562)
7 août 2014 13:05:13 com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
ATTENTION: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#637d7dd6 -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
7 août 2014 13:05:13 com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector run
ATTENTION: com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#637d7dd6 -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 8
Active Threads: 8
Active Tasks:
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#62ac4df7 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#e8fdc6f (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#4140546 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#3)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#70a25e6b (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#7)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#33e4fc92 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#4a6064a4 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#4)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#466c6e60 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#6)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#27e87047 (com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#5)
Pending Tasks:
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#bdba1dc
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#7be684ff
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#b9b2923
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#292c79d8
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#26f36906
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#114d8aa
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#7bf6e678
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#5c447da5
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#14950af6
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#3124219d
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask#7a92974e
Pool thread stack traces:
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,main]
org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.getAndSendCurrentStackTrace(ProfilerRuntimeMemory.java:221)
org.netbeans.lib.profiler.server.ProfilerRuntimeObjLiveness.traceObjAlloc(ProfilerRuntimeObjLiveness.java:281)
com.mchange.v2.c3p0.impl.NewProxyConnection.<init>(NewProxyConnection.java:32)
com.mchange.v2.c3p0.impl.NewProxyConnection.<init>(NewProxyConnection.java:1376)
com.mchange.v2.c3p0.impl.NewPooledConnection.getConnection(NewPooledConnection.java:151)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishResourceOnCheckin(C3P0PooledConnectionPool.java:285)
com.mchange.v2.resourcepool.BasicResourcePool.attemptRefurbishResourceOnCheckin(BasicResourcePool.java:1615)
com.mchange.v2.resourcepool.BasicResourcePool.access$200(BasicResourcePool.java:32)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1228)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,main]
org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.getAndSendCurrentStackTrace(ProfilerRuntimeMemory.java:221)
org.netbeans.lib.profiler.server.ProfilerRuntimeObjLiveness.traceObjAlloc(ProfilerRuntimeObjLiveness.java:281)
com.mchange.v2.c3p0.impl.NewProxyConnection.<init>(NewProxyConnection.java:32)
com.mchange.v2.c3p0.impl.NewProxyConnection.<init>(NewProxyConnection.java:1376)
com.mchange.v2.c3p0.impl.NewPooledConnection.getConnection(NewPooledConnection.java:151)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishResourceOnCheckin(C3P0PooledConnectionPool.java:285)
com.mchange.v2.resourcepool.BasicResourcePool.attemptRefurbishResourceOnCheckin(BasicResourcePool.java:1615)
com.mchange.v2.resourcepool.BasicResourcePool.access$200(BasicResourcePool.java:32)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1228)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#3,5,main]
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1229)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#7,5,main]
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1229)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,main]
org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.getAndSendCurrentStackTrace(ProfilerRuntimeMemory.java:221)
org.netbeans.lib.profiler.server.ProfilerRuntimeObjLiveness.traceObjAlloc(ProfilerRuntimeObjLiveness.java:281)
java.util.HashMap.addEntry(Unknown Source)
java.util.HashMap.put(Unknown Source)
java.util.HashSet.add(Unknown Source)
com.mchange.v2.c3p0.util.ConnectionEventSupport.addConnectionEventListener(ConnectionEventSupport.java:39)
com.mchange.v2.c3p0.impl.NewPooledConnection.addConnectionEventListener(NewPooledConnection.java:194)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishResourceOnCheckin(C3P0PooledConnectionPool.java:290)
com.mchange.v2.resourcepool.BasicResourcePool.attemptRefurbishResourceOnCheckin(BasicResourcePool.java:1615)
com.mchange.v2.resourcepool.BasicResourcePool.access$200(BasicResourcePool.java:32)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1228)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#4,5,main]
org.netbeans.lib.profiler.server.ProfilerRuntimeMemory.getAndSendCurrentStackTrace(ProfilerRuntimeMemory.java:221)
org.netbeans.lib.profiler.server.ProfilerRuntimeObjLiveness.traceObjAlloc(ProfilerRuntimeObjLiveness.java:281)
java.util.HashMap.clone(Unknown Source)
java.util.HashSet.clone(Unknown Source)
com.mchange.v2.c3p0.util.ConnectionEventSupport.fireConnectionClosed(ConnectionEventSupport.java:49)
com.mchange.v2.c3p0.impl.NewPooledConnection.fireConnectionClosed(NewPooledConnection.java:439)
com.mchange.v2.c3p0.impl.NewPooledConnection.markClosedProxyConnection(NewPooledConnection.java:318)
com.mchange.v2.c3p0.impl.NewProxyConnection.close(NewProxyConnection.java:1246)
com.mchange.v1.db.sql.ConnectionUtils.attemptClose(ConnectionUtils.java:41)
com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.refurbishResourceOnCheckin(C3P0PooledConnectionPool.java:292)
com.mchange.v2.resourcepool.BasicResourcePool.attemptRefurbishResourceOnCheckin(BasicResourcePool.java:1615)
com.mchange.v2.resourcepool.BasicResourcePool.access$200(BasicResourcePool.java:32)
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1228)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#6,5,main]
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1229)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#5,5,main]
com.mchange.v2.resourcepool.BasicResourcePool$1RefurbishCheckinResourceTask.run(BasicResourcePool.java:1229)
com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:557)
It seems that there are plenty of RefurbishCheckinResourceTask, and according to the trace, c3p0 fails to getConnection. I'm using Spring, hibernate, c3p0.
My c3p0 configuration:
<bean id="pmiDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close">
<property name="driverClass" value="oracle.jdbc.driver.OracleDriver" />
<property name="jdbcUrl" value="jdbc:oracle:thin:********" />
<property name="user" value="****" />
<property name="password" value="****" />
<property name="acquireIncrement" value="5" />
<property name="numHelperThreads" value="6" />
<property name="minPoolSize" value="10" />
<property name="initialPoolSize" value="15" />
<property name="maxPoolSize" value="25" />
</bean>
My hibernate properties:
<property name="hibernateProperties">
<value>
hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
hibernate.auto_close_session=true
hibernate.validator.autoregister_listeners=false
hibernate.validator.apply_to_ddl=false
hibernate.cache.use_second_level_cache=true
hibernate.cache.region.factory_class=net.sf.ehcache.hibernate.EhCacheRegionFactory
hibernate.cache.use_query_cache=false
javax.persistence.validation.mode=none
hibernate.generate_statistics=true
</value>
</property>
I've checked a lot of forum about this, one of the most similar case was this.
c3p0 apparent deadlock / creating emergency threads.
APPARENT DEADLOCK 0.9.1.2
Could some help please?
It's not a c3p0 problem, in fact its my own fault.
I used a TaskExecutor to run a multi-treaded inner runnalbe task. Since this procedure makes call from inner instance to outer class method, it has to deal with the synchronization problem, which I ignored in the first place.
When the inner instances run into some error due to synchronization, it will block some c3p0 connection from returning back to the connection-pool.
So this turned out to be an outer-inner-class instance method call problem. Just need to be careful with synchronization issues.
Sometimes the error won't show on a development machine, but once it's published on a more powerful production machine with many cores, the error can easily happen.

Resources