I have one application WAR. I have created two profiles AppSrv01, AppSrv02 on Websphere 8.0.0.0.
My application is having MQ connectivity, J2ee application on struts 1.1.
I have deployed WAR on both the server AppSrv01 as a context A and AppSrv02 as a context B.
I have session listener in my application which will take note of total available session.
The scenario is as given below....
Going and login to url localhost:9082/A, go to some page in it.
Going and login to url localhost:9082/B, go to some page in it in another tab of same browser.
Going to url localhost:9082/A, clicking on some of link or url from the existing page.
on remote debug from server when it come to SessionListener ... it shows below given stack
Daemon Thread [WebContainer : 10] (Suspended (breakpoint at line 24 in SessionCounterListener))
SessionCounterListener.sessionCreated(HttpSessionEvent) line: 24
WasHttpSessionObserver(HttpSessionObserver).sessionCreated(ISession) line: 111
SessionEventDispatcher.sessionCreated(ISession) line: 98
SessionManager.createISession(String, int, boolean) line: 268
SessionManager.createSession(ServletRequest, ServletResponse, SessionAffinityContext, boolean) line: 640
WsSessionContext(SessionContext).getIHttpSession(HttpServletRequest, HttpServletResponse, boolean, boolean) line: 485
WsSessionContext(SessionContext).getIHttpSession(HttpServletRequest, HttpServletResponse, boolean) line: 419
SRTRequestContext.getSession(boolean, WebApp) line: 104
SRTServletRequest.getSession(boolean) line: 2099
SRTServletRequest.getSession() line: 2083
RequestProcessor.processLocale(HttpServletRequest, HttpServletResponse) line: 631
RequestProcessor.process(HttpServletRequest, HttpServletResponse) line: 230
ActionServlet.process(HttpServletRequest, HttpServletResponse) line: 1482
ActionServlet.doPost(HttpServletRequest, HttpServletResponse) line: 525
ActionServlet(HttpServlet).service(HttpServletRequest, HttpServletResponse) line: 595
ActionServlet(HttpServlet).service(ServletRequest, ServletResponse) line: 668
ServletWrapperImpl(ServletWrapper).service(ServletRequest, ServletResponse, WebAppServletInvocationEvent) line: 1147
ServletWrapperImpl(ServletWrapper).handleRequest(ServletRequest, ServletResponse, WebAppDispatcherContext) line: 722
ServletWrapperImpl(ServletWrapper).handleRequest(ServletRequest, ServletResponse) line: 449
ServletWrapperImpl.handleRequest(ServletRequest, ServletResponse) line: 178
WebAppFilterManagerImpl(WebAppFilterManager).invokeFilters(ServletRequest, ServletResponse, IServletContext, RequestProcessor, EnumSet<CollaboratorInvocationEnum>) line: 1020
CacheServletWrapper.handleRequest(ServletRequest, ServletResponse) line: 87
WSWebContainer(WebContainer).handleRequest(IRequest, IResponse) line: 883
WSWebContainer.handleRequest(IRequest, IResponse) line: 1659
WCChannelLink.ready(VirtualConnection) line: 195
HttpInboundLink.handleDiscrimination() line: 452
HttpInboundLink.handleNewRequest() line: 511
HttpInboundLink.processRequest() line: 305
HttpInboundLink.ready(VirtualConnection) line: 276
NewConnectionInitialReadCallback.sendToDiscriminators(VirtualConnection, TCPReadRequestContext, boolean) line: 214
NewConnectionInitialReadCallback.complete(VirtualConnection, TCPReadRequestContext) line: 113
AioReadCompletionListener.futureCompleted(IAbstractAsyncFuture, Object) line: 165
AsyncFuture(AbstractAsyncFuture).invokeCallback(ICompletionListener, AbstractAsyncFuture, Object) line: 217
AsyncFuture(AsyncChannelFuture).fireCompletionActions() line: 161
AsyncFuture.completed(long) line: 138
ResultHandler.complete(AsyncFuture, int, int) line: 204
ResultHandler.runEventProcessingLoop(boolean) line: 775
ResultHandler$2.run() line: 905
ThreadPool$Worker.run() line: 1648
It is creating another session instance. need to understand why it is creating another instance.
When you open the tabs in IE, for IE its same session. So you will have only one cookie JSESSIONID per domain.
when you go to localhost:9082/A - you have JSESSIONID like SESSIONID-A.
when you go to localhost:9082/B - you have JSESSIONID like SESSIONID-B.
So when you access application on AppSrv01, JSESSIONID goes to AppSrv01 is SESSIONID-B. Appserver doesn't know about this session id, so it thinks its a new user and creates a session.
Try opening a IE-> File -> New Session and access the application on AppSrv02 you won't face the above issue.
If you have something like IEWATCH or Fidler, you will be able to see the JSESSIONID going from client/browser.
There's also a setting HttpSessionIdReuse that might help with this as well. I know it works for multiple Servers in the same Cell, but I'm not sure whether it works across Profiles.
http://www-01.ibm.com/support/docview.wss?uid=swg21210881
Related
I am executing the scale on socket.io using artillery using the steps mentioned in the blog - https://www.artillery.io/blog/load-testing-socketio-with-artillery with following modification (adding more users)
config:
target: "http://localhost:3000"
phases:
- duration: 60
arrivalRate: 5
- duration: 60
arrivalRate: 5
rampTo: 50
I am executing the test against the code available at https://github.com/socketio/socket.io/tree/main/examples/chat
Here is the findings
engine.socketio.emit: .......................................................... 1877
engine.socketio.emit_rate: ..................................................... 2/sec
errors.ECONNREFUSED: ........................................................... 111
errors.Error: xhr poll error: .................................................. 591
errors.Error: xhr post error: .................................................. 388
vusers.completed: .............................................................. 860
vusers.created: ................................................................ 1950
vusers.created_by_name.A chatty user: .......................................... 188
vusers.created_by_name.A mostly quiet user: .................................... 282
vusers.created_by_name.A user that just lurks: ................................. 1480
vusers.failed: ................................................................. 1090
vusers.session_length:
min: ......................................................................... 60007.8
max: ......................................................................... 174740.5
median: ...................................................................... 64236
p95: ......................................................................... 161191.7
p99: ......................................................................... 171159.6
Any idea, why it is failing for so many users?
Has anyone already found this issue at capturing traffic of snapchat?
Every https data deriving from other sites via application (of ios, android) are captured successfully by Fiddler but a few apps (appstore, snapchat) don't show anything just that request:
CONNECT app.snapchat.com:443 HTTP/1.1
Host: app.snapchat.com
User-Agent: Snapchat/10.8.1.0 (iPhone8,1; iOS 10.2.1; gzip)
Connection: keep-alive
Connection: keep-alive
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.3 (TLS/1.2)
Random: 59 23 9E E1 1C 23 49 F1 A1 21 6E 60 C5 94 AB E2 9F 09 10 C3 E0 C3 99 9B 78 9B 97 1F 74 69 5F 1C
"Time": 2089.12.12. 15:48:57
SessionID: empty
Extensions:
server_name app.snapchat.com
elliptic_curves secp256r1 [0x17], secp384r1 [0x18], secp521r1 [0x19]
ec_point_formats uncompressed [0x0]
signature_algs sha256_rsa, sha1_rsa, sha384_rsa, sha512_rsa, sha256_ecdsa, sha1_ecdsa, sha384_ecdsa, sha512_ecdsa
NextProtocolNego empty
ALPN http/1.1, http/1.0
status_request OCSP - Implicit Responder
SignedCertTimestamp (RFC6962) empty
extended_master_secret empty
Ciphers:
[00FF] TLS_EMPTY_RENEGOTIATION_INFO_SCSV
[C02C] TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
[C02B] TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
[C024] TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
[C023] TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
[C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C030] TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[C02F] TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[C028] TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
[C027] TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
[C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA
[C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA
Compression:
[00] NO_COMPRESSION
What would be done for the unimpeded working?
Extending this issue more what's the reason the fiddler (or other interceptors) isn't able to capture all https data but their Connect handshakes?
Probably this is due to certificate pinning. Snapchat is know to use this to prevent MITM attacks.
Take a look at this answer here:
https://stackoverflow.com/a/40543302/1353689
and the links related to it as well.
We are facing an issue when using Spring JTA + Hibernate in a multiple EAR context.
We have multiple transactional services (using #Transactional).
Our services are POJOs that have a reference to a DAO.
We also have multiple persistence units. For sake of simplicity, lets assume that each service/dao uses a distinct PU.
If Service1 in EAR1 starts a transaction and calls service2 in EAR2 multiple times then :
Calls to Service2 will create a new hibernate session per call.
All the sessions will be flushed/committed at the end of the transactions (i.e. when the Service1 calls is done and commits the transaction). -
Important note : Spring and Hibernate are not loaded by the root classloader, but by the EAR classloaders.
This implies that Spring and Hibernate classes are loaded once per EAR.
Looking into the code and debugging led us to believe that the issue comes from thread local synchronization (i.e. the TransactionSynchronizationManager class).
What happens is:
A transaction is started by Service1 in EAR1
Service1 calls Service2 in EAR2
Because Service2 is also transacted the TransactionAspectSupport kicks in around the Service2 method that is being called.
Down the road the EntityManagerFactoryUtils will check if an EntityManager is registered with TransactionSynchronizationManager. This happens when SharedEntityManagerCreator calls EntityManagerFactoryUtils.doGetTransactionalEntityManager(). See call stack 1 below.
Because no EntityManager is registered, it will create a new EntityManager and a new Hibernate Session. See call stack 2 below.
When returning from the Service2 call, the synchronization is removed because it is considered as a new one. This happens in TransactionAspectSupport.commitTransactionAfterReturning(). See call stack 3 below.
Subsequent calls to Service2 will go through the process of creating a new entity manager + session + synchronization again.
Because all the sessions are also registered with the global transaction, they all get flushed/committed at the end of said transaction.
A side effect, is that if the calls to Service2 modify the same entity, then that entity gets stored multiple times to the RDBMS (i.e. multiple identical update queries are fired).
The same scenario with both Service1 and Service2 deployed in the same EAR yields completely different results:
Only one hibernate session is created
That unique session is flushed at the end of the transaction (and a single update query is fired)
In some of our scenarios, we are seeing hundreds of sessions being created over a short period of time and this is causing performance issues.
It is possible that moving Spring and Hibernate to the root ClassLoader could fix the issue but it does not feel right.
What if the the two EARs are deployed in separate JVMs ?
Moving spring and hibernate up in the ClassLoader hierarchy would also badly break our spring context setup...
Is this the expected behavior of Spring Transactions ?
Is it possible to avoid the thread local synchronization and use transaction synchronization ?
Is our setup (see below) wrong with regard to what we are trying to achieve ?
We are using :
Spring version : 3.2.8.RELEASE
Hibernate version : 4.1.12.Final
Here are some relevant fragments of our Spring setup :
<tx:jta-transaction-manager />
<bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"
abstract="true">
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="database" value="${jpa.database}" />
<property name="generateDdl" value="${jpa.generateDdl}" />
<property name="showSql" value="true" />
</bean>
</property>
<property name="dataSource" ref="coreDataSource" />
<property name="jpaProperties">
<props>
<prop key="hibernate.dialect">${hibernate.dialect}</prop>
<prop key="hibernate.transaction.jta.platform">${jpa.transaction.jta.platform}</prop>
<prop key="hibernate.jdbc.wrap_result_sets">${jpa.wrap_result_sets}</prop>
<prop key="hibernate.generate_statistics">${jpa.generateStatistics}</prop>
<prop key="hibernate.cache.region.factory_class">${jpa.cache.factory_class}</prop>
<prop key="hibernate.cache.use_second_level_cache">true</prop>
<prop key="hibernate.cache.use_query_cache">true</prop>
<prop key="hibernate.format_sql">${jpa.formatSql}</prop>
<prop key="hibernate.use_sql_comments">${jpa.useSqlComments}</prop>
</props>
</property>
</bean>
Where jpa.transaction.jta.platform = org.hibernate.service.jta.platform.internal.WeblogicJtaPlatform
<bean id="referentialCoreEntityManagerFactory" parent="entityManagerFactory">
<property name="persistenceXmlLocation" value="classpath:META-INF/referential-core-persistence.xml" />
<property name="persistenceUnitName" value="referential-core" />
</bean>
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="referential-core" transaction-type="JTA" />
</persistence>
Base class of our DAOs showing how the entity managers are defined:
public abstract class BaseDAO<E extends IBaseEntity> extends com.sungard.decalog.framework.domain.dao.jpa.BaseDAO<E> {
private EntityManager entityManager;
/**
* #param entityClass
*/
public <U extends E> BaseDAO(Class<U> entityClass) {
super(entityClass);
}
#Override
protected final EntityManager getEntityManager() {
return entityManager;
}
#PersistenceContext(unitName = "referential-core")
public void setEntityManager(EntityManager entityManager) {
this.entityManager = entityManager;
}
}
Call stack 1 : Creation of new EntityManager and Session:
Daemon Thread [[ACTIVE] ExecuteThread: '15' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended (breakpoint at line 239 in SessionImpl))
-> SessionImpl.<init>(Connection, SessionFactoryImpl, SessionOwner, TransactionCoordinatorImpl, boolean, long, Interceptor, boolean, boolean, ConnectionReleaseMode, String) line: 239
SessionFactoryImpl$SessionBuilderImpl.openSession() line: 1597
EntityManagerImpl.getRawSession() line: 121
EntityManagerImpl.getSession() line: 97
EntityManagerImpl(AbstractEntityManagerImpl).joinTransaction(boolean) line: 1207
EntityManagerImpl(AbstractEntityManagerImpl).postInit() line: 178
-> EntityManagerImpl.<init>(EntityManagerFactoryImpl, PersistenceContextType, PersistenceUnitTransactionType, boolean, Class, Map) line: 89
EntityManagerFactoryImpl.createEntityManager(Map) line: 179
EntityManagerFactoryImpl.createEntityManager() line: 174
GeneratedMethodAccessor450.invoke(Object, Object[]) line: not available
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
LocalContainerEntityManagerFactoryBean(AbstractEntityManagerFactoryBean).invokeProxyMethod(Method, Object[]) line: 376
AbstractEntityManagerFactoryBean$ManagedEntityManagerFactoryInvocationHandler.invoke(Object, Method, Object[]) line: 519
$Proxy116.createEntityManager() line: not available
-> EntityManagerFactoryUtils.doGetTransactionalEntityManager(EntityManagerFactory, Map) line: 202
SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(Object, Method, Object[]) line: 211
$Proxy118.find(Class, Object) line: not available
GenericLoaderCodifDAO(BaseDAO<E>).findById(Serializable) line: 57
GenericLoaderCodifService.findGenericLoaderCodif(String, String, String) line: 18
ReferentialFacade.findGenericLoaderCodif(String, String, String) line: 161
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 317
ReflectiveMethodInvocation.invokeJoinpoint() line: 183
ReflectiveMethodInvocation.proceed() line: 150
TransactionInterceptor$1.proceedWithInvocation() line: 96
TransactionInterceptor(TransactionAspectSupport).invokeWithinTransaction(Method, Class, TransactionAspectSupport$InvocationCallback) line: 260
TransactionInterceptor.invoke(MethodInvocation) line: 94
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy124.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_Impl(ReferentialFacadeBean).findGenericLoaderCodif(String, String, String) line: 271
ReferentialFacadeBean_590diy_EOImpl.__WL_invoke(Object, Object[], int) line: not available
SessionRemoteMethodInvoker.invoke(BaseRemoteObject, MethodDescriptor, Object[], int, String, Class<?>) line: 40
ReferentialFacadeBean_590diy_EOImpl.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_EOImpl_WLSkel.invoke(int, Object[], Object) line: not available
ServerRequest.sendReceive() line: 174
ClusterableRemoteRef.invoke(RemoteReference, RuntimeMethodDescriptor, Object[], Method) line: 345
ClusterableRemoteRef.invoke(Remote, RuntimeMethodDescriptor, Object[], Method) line: 259
ReferentialFacadeBean_590diy_EOImpl_1036_WLStub.findGenericLoaderCodif(String, String, String) line: not available
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
RmiClientInterceptorUtils.invokeRemoteMethod(MethodInvocation, Object) line: 116
SimpleRemoteStatelessSessionProxyFactoryBean(SimpleRemoteSlsbInvokerInterceptor).doInvoke(MethodInvocation) line: 99
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractRemoteSlsbInvokerInterceptor).invokeInContext(MethodInvocation) line: 141
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractSlsbInvokerInterceptor).invoke(MethodInvocation) line: 189
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy314.findGenericLoaderCodif(String, String, String) line: not available
ReferentialService.findGenericLoaderCodif(String, String, String) line: 54
SessionService.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: 1114
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 317
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 198
$Proxy357.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: not available
SessionFacadeService.loadSession(ISessionContext, ISessionSnapshot, ISessionData, ReloadSession) line: 880
SessionFacadeService.loadSession(long, ReloadSession) line: 848
...
<snip>
Call stack 2 : Registration of newly created EntityManager synchronization :
Daemon Thread [[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended)
TransactionSynchronizationManager.registerSynchronization(TransactionSynchronization) line: 289
EntityManagerFactoryUtils.doGetTransactionalEntityManager(EntityManagerFactory, Map) line: 211
SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(Object, Method, Object[]) line: 211
$Proxy118.find(Class, Object) line: not available
GenericLoaderCodifDAO(BaseDAO<E>).findById(Serializable) line: 57
GenericLoaderCodifService.findGenericLoaderCodif(String, String, String) line: 18
ReferentialFacade.findGenericLoaderCodif(String, String, String) line: 161
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 317
ReflectiveMethodInvocation.invokeJoinpoint() line: 183
ReflectiveMethodInvocation.proceed() line: 150
TransactionInterceptor$1.proceedWithInvocation() line: 96
TransactionInterceptor(TransactionAspectSupport).invokeWithinTransaction(Method, Class, TransactionAspectSupport$InvocationCallback) line: 260
TransactionInterceptor.invoke(MethodInvocation) line: 94
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy124.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_Impl(ReferentialFacadeBean).findGenericLoaderCodif(String, String, String) line: 271
ReferentialFacadeBean_590diy_EOImpl.__WL_invoke(Object, Object[], int) line: not available
SessionRemoteMethodInvoker.invoke(BaseRemoteObject, MethodDescriptor, Object[], int, String, Class<?>) line: 40
ReferentialFacadeBean_590diy_EOImpl.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_EOImpl_WLSkel.invoke(int, Object[], Object) line: not available
ServerRequest.sendReceive() line: 174
ClusterableRemoteRef.invoke(RemoteReference, RuntimeMethodDescriptor, Object[], Method) line: 345
ClusterableRemoteRef.invoke(Remote, RuntimeMethodDescriptor, Object[], Method) line: 259
ReferentialFacadeBean_590diy_EOImpl_1036_WLStub.findGenericLoaderCodif(String, String, String) line: not available
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
RmiClientInterceptorUtils.invokeRemoteMethod(MethodInvocation, Object) line: 116
SimpleRemoteStatelessSessionProxyFactoryBean(SimpleRemoteSlsbInvokerInterceptor).doInvoke(MethodInvocation) line: 99
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractRemoteSlsbInvokerInterceptor).invokeInContext(MethodInvocation) line: 141
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractSlsbInvokerInterceptor).invoke(MethodInvocation) line: 189
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy314.findGenericLoaderCodif(String, String, String) line: not available
ReferentialService.findGenericLoaderCodif(String, String, String) line: 54
SessionService.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: 1114
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 317
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 198
$Proxy357.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: not available
SessionFacadeService.loadSession(ISessionContext, ISessionSnapshot, ISessionData, ReloadSession) line: 880
SessionFacadeService.loadSession(long, ReloadSession) line: 848
...
<snip>
Call stack 3 : Clearing of EntityManager synchronization (this will cause a new one to be created for each Service2 call) :
Daemon Thread [[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended)
TransactionSynchronizationManager.clearSynchronization() line: 328
TransactionSynchronizationManager.clear() line: 464
WebLogicJtaTransactionManager(AbstractPlatformTransactionManager).cleanupAfterCompletion(DefaultTransactionStatus) line: 1006
WebLogicJtaTransactionManager(AbstractPlatformTransactionManager).processCommit(DefaultTransactionStatus) line: 805
WebLogicJtaTransactionManager(AbstractPlatformTransactionManager).commit(TransactionStatus) line: 724
TransactionInterceptor(TransactionAspectSupport).commitTransactionAfterReturning(TransactionAspectSupport$TransactionInfo) line: 475
TransactionInterceptor(TransactionAspectSupport).invokeWithinTransaction(Method, Class, TransactionAspectSupport$InvocationCallback) line: 270
TransactionInterceptor.invoke(MethodInvocation) line: 94
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy124.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_Impl(ReferentialFacadeBean).findGenericLoaderCodif(String, String, String) line: 271
ReferentialFacadeBean_590diy_EOImpl.__WL_invoke(Object, Object[], int) line: not available
SessionRemoteMethodInvoker.invoke(BaseRemoteObject, MethodDescriptor, Object[], int, String, Class<?>) line: 40
ReferentialFacadeBean_590diy_EOImpl.findGenericLoaderCodif(String, String, String) line: not available
ReferentialFacadeBean_590diy_EOImpl_WLSkel.invoke(int, Object[], Object) line: not available
ServerRequest.sendReceive() line: 174
ClusterableRemoteRef.invoke(RemoteReference, RuntimeMethodDescriptor, Object[], Method) line: 345
ClusterableRemoteRef.invoke(Remote, RuntimeMethodDescriptor, Object[], Method) line: 259
ReferentialFacadeBean_590diy_EOImpl_1036_WLStub.findGenericLoaderCodif(String, String, String) line: not available
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
RmiClientInterceptorUtils.invokeRemoteMethod(MethodInvocation, Object) line: 116
SimpleRemoteStatelessSessionProxyFactoryBean(SimpleRemoteSlsbInvokerInterceptor).doInvoke(MethodInvocation) line: 99
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractRemoteSlsbInvokerInterceptor).invokeInContext(MethodInvocation) line: 141
SimpleRemoteStatelessSessionProxyFactoryBean(AbstractSlsbInvokerInterceptor).invoke(MethodInvocation) line: 189
ReflectiveMethodInvocation.proceed() line: 172
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 204
$Proxy314.findGenericLoaderCodif(String, String, String) line: not available
ReferentialService.findGenericLoaderCodif(String, String, String) line: 54
SessionService.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: 1114
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
AopUtils.invokeJoinpointUsingReflection(Object, Method, Object[]) line: 317
JdkDynamicAopProxy.invoke(Object, Method, Object[]) line: 198
$Proxy357.refreshSession(ISessionContext, ISessionSnapshot, ISessionData) line: not available
SessionFacadeService.loadSession(ISessionContext, ISessionSnapshot, ISessionData, ReloadSession) line: 880
SessionFacadeService.loadSession(long, ReloadSession) line: 848
...
<snip>
Try to avoid the use of shared classloaders, this can give rise to all sorts of problems with static variables being shared accidentally between applications.
For example static loggers will be shared, so if you turn on trace debug in one application, you will see the other application slowdown as well, this is just an example of what the shared classloader can cause as unintended side effect.
Something similar could be happening not necessarily in Spring or Hibernate but in some transitive dependency that they use, most of these libraries are meant to be deployed at the application classloader level.
Currently there is no really good reason to put spring and hibernate at a shared classloader, they hardly consume any memory compared to other things.
I personally would put Spring and Hibernate in each WAR instead of moving it further down to the JVM, that would actually make the application un-redeployable, and would try to reduce the use of shared libraries to a minimum.
Also cross EAR calls usually are not needed, it's a programming model that is mostly abandoned due to complexity problems. The usual solution is to ship the service/DAO layer in a separate jar, and deploy the jar in the two applications. The application code can then call the service layer directly without the need for a call to another EAR.
The native transaction manager auto-detected by tx:jta-transaction-manager is then no longer needed and can be replaced by something like JpaTransactionManager or HibernateTransactionManager.
My website Joomla 3.2 website was running well recently, when I got this error message and things don't work anymore.
Notice: Undefined offset: 0 in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 479
Notice: Trying to get property of non-object in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 483
Warning: Creating default object from empty value in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 483
Notice: Undefined offset: 0 in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 479
Notice: Trying to get property of non-object in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 483
Warning: Creating default object from empty value in C:\Apache2\htdocs\ictogcio\libraries\cms\application\site.php on line 483
Error displaying the error page: Could not find template "".
have a batch process and a regular application that updates the same table.My batch have multiple threads that run on multiple sessions. I got the following errors in my batch Tomcat:
2012-09-10 11:30:17,043 [SyncDataThread567] ERROR org.springframework.batch.core.step.AbstractStep - Encountered an error executing the step
aaa.bbb.ccc.framework.orm.DAOException:
--- The error occurred in abc.xml.
--- The error occurred while applying a parameter map.
--- Check the ear.updateServiceTimeParamMap.
--- Check the statement (update procedure failed).
--- Cause: java.sql.SQLException: ORA-20011: FUNC_UPDATESERVICETIME : Error occured ORA-00060: deadlock detected while waiting for resource
ORA-06512: at "ER.FUNC_UPDATESERVICETIME", line 154
ORA-06512: at line 1
at aaa.bbb.ccc.ddd.eee.Sss.updateServiceTimes(ServiceOrderDAOImpl.java:76)
at sun.reflect.GeneratedMethodAccessor352.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy6.updateServiceTimes(Unknown Source)
at aaa.bbb.ccc.ddd.eeee.Inbddd.updateServiceTimes(InbDataWriter.java:144)
at aaa.bbb.ccc.ddd.eeee.Inbddd.write(InbDataWriter.java:74)
at sun.reflect.GeneratedMethodAccessor270.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy7.write(Unknown Source)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.writeItems(SimpleChunkProcessor.java:171)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.doWrite(SimpleChunkProcessor.java:150)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.write(SimpleChunkProcessor.java:268)
at org.springframework.batch.core.step.item.SimpleChunkProcessor.process(SimpleChunkProcessor.java:194)
at org.springframework.batch.core.step.item.ChunkOrientedTasklet.execute(ChunkOrientedTasklet.java:74)
at org.springframework.batch.core.step.tasklet.TaskletStep$ChunkTransactionCallback.doInTransaction(TaskletStep.java:386)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:128)
at org.springframework.batch.core.step.tasklet.TaskletStep$2.doInChunkContext(TaskletStep.java:264)
at org.springframework.batch.core.scope.context.StepContextRepeatCallback.doInIteration(StepContextRepeatCallback.java:76)
at org.springframework.batch.repeat.support.RepeatTemplate.getNextResult(RepeatTemplate.java:367)
at org.springframework.batch.repeat.support.RepeatTemplate.executeInternal(RepeatTemplate.java:214)
at org.springframework.batch.repeat.support.RepeatTemplate.iterate(RepeatTemplate.java:143)
at org.springframework.batch.core.step.tasklet.TaskletStep.doExecute(TaskletStep.java:250)
at org.springframework.batch.core.step.AbstractStep.execute(AbstractStep.java:195)
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:109)
at org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler$1.call(TaskExecutorPartitionHandler.java:107)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at org.springframework.core.task.SimpleAsyncTaskExecutor$ConcurrencyThrottlingRunnable.run(SimpleAsyncTaskExecutor.java:192)
at java.lang.Thread.run(Thread.java:619)
Caused by: com.ibatis.common.jdbc.exception.NestedSQLException:
--- The error occurred in ael.xml.
--- The error occurred while applying a parameter map.
--- Check the eraa.updateServiceTimeParamMap.
--- Check the statement (update procedure failed).
--- Cause: java.sql.SQLException: ORA-20011: FUNC_UPDATESERVICETIME : Error occured ORA-00060: deadlock detected while waiting for resource
ORA-06512: at "ER.FUNC_UPDATESERVICETIME", line 154
ORA-06512: at line 1
at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:201)
at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryForObject(MappedStatement.java:120)
at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForObject(SqlMapExecutorDelegate.java:518)
at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelegate.queryForObject(SqlMapExecutorDelegate.java:493)
at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.queryForObject(SqlMapSessionImpl.java:106)
at com.iit.integration.erl.orm.ServiceOrderDAOImpl.updateServiceTimes(ServiceOrderDAOImpl.java:71)
... 44 more
Caused by: java.sql.SQLException: ORA-20011: FUNC_UPDATESERVICETIME : Error occured ORA-00060: deadlock detected while waiting for resource
ORA-06512: at "ER.FUNC_IIT_UPDATESERVICETIME", line 154
ORA-06512: at line 1
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:112)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:331)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:288)
at oracle.jdbc.driver.T4C8Oall.receive(T4C8Oall.java:743)
at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:215)
at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:954)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1168)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3285)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3390)
at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4223)
at org.apache.tomcat.dbcp.dbcp.DelegatingPreparedStatement.execute(DelegatingPreparedStatement.java:169)
at com.ibatis.sqlmap.engine.execution.SqlExecutor.executeQueryProcedure(SqlExecutor.java:278)
at com.ibatis.sqlmap.engine.mapping.statement.ProcedureStatement.sqlExecuteQuery(ProcedureStatement.java:39)
at com.ibatis.sqlmap.engine.mapping.statement.MappedStatement.executeQueryWithCallback(MappedStatement.java:189)
... 49 more
This is my oracle trace file:
Redo thread mounted by this instance: 1
Oracle process number: 63
Windows thread id: 2464, image: ORACLE.EXE (SHAD)
*** 2012-09-10 11:30:12.384
*** SERVICE NAME:(SYS$USERS) 2012-09-10 11:30:12.244
*** SESSION ID:(411.3766) 2012-09-10 11:30:12.244
DEADLOCK DETECTED
[Transaction Deadlock]
Current SQL statement for this session:
UPDATE SP SET SRVC_TM = :B4 , MODIFICATION_DTM=SYSDATE WHERE OPERATION_AREA_CD = :B3 AND ROUTE_TYP = :B2 AND OBJECTID = :B1
----- PL/SQL Call Stack -----
object line object
handle number name
000000057D9B52E8 134 function ER.FUNC_UPDATESERVICETIME
000000057C3A5848 1 anonymous block
The following deadlock is not an ORACLE error. It is a
deadlock due to user error in the design of an application
or from issuing incorrect ad-hoc SQL. The following
information may aid in determining the deadlock:
Deadlock graph:
---------Blocker(s)-------- ---------Waiter(s)---------
Resource Name process session holds waits process session holds waits
TX-00040020-0017465b 63 411 X 94 364 X
TX-00020020-00166804 94 364 X 63 411 X
session 411: DID 0001-003F-00000033 session 364: DID 0001-005E-00000016
session 364: DID 0001-005E-00000016 session 411: DID 0001-003F-00000033
Rows waited on:
Session 364: obj - rowid = 0000CC64 - AAAMxkAA2AAA1q2AAY
(dictionary objn - 52324, file - 54, block - 219830, slot - 24)
Session 411: obj - rowid = 0000CC64 - AAAMxkAA2AAA1q2AAR
(dictionary objn - 52324, file - 54, block - 219830, slot - 17)
Information on the OTHER waiting sessions:
Session 364:
pid=94 serial=6104 audsid=693767 user: 57/ER
O/S info: user: , term: , ospid: 1234, machine: abc
program:
Current SQL Statement:
UPDATE SP SET ORIG_NO='751' ,ORIG_SEQ_NO=0,SP_ROUTING_STATUS='A', USER_ID='XXXX', MODIFICATION_DTM=SYSDATE WHERE OBJECTID IN ('104883389','104883404','104883407','104883440','104883443','104883455','104883467','104883509','104883545','104883764','104883788','104883806','104883812','104883821','104883836','104883854','104883863','104883893','104883899','104883931','104883937','104883964','104884084','104884117','104884120','104884138','104884141','104885439','104883386','104883422','104883560','104883587','104883767','104883785','104883809','104883824','104883845','104883851','104883884','104883890','104883955','104883958','104884012','104884093','104884114','104885412','104885436','104885442','104885445','104883383','104883395','104883413','104883419','104883464','104883494','104883524','104883773','104883842','104883917','104883920','104883943','104883949','104883967','104883997','104884051','104884105','104884108','104885451','104883437','104883461','104883476','104883497','104883500','104883503','104883566','104883584','104883614','104883794','104883800','104883815','104883830','104883857','104883869','104883923','104883952','104884048','104884057','104884063','104884066','104884081','104884087','104884102','104884111','104884135','104885415','104885424','104885427','104886297','104886308','104883398','104883410','104883458','104883473','104883512','104883515','104883527','104883530','104883536','104883554','104883596','104883770','104883782','104883803','104883827','104883833','104883839','104883848','104883866','104883875','104883878','104883896','104883902','104883914','104883970','104883976','104884060','104884069','104884072','104884123','104884132','104885409','104885430','104883425','104883431','104883446','104883449','104883452','104883482','104883506','104883518','104883539','104883548','104883569','104883575','104883578','104883623','104883779','104883797','104883818','104883860','104883925','104883934','104883940','104883946','104883973','104883979','104883982','104884078','104884090','104884096','104885421','104885448','104885454','104883392','104883416','104883428','104883479','104883491','104883521','104883542','104883551','104883557','104883563','104883872','104883911','104883928','104883961','104883994','104884018','104884054','104884099','104884129','104886299','104883401','104883434','104883470','104883485','104883533','104883572','104883581','104883776','104883791','104883881','104883887','104883905','104883908','104884075','104884126','104885418','104885433')
End of information on OTHER waiting sessions.
===================================================
PROCESS STATE
-------------
Process global information:
process: 000000057B3343D8, call: 0000000574FCBF78, xact: 0000000576A07F60, curses: 000000057E48D858, usrses: 000000057E48D858
----------------------------------------
SO: 000000057B3343D8, type: 2, owner: 0000000000000000, flag: INIT/-/-/0x00
(process) Oracle pid=63, calls cur/top: 0000000574FCBF78/0000000574FD4C48, flag: (0) -
int error: 0, call error: 0, sess error: 0, txn error 0
(post info) last post received: 108 0 4
last post received-location: aaa
last process to post me: 7e31d890 1 6
last post sent: 0 0 112
last post sent-location: bbb
last process posted by me: 7b334c00 3 0
(latch info) wait_event=0 bits=10
holding (efd=19) 4745310 Parent+children enqueue hash chains level=4
Location from where latch is held: cmi: gpl:
Context saved from call: 0
state=busy, wlstate=free
recovery area:
Dump of memory from 0x000000057E300810 to 0x000000057E300830
57E300810 00000000 00000000 00000000 00000000 [...............
I have been reaserching this issue from past few days. From what I saw few are saying its a indexing issue, few are saying its INITRANS... I am not sure.. But this deadlock happens very rare. But whenever it happens its a big issue.
So please help me guys.. what to look for.. and how I can solve this issue..
Look at your two UPDATE statements and try to understand why they would request the same row, but in a different order. That's how almost all
deadlocks happen.
There are several possible ways to avoid this error:
1) Update rows in the same order. You may be able to do this with a hint to force a full table scan or index. (I'm not 100% certain that using the same access method will always avoid this issue, but in practice it does seem to fix it. See my old question for a painful discussion about deadlocks when the access method is the same.)
2) Do not run your two processes at the same time.
3) Handle exceptions. For example, something like this:
declare
deadlock exception;
pragma exception_init(deadlock, -00060);
begin
<code>
exception when deadlock then
<do something about it here, such as re-try>
end;
/
You need to add the exception handling to both blocks of code. And a deadlock will still generate a trace file, which will probably
slow things down and take up a lot of space where no one expects it.