I have a website deployed to a tomcat server which has been very rapidly using up its available heap space (Old Gen) then crashing. When I took a heap dump i found that most (if not all) of the space is being taken up by lots of org.hibernate.impl.SessionFactoryImpl instances (802 to be precise with a keep alive size of 541mb) (referenced from org.apache.catalina.loader.WebappClassLoader -> java.util.concurrent.ConcurrentHashMap$HashEntry)
About the server
The server is tomcat 6 proxied by apache2 using proxypass.
About the software
I am using opencms to content manage the webapp and as such all the code is called through its template classes
Hibernate is accessed using springs HibernateTemplate. The data source is held in tomcat and is accessed through org.springframework.jndi.JndiObjectFactoryBean and injected into my datasource (org.apache.commons.dbcp.BasicDataSource), the session factory is configured as below.
<bean id="sessionFactory" class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="annotatedClasses">
<list><value>com.someobjects.SomeObject</value>
</list>
</property>
Does any one know why org.hibernate.impl.SessionFactoryImpl is not being garbage collected?
I can provide any further information required... I really am at a loss with this one. Any help is much appreciated.
Related
I have spring apps A, B and C that are running on single Tomcat 7 instance.
A is only entry point and relies on B & C functionality. B and C expose their service bean via rmi.
In future there would be more apps like B and C (D, E ...).
At the moment rmi registry is initiated by A on load via context loader of
<bean id="registry" class="org.springframework.remoting.rmi.RmiRegistryFactoryBean">
<property name="port" value="${batch.rmi.port}"/>
</bean>
<bean id="serverConnector" class="org.springframework.jmx.support.ConnectorServerFactoryBean" depends-on="registry">
<property name="objectName" value="connector:name=rmi"/>
<property name="serviceUrl" value="service:jmx:rmi://localhost:${batch.rmi.port}"/>
</bean>
My problem is that when A is restarted rmi registry also is restarted and B & C lose their entry in registry. Moreover on server restart I can't guarantee that A will load before B & C (it is ok so far ... due to naming order?).
I can't find a mention of ~setting some listener in server.xml so that registry is loaded on server start. All the google has for me is JmxRemoteLifecycleListener that starts jmx - which I don't want to be running (remote config is discouraged in my case).
Edit #1
Apparently I could did so via MBean loaded by org.apache.catalina.mbeans.ServerLifecycleListener ... up to 6, got dropped from 7.
Edit #2
Initial temp solution was to manually restart failed apps after restart.
Current temp solution is dedicated app (rmi context only) that has path "a..." (I guess some symbols/numbers would be more suitable) so that Tomcat loads it first. Not pretty but works for now - until there are more "a..." apps or tomcat behavior is changed in next update.
Edit #3
B & C are exposed as
<bean id="registry" class="org.springframework.remoting.rmi.RmiRegistryFactoryBean">
<property name="host" value="${batch.rmi.host}"/>
<property name="port" value="${batch.rmi.port}"/>
</bean>
<bean class="org.springframework.remoting.rmi.RmiServiceExporter">
<property name="serviceName" value="ExampleBatchJob"/>
<property name="service" ref="jobLauncherController"/>
<property name="serviceInterface" value="org.example.IJobLauncherController"/>
<property name="registry" ref="registry"/>
</bean>
When host parameter is specified app looks up registry and never attempts to create if not found. Another approach to problem is that there is no setting for RmiRegistryFactoryBean to "lookup registry, initiate one if not found" -> my misunderstanding, explained by aecolley.
By hooking the RMIRegistry to A (and nothing else) with depends-on, you're assuring that the RMIRegistry will be destroyed shortly after A is destroyed. (See Spring docs.)
There is indeed a setting for RmiRegistryFactoryBean to create a registry if one is not found: simply omit the host property. However, I suggest making all the registry-using applications depend on the same registry with depends-on if possible, to eliminate any randomness about which one gets started first.
We have a web site using spring.
I found in the code 2 ways of connecting to the oracle database, both use a bean called phareDataSource :
1st method :
<bean id="phareDataSource"
class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName"
value="oracle.jdbc.driver.OracleDriver" />
<property name="url"
value="${hibernate.connection.url}" />
<property name="username" value="${hibernate.connection.username}" />
<property name="password" value="${hibernate.connection.password}" />
</bean>
and 2nd method :
<bean id="phareDataSource"
class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName">
<value>jdbc/PHARE</value>
</property>
</bean>
In Jonas Directory : jonas.properties
jonas.service.dbm.class org.ow2.jonas.dbm.internal.JOnASDataBaseManagerService
jonas.service.dbm.datasources PHARERH
PHARERH.properties :
datasource.name jdbc/PHARE
datasource.url jdbc\:oracle\:thin\:#blabla\:1521\:R59A001
datasource.classname oracle.jdbc.driver.OracleDriver
datasource.username bla
datasource.password bla
The first method or the second one is picked when we are building (maven active profile).
The first uses a simple configuration file, the second uses a configuration file located in jonas conf directory.
Since we use tomcat in our dev environment, we picked the first method.
But in our int and prod environment with Jonas installed, shouldn't we use the second method ?
Is it better in performance ?
Both will use a JDBC connection pool to manage your connections so I would not expect there to be a major difference in performance.
Method 1 doesn't use any of the JOnAS features to create or manage the JDBC connections + pool. The same configuration will work outside of the application server. This is useful for testing without the requirement to deploy to the application server.
Method 2 will use the JDBC connection pool configured and managed by JOnAS (the application server).
Typically, if you have made the decision to go with an application container (e.g. JOnAS, JBoss, Websphere, etc) it is usually a good idea to let the container manage the resources that you use. That's what they are designed to do, and you may require some of the advanced features for managing the configured resources. There is also the benefit that your application doesn't have to know what implementation/driver is being used (or username/password, so you can deploy your EAR/WAR to different application servers without having to change your application configuration. These details are configured in the server instance instead.
If using Method 1 inside an application server, the server will have no control over the threads being created, because it knows nothing about them.
I have developed a small webapp using and SpringMVC(3.1.3.RELEASE) and Hibernate 4.2.0.Final.
I'm trying to convert it to be a multi-tenant application.
Similar topics have been covered in other threads, but I couldn't find a definitive solution to my problem.
What I am trying to achieve is to design a web app which is able to:
Read a datasource configuration at startup (an XML file containing multiple datasource definitions, which is placed outside the WAR file and it's not the application-context or hibernate configuration file)
Create a session factory for each one of them (considering that each datasource is a database with a different schema).
How can i set my session factory scope as session? ( OR Can i reuse the same session factory ?) .
Example:
Url for client a - URL: http://project.com/a/login.html
Url for client b - URL: http://project.com/b/login.html
If client "a" make request,read the datasource configuration file and Create a session factory using that XML file for the client "a".
This same process will be repeating if the client "b" will send a request.
What I am looking, how to implement datasource creation upon customer subscription without editing the Spring configuration file. It needs to be automated.
Here is my code ,that i have done so far.
Please anyone tell me,What modifications i need to be made?
Please give an answer with some example code..I am quite new in spring and hibernate world.
Spring.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close" p:driverClassName="${jdbc.driverClassName}"
p:url="${jdbc.databaseurl}"
p:username="${jdbc.username}" p:password="${jdbc.password}" />
<bean id="sessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">${jdbc.dialect}</prop>
<prop key="hibernate.show_sql">true</prop>
</props>
</property>
</bean>
<bean id="transactionManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
JDBC.properties File
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.dialect=org.hibernate.dialect.MySQLDialect
jdbc.databaseurl=jdbc:mysql://localhost:3306/Logistics
jdbc.username=root
jdbc.password=rot#pspl#12
hibernate.cfg.xml File
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
"-//Hibernate/Hibernate Configuration DTD//EN"
"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">
<hibernate-configuration>
<session-factory>
<mapping class="pepper.logis.organizations.model.Organizaions" />
<mapping class="pepper.logis.assets.model.Assets" />
</session-factory>
</hibernate-configuration>
Thanks,
First create a table for Tenant with tenant_id and associate it with all users.Now, you can fetch this details while the user logs in and set it in session.
We are using AbstractRoutingDataSource to switch DataSource for every request on Spring Boot. I think it is Hot Swapable targets/datasource mentioned by #bhantol above.
It solves our problems but I don't think it is sound solution. I guess JNDI could be a better one than AbstractRoutingDataSource.
Wondering what you ended up with.
Here are some ideas for you.
Option 1) Single Application Instance.
It is somewhat ambitious to to this using what you are actually trying to achieve.
The gist is to simply deploy the same exact application with different context root on the same JVM. You can still tune the JVM as a whole like you would have if you had a truely multi-tenant application. But this comes at the expense of duplication of classes, contexts, local caching, start up times etc.
But as of today the Spring Framework 4.0 does not provide much of an multi-tenancy support (other than Hot Swapable targets/datasource) etc. I am looking for a good framework but it may be a wash to move away from Spring at this time for me.
Option 2) Multiple deployments of same application (more practical as of today)
Just have your same exact application deploy to the same application server JVM instance or even different.
If you use the same instance you may now need to bootstrap your app to pickup a DataSource based on what the instance should serve e.g. client=a property would be enough to pickup a **a**DataSource" or **b**DataSource I myself ended up going this approach.
If you have a different application server instance you could just configure a different JNDI path and treat things generically. No need for client="a" property because you have liberty to define your datasource differently with the same name.
I'm currently trying to configure hibernate search via spring across 3 machines for the purpose of using a JMS distributed index. Due to the way we deploy our software I have to use the same configuration across all three machines but I need a way to set one of them to be the JMS Master.
Currently hibernate is being configured via Spring using the following bean declaration:
<bean class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean"
id="productSessionFactory">
<property name="dataSource">
<ref local="productDataSource"/>
</property>
<property name="configLocation">
<value>classpath:hibernate.cfg.xml</value>
</property>
<property name="entityInterceptor" ref="builderInterceptor"/>
<property name="eventListeners">
<map key-type="java.lang.String" value-type="java.lang.Object">
<entry key="save" value-ref="saveEventListener"/>
</map>
</property>
</bean>
On one of the three machines I need to set the property hibernate.search.default.directory_provider to filesystem-master and on the other two I need to set it to filesystem-slave.
I have the ability to set flags on the individual machines to identify which machine should be the master but due to all the configuration being XML I dont have any ability to add logic to set the parameters correctly.
Is there an way to set this parameter programmatically while leaving the rest of the configuration alone?
Thanks!
A programmatic way is generally possible, but I am not sure exactly how you do that in Spring. Instead of putting your properties into a config file you would have to build the properties dynamically (or at least partly dynamically) and pass it to AnnotationSessionFactoryBean. If I am not mistaken there are hooks in the Spring SPI which should allow you to do that, eg BeanDefinitionRegistryPostProcessor.
The other approach would be to write your own DirectoryProvider. Have a look at org.hibernate.search.store.impl.FSMasterDirectoryProvider and org.hibernate.search.store.impl.FSSlaveDirectoryProvider and write a provider which can act as slave or master depending on the flag you can read on the machine.
I have been trying to upgrade my ojdbc code from ojdbc14-10.2.0.1.0 to ojdbc6-11.1.0.7.0. We have been using OracleConnectionCacheImpl for datasource connections and then moved to the Universal Connection Pool using OracleDataSource at the heart. Here is how we currently have it configured in Spring:
<bean id="myDatasource" class="oracle.ucp.jdbc.PoolDataSourceFactory" factory-method="getPoolDataSource">
<property name="URL" value="#JDBC_URL#"/>
<property name="user" value="#JDBC_USERNAME#"/>
<property name="password" value="#JDBC_PASSWORD#"/>
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleDataSource"/>
<property name="connectionPoolName" value="MFR_RTE_POOL"/>
<property name="minPoolSize" value="5"/>
<property name="maxPoolSize" value="100"/>
<property name="validateConnectionOnBorrow" value="true" />
<property name="connectionWaitTimeout" value="30"/>
<property name="connectionHarvestMaxCount" value="25"/>
<property name="connectionHarvestTriggerCount" value="5"/>
<property name="maxStatements" value="100"/>
</bean>
It took a bit to get it to run without closed connection errors, but now I have an issue with memory management. I have run jconsole against an application I have that uses a ThreadPool. This application uses a Thread Pool and uses ThreadPoolExecutors to create fee requests based on the data passed from the file. A file can have hundreds of thousands of fee requests. My issue is that long term memory in the Heap is filling up and is not releasing objects. In the performance test I have set up, long term memory in Garbage Collection is filling up in about 20-25 minutes and does not ever free up. The application eventually hits the GC Limit Exceeded Exception and comes to grinding halt.
When I run the same test using the old OracleConnectionCacheImpl class it just runs with no problem. Granted the thread pool and all accompanying code was written to run using older versions of Spring (1.2.6) and old ojdbc driver, but is there really that big of difference in the way OracleConnectionCacheImpl works versus Universal Connection Pooling? Am I looking at rewriting my domain model if I want to accommodate the latest versions of Oracle's JDBC driver code. I have tried OracleDataSource connection and it failed miserably with NullPointerExceptions after working on several files concurrently. I then went to UCP (at the suggestion of another post in this forum) which works fine in all but one application. At this point I'm trying to figure out whether I can further optimize the Spring config bean for my datasource or do I need to start thinking about upgrading the code base. As stated previously, this code runs very well against the old ojdbc class, but I have had issues every step of the way trying to implement UCP. I'm startg to wonder if its even worth upgrading.
This problem had bugged me for months, I hope what I came up with helps someone else out there:
I did finally figure out a solution to my issue. Instead of using OracleDataSource as the connection factory :
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleDataSource"/>
I would suggest trying OracleConnectionPoolDataSource:
<property name="connectionFactoryClassName" value="oracle.jdbc.pool.OracleConnectionPoolDataSource"/>
OracleConnectionPoolDataSource extends OracleDataSource and seems to do better in applications where multiple connections need to be opened by multiple resources. In my case I have an application that requires processing multiple batch files. The same SQL code is run over and over, but the application needs a new connection for each new file. Under these circumstances OracleDataSource often times had failed connection errors or some sort (e.g. SQLException: closed connection, NullPointerException: connection closed with or without UCP), lead to issues with Garbage Collection (long-term GC would fill up and cause GC to ultimately fail no matter how much memory I added to the JVM).
I found OracleDataSource to work well on applications that do not do use a lot of batch processing. For instance another application I use is a file processing application but it only works on one file at a time. OracleDataSource works great in this circumstance. It also seems to work well for Web Applications as well. We have a web app we installed OracleDataSource in 9 months ago and has had no issues.
I'm sure there are ways to make OracleDataSource work as well as OracleConnectionPoolDataSource, but this is worked for me.