I was using the SimpleMessageStore ( memory ) as the default Store for an Aggregator, but as i need
persistent store, i started using a jdbc solution backed up by Mysql 5.5
But i'm really concerned about its performance, as using a PerfTest ( databene's contiperf ), memory store achieved up to 6000 messages/sec, whereas using mysql ( or redis, or mongo ), i can't achieve more than 15-20 messages/sec.
<int:aggregator id="XDRAggegator"
input-channel="mqChannel"
output-channel="publishChannel"
message-store="jdbc-messageStore"
....
<bean id="reaper" class="org.springframework.integration.store.MessageGroupStoreReaper">
<property name="messageGroupStore" ref="jdbc-messageStore"/>
<property name="timeout" value="${msg.timeout}"/>
</bean>
<task:scheduler id="taskScheduler"/>
<task:scheduled-tasks scheduler="taskScheduler">
<task:scheduled ref="reaper" method="run" fixed-rate="${reaper.rate}"/>
</task:scheduled-tasks>
<task:executor id="taskExecutor" pool-size="1"/>
<task:annotation-driven executor="taskExecutor" scheduler="taskScheduler"/>
<int-jdbc:message-store id="jdbc-messageStore" data-source="dataSource" />
<bean id="dataSource" class="com.jolbox.bonecp.BoneCPDataSource" >
<property name="driverClass" value="com.mysql.jdbc.Driver" />
<property name="jdbcUrl" value="jdbc:mysql://${database.host}:3306/XDR?autoReconnect=true&cachePrepStmts=true&prepStmtCacheSize=50" />
<property name="username" value="${database.username}"/>
<property name="password" value="${database.password}"/>
<property name="idleConnectionTestPeriodInMinutes" value="1"/>
<property name="idleMaxAgeInMinutes" value="2"/>
<property name="connectionTimeoutInMs" value="1000" />
<property name="maxConnectionsPerPartition" value="10"/>
<property name="minConnectionsPerPartition" value="4"/>
<property name="partitionCount" value="4"/>
<property name="acquireIncrement" value="2"/>
<property name="statementsCacheSize" value="10"/>
<property name="closeConnectionWatch" value="false"/>
</bean>
Am i missing something ?
I'm using default schema that comes into spring-integration-jdbc library.
Related
(I hope this below makes sense, I had a bit of a struggle with formatting).
We have an application (currently being refreshed from technology of circa 2005) whose owners require the use of Oracle AQ for message handling.
We are using the Spring JDBC extensions, and when the application begins to listen to the queue it gets a ClassCastException saying "X cannot be cast to oracle.jdbc.internal.OracleConnection".
Where X is either a WrappedConnectionJDK7 from jboss when we use a datasource defined in Wildfly e.g.
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
or X is a PoolGuardConnectionWrapper from commons.dbcp when we define the datasource like this:
<bean id="dataSource-supsrep"
class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:XE" />
<property name="username" value="...." />
<property name="password" value="...." />
</bean>
Message Listener bean is defined like this :
<bean id="messageListener" class="xxx.report.dispatch.DispatchMDB" />
Message Listener container bean is like this :
<bean id="reportProcessorXslfoHigh"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="sessionTransacted" value="true" />
<property name="connectionFactory" ref="AQconnectionFactory" />
<property name="destinationName" value="Q1_XSLFO_HIGH" />
<property name="messageListener" ref="messageListener" />
<property name="transactionManager" ref="transactionManager" />
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="1" />
<property name="receiveTimeout" value="30000" />
<property name="idleTaskExecutionLimit" value="30" />
<property name="idleConsumerLimit" value="1" />
</bean>
The connection factory is like this :
<orcl:aq-jms-connection-factory id="AQconnectionFactory"
data-source="dataSource-supsrep" native-jdbc- extractor="oracleNativeJdbcExtractor" />
and it uses a Native JDBC Extractor :
<bean id="oracleNativeJdbcExtractor"
class="org.springframework.jdbc.support
.nativejdbc.OracleJdbc4NativeJdbcExtractor" />
So my question is - can this be overcome by Spring configuration details - or is there another way round it? Clearly the underlying connection is needed but after 3 days of research I cannot see how it can be done.
Any clues gratefully received !
Thanks,
John
I have same database schema database_1 and database_2 and want to configuration of these two database(database schema is same) and decide run time which database is use.
My persistence.xml is as below:
<persistence-unit name="first" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<class>com.model.AboutUs</class>
<class>com.model.AccessCards</class>
<properties>
<property name="hibernate.dialect" value="com.util.customMySQLDialect" />
<property name="hibernate.hbm2ddl.auto" value="none" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.cache.provider_class" value="org.hibernate.cache.HashtableCacheProvider" />
<property name="hibernate.jdbc.batch_size" value="20"/>
<property name="hibernate.cache.use_query_cache" value="true"/>
<property name="hibernate.cache.use_second_level_cache" value="true"/>
</properties>
</persistence-unit>
Persistence name second configure here as below :
<persistence-unit name="second" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<class>com.model.AboutUs</class>
<class>com.model.AccessCards</class>
<properties>
<property name="hibernate.dialect" value="com.util.customMySQLDialect" />
<property name="hibernate.hbm2ddl.auto" value="none" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.cache.provider_class" value="org.hibernate.cache.HashtableCacheProvider" />
<property name="hibernate.jdbc.batch_size" value="20"/>
<property name="hibernate.cache.use_query_cache" value="true"/>
<property name="hibernate.cache.use_second_level_cache" value="true"/>
</properties>
</persistence-unit>
Both persistence units load same classes .
and now my database-configure.xml as where i configure persistence unit with data source. as below first persistence unit name configare with datasource name datasource as :
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="persistenceUnitName" value="first" />
<property name="persistenceXmlLocation" value="classpath*:META-INF/persistence.xml"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
</bean>
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${datasource.driverClassName}" />
<property name="jdbcUrl" value="${datasource.url}" />
<property name="user" value="${datasource.username}" />
<property name="password" value="${datasource.password}" />
<property name="acquireIncrement" value="1" />
<property name="acquireRetryAttempts" value="1" />
<property name="maxIdleTime" value="300" />
<property name="maxPoolSize" value="20" />
<property name="minPoolSize" value="3" />
<property name="maxStatements" value="300" />
<property name="testConnectionOnCheckin" value="true" />
</bean>
Now this code for configure second persistence unit name with data source name datasource5 as below :
<bean id="transactionManager5" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory5" />
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource5" />
<property name="persistenceUnitName" value="second" />
<property name="persistenceXmlLocation" value="classpath*:META-INF/persistence.xml"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter" />
</property>
</bean>
<bean id="dataSource5" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${datasource.driverClassName}" />
<property name="jdbcUrl" value="${datasource5.url}" />
<property name="user" value="${datasource5.username}" />
<property name="password" value="${datasource5.password}" />
<property name="acquireIncrement" value="1" />
<property name="acquireRetryAttempts" value="1" />
<property name="maxIdleTime" value="300" />
<property name="maxPoolSize" value="20" />
<property name="minPoolSize" value="3" />
<property name="maxStatements" value="300" />
<property name="testConnectionOnCheckin" value="true" />
</bean>
Now i use entity manager for save data in database and i am using mysql database . I got entity manager with persistence unit name first and second .
#PersistenceContext(unitName="first")
private EntityManager entityManager;
#PersistenceContext(unitName="second")
private EntityManager entityManager2;
Now entity manager save data in database using persist( object) method successfully .
entityManager.persist(project);
and entityManager2 save data in database using persist( object ) method successfully (No Exception here). But data does not store in database datbase_2 .
entityManager2.persist(project);
If I change the order of persistence unit first and second in persistence.xml then entityManager2 save data in database and entitymanager does not save data in database .
Any one have idea how i can make multiple entitymanager for same database schema .
There are a few limitations:
#Transactional corresponds to single TransactionManager.
JpaTransactionManager corresponds to single EntityManager(Factory).
Thus use separate methods with #Transactional annotation for each transaction manager:
#Transactional(transactionManager = "tmBeanName")
After that entity managers will work fine.
Also, some workaround exists to break the second limitation (DO NOT USE IT in this case):
It is possible to use single JtaTransactionManager instead of several JpaTransactionManager: distributed transaction will cover both entity managers and they will work fine too.
I am trying to configure the multiple JPA repositories in my web application. Currently I have one Repository as mentioned below. I am using Crud Repository
<jpa:repositories base-package="com.mypackage1" factory-class="org.springframework.data.jpa.repository.support.JpaRepositoryFactoryBean" transaction-manager-ref="transactionManager" entity-manager-factory-ref="entityManagerFactory">
<repository:include-filter type="assignable" expression="com.MyFirstRepository"/>
</jpa:repositories>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="com.mypackage1.Model" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="generateDdl" value="true" />
<property name="database" value="H2" />
</bean>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:database/~test" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
If I need to configure multiple repositories,
<bean id="entityManagerFactory2"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource2" />
<property name="packagesToScan" value="com.mypackage2.Model" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="generateDdl" value="true" />
<property name="database" value="H2" />
</bean>
</property>
</bean>
<bean id="transactionManager2" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory2" />
</bean>
<bean id="dataSource2"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:database/~test2" />
<property name="username" value="sa" />
<property name="password" value="" />
</bean>
I am not able to find any appropriate configuratiuon for this.
Appreciate help on this.
I got one strange problem.
when I config a DataSourceTransactionManager with spring xml, the concurrent consumers of ActiveMQ were suppressed whatever I change "maxConcurrentConsumers" property value. I have 5 queues, the total concurrent consumers of all 5 queue always kept at 8.
if I remove DataSourceTransactionManager bean, each queue's concurrent consumers reached the max number 5 declared in "maxConcurrentConsumers" .
The DataSourceTransactionManager work for dataSource, i cannot understand why it affected to ActiveMQ.
version:
Spring 3.2.5.RELEASE
ActiveMq 5.9.0
application.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
</bean>
<!-- once I add this, activemq total consumers always kept at 8 -->
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<!-- activemq consumer connection -->
<bean id="consumerConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory"
destroy-method="stop">
<property name="connectionFactory">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://localhost:61616</value>
</property>
</bean>
</property>
<property name="maxConnections" value="5"></property>
</bean>
<!-- i have 5 queues -->
<bean id="test_1" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_1}" />
</bean>
<bean id="test_2" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_2}" />
</bean>
<bean id="test_3" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_3}" />
</bean>
<bean id="test_4" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_4}" />
</bean>
<bean id="test_5" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg index="0" value="test_5}" />
</bean>
<!-- consumer listener container -->
<bean id="testOneMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_1"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testTwoMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_2"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testThreeMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_3"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFourMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_4"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
<bean id="testFiveMessageListenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="consumerConnectionFactory"></property>
<property name="concurrentConsumers" value="1" />
<property name="maxConcurrentConsumers" value="5" />
<property name="destination" ref="test_5"></property>
<property name="messageListener" ref="demoBusinessListener"></property>
</bean>
can someone help me!!!
after some test, i found a way to resolve this problem.
when i change the dataSource "maxActive" parameter to a number greater than sum of all mq listener maxConcurrentConsumers. it work fine.
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="${jdbc.driver}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.user}" />
<property name="password" value="${jdbc.password}" />
<property name="maxActive" value="120" />
</bean>
it seems that the max number of activemq listener thread affected by Datasource maxActive parameter
I was wondering if I can run multiple c3p0 datasources for one DB, something like:
<bean id="dataSource1" class = "com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${db.driverClassName}"/>
<property name="jdbcUrl" value="${db.url}/schema1"/>
<property name="user" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="acquireIncrement" value="1" />
<property name="idleConnectionTestPeriod" value="100"/>
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="1800" />
</bean>
<bean id="dataSource2" class = "com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${db.driverClassName}"/>
<property name="jdbcUrl" value="${db.url}/schema2"/>
<property name="user" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="acquireIncrement" value="1" />
<property name="idleConnectionTestPeriod" value="100"/>
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="1800" />
</bean>
They will be used by difference persistence services.
Thanks.
That's absolutely fine. You'll run into some configuration issues like:
autowiring DataSource by type won't work
#Transactional/declarative transactions will work only against one, selected DataSource. Alternatively you'll have to manually tell which transaction manager you want to use (thus you'll need two transaction managers as well: transactionManager1 and transactionManager2):
#Transactional("transactionalManager2")
Besides, there's nothing wrong with this configuration. Actually it's a pretty good idea (+1): if some layers/components of your application saturate the pool, others can still operate.
The only thing I recommend is to use lesser known abstract/parent bean declarations to avoid repetition:
<bean id="dataSource" abstract="true" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${db.driverClassName}"/>
<property name="user" value="${db.username}"/>
<property name="password" value="${db.password}"/>
<property name="acquireIncrement" value="1" />
<property name="idleConnectionTestPeriod" value="100"/>
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="50" />
<property name="maxIdleTime" value="1800" />
</bean>
<bean id="dataSource1" parent="dataSource">
<property name="jdbcUrl" value="${db.url}/schema1"/>
</bean>
<bean id="dataSource2" parent="dataSource">
<property name="jdbcUrl" value="${db.url}/schema2"/>
</bean>
See also:
What is meant by abstract="true" in spring?