I am currently using Spring Batch to import data from a SQL server. In order to make the datasource configurable I needed to "step scope" the datasource bean. However, this concerns me. If the datasource bean, which does connection pooling, is step scoped, then how can it manage connection in a pool and is there even a benefit to using it.
My datasource is configured as follows:
<bean id="dataSourceMssql" class="org.apache.tomcat.jdbc.pool.DataSource" scope="step">
<property name="driverClassName" value="${batch.mssql.driver}" />
<property name="username" value="${batch.mssql.user}" />
<property name="password" value="${batch.mssql.password}" />
<property name="removeAbandoned" value="true" />
<property name="removeAbandonedTimeout" value="3610" />
<property name="url"
value="${batch.mssql.connect}#{jobParameters['dburl']}:#{jobParameters['port']}/#{jobParameters['databaseName']}" />
</bean>
Why is it step scoped? Because I needed to retrieve the jobParameters to configure the datasource.
What do I want to know?
Will connection pooling still occur? (Perhaps the beans resources stay alive and are reclaimed)
I appreciate the help.
The scope "step" is only usable on spring batch beans. Other beans (Spring) only know the scope : singleton, prototype, request or session.
Normal way to handle this is to set these parameters in a properties file read by your applicatioonContext.xml.
JobParameters are used to pass Job related parameters (path, filename, date, seqNo etc) since a job with the same JobParameters won't be able to run twice.
EDIT: Is your job multithreaded? Because majority of the jobs created are single threaded! If your job is in fact single thread, i would ask myself why pooling connections is needed!
Regards
Related
Am running a Spring Application deployed in Apache Tomcat server
With below bean,
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/appln" />
<property name="username" value="root" />
<property name="password" value="password" />
<property name="initialSize" value="3" />
<property name="maxActive" value="10" />
</bean>
How does this bean works, Will this bean validates the DB credentials for every DB request made from application ?
When application is running, and the DataBase password is updated, will this bean fails for any application request or it works since it is already validated
For the first question, when Datasource object as bean(singleton) is created, the database information is validated the first time. After that when you inject your Datasource bean to the transaction manager bean, the database operations will be managed by these beans so open session, commit, rollback etc. Look the #Transactional annotation usage.
For the second one, you could change the bean definition in jar/war/ear, after that restart your application. But you can pass these information in the configuration file like application.properties/datasource.properties that you specified in applicationContext.xml. This will simplify your deployments when you change anythings. Without the restart, you cannot do to pass new password for database.
Adding to what Semih Okan Pehlivan said,
It is possible to refresh the password for the database if you put the properties in the application.properties, though you would need to add spring-cloud-starter to your dependencies.
In spring I have a datasource defined in this way:
<bean id="dataSource" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<property name="URL" value="${jdbc.url}" />
<property name="user" value="${jdbc.username}"/>
<property name="password" value="${jdbc.password}"/>
<property name="connectionCachingEnabled" value="true"/>
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
This datasource is used by my REST service and everything works fine...anyway first REST call is very slow (about ~5 secs), after that EVERY other call is fast.
I think this is an initialization related problem, in the sense that initialization is made when first DB call is received.
Is there a way to tell spring to initialize this datasource on server startup?
I think this is an initialization related problem, in the sense that
initialization is made when first DB call is received.
With your current config I think that's what's happening.
Is there a way to tell spring to initialize this datasource on server
startup?
It's the behavior of the connection pool, not Spring. Spring is creating the bean when your app starts (you aren't using lazy-init="true" on the bean). However, the connection pool isn't creating connections to the database when Spring instantiates it. From the Oracle docs:
The initial pool size property specifies the number of available
connections that are created when the connection pool is initially
created or re-initialized. This property is typically used to reduce
the ramp-up time incurred by priming the pool to its optimal size.
A value of 0 indicates that no connections are pre-created. The
default value is 0.
Try setting a non-zero value for initialPoolSize.
Edit: Try setting ConnectionCacheProperties instead:
<property name="connectionCacheProperties">
<props merge="default">
<prop key="InitialLimit">5</prop>
</props>
</property>
I am using Spring Batch to import large set of data from XML to Oracle database.
My application is multi tenant aware where each tenant can have separate db schema or few tenants can also share single db schema at the same time.
From my XML, I am getting unique tenant identifier which I used as a tenant database resolution identifier to select DB at a runtime.
I am using Spring MVC, hibernate.
My XML
<users>
<user>
<userAccountDetail>
<tenantId>acmebank</tenantId>
<emailAddress>966019620abc5009254#6657170682.com</emailAddress>
</userAccountDetail>
</user>
</users>
My Spring Batch Configuration
<batch:job id="walletUsersImportJob">
<batch:step id="importUsers" allow-start-if-complete="true">
<batch:tasklet>
<batch:chunk reader="xmlItemReader" writer="userDetailWriter"
processor="userDetailProcessor" commit-interval="2147483647">
</batch:chunk>
</batch:tasklet>
<batch:listeners>
<batch:listener>
<bean
class="com.masterpass.datamigration.batch.core.listener.ItemFailureHandler" />
</batch:listener>
</batch:listeners>
</batch:step>
</batch:job>
DataSource Configuration
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="routingDataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="true" />
</bean>
</property>
</bean>
<bean id="routingDataSource"
class="com.csam.wsc.enabling.tenant.jdbc.datasource.lookup.TenantRoutingDataSource" >
<property name="defaultTargetDataSource" ref="globalDataSource"/>
<property name="tenantMetadataLookupStrategy" ref="tenantMetadataLookupStrategy" />
</bean>
Here tenantMetadataLookupStrategy needs to be looked for tenant identifier i.e tenantId received as a part of XML and I have a map like
tenantId = Datasource
acmebank = java:/ACMEBANK_DS
anybank = java:/ANYBANK_DS
which will give me which database to be used for any DAO operation.
Hope configurations wise I am fine.
Let me tell my problem.
When Spring batch application boot strap, org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.determineTargetDataSource() called which needs tenantID to identify target datasource.
This tenantID, I have in XML and will not be available until ItemReader complete.
My Spring Batch configuration using batch:tasklet which surrounds reader, writer, and processor in a single transaction.
Due to above, during boot strap I suspect when initializing transaction manager , entityManagerFactory looks for datasource i.e routingDataSource and which in turn needed on tenantId.
I think if I remove reader and processor from transaction body, I will get some place to capture tenantId and set it in context somewhere so that it can select DB at runtime.
Even I do not want reader & processor transnational aware so better If I can wrap only writer in transaction.
Please suggest.
Two things here:
To "remove the read and process from the transaction body" in Spring Batch is virtually impossible. You'd have to rewrite the TaskletStep (one of the most sensitive pieces of code in the framework) which I would not recommend.
I believe you are correct in the reason the routing data source isn't working. That being said, I'd actually recommend a slightly different approach. Instead of using the routing data source technique, why not use the ClassifierCompositeItemWriter? This moves the routing up a level (from the connection to the item writer) which allows each writer instance to obtain a connection for the life of the transaction.
You can read more about the ClassifierCompositeItemWriter in the javadoc here: http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/support/ClassifierCompositeItemWriter.html
I'm using Hibernate with Spring, relevant config:
<bean id="transactionManager"
class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="openSessionInViewInterceptor" class="org.springframework.orm.hibernate3.support.OpenSessionInViewInterceptor">
<property name="sessionFactory"><ref bean="sessionFactory" /></property>
</bean>
<tx:annotation-driven />
<aop:aspectj-autoproxy />
Think about it...
Some code wants to obtain a Connection from a DataSource. Probably in order to start a transaction and run some SQL query
AbstractRoutingDataSource executes determineCurrentLookupKey() in order to find suitable DataSource from a set of available ones
Lookup key is used to obtain current DataSource. AbstractRoutingDataSource returns JDBC connections from that data source.
Connection is returned from AbstractRoutingDataSource as if it was a normal source.
Now you are asking why determineCurrentLookupKey() is not running within a transaction? First Spring would have to go to point 1. to fetch some database connection required to start a transaction. Look at the next point. See the problem? Smells like infinite recursion to me.
Simply put - determineCurrentLookupKey() can't run within a transaction because transaction needs a connection and the purpose of that method is to determine which DataSource to use to obtain a connection. See also: Chicken or the egg.
Similarly, the engineers couldn't use a computer to design the first computer.
I'm new to JBoss, having been using tomcat for years. I have a Spring 3.0.x application in which I need to run a job on a regular basis. In the past, I would simply create my job class as a regular POJO, and then create my job/trigger as Spring's CronTriggerBean passing a MethodInvokingJobDetailFactoryBean as my jobDetail.
Ex:
<bean id="session.manage.UserSessionPurgeAction.trigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="cronExpression" value="0 */5 * * * ? *" />
<property name="jobDetail">
<bean class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="name"><idref bean="session.manage.UserSessionPurgeAction" /></property>
<property name="group" value="cleanup" />
<property name="targetObject" ref="session.manage.UserSessionPurgeAction" />
<property name="targetMethod" value="execute" />
<property name="concurrent" value="false" />
</bean>
</property>
</bean>
On this new project, the system architect has called for running the Spring application under JBoss 6. I know that JBoss has a quartz scheduler built in, so I am not sure how to package/declare my job such that it is using JBoss' scheduler as opposed to building it into the app as I have done in the past.
I've searched online, but cannot seem to find the necessary glue information that I need. I know that there is a #Schedule annotation in javax.ejb but is that all I need to add to my method? I would think/expect that I need additional configuration somewhere, but not sure where.
Can anyone point me in the right direction please?
Thanks,
Eric
If you really want to use the jboss one, I would try matching the schedulerName property passed to SchedulerFactoryBean with one of the bundled scheduler. The bundled scheduler can be retrieved via JNDI, it's under jndi name "Quartz" I think.
Have a look at org.springframework.scheduling.quartz.SchedulerFactoryBean#createScheduler, it tries to look up the scheduler in static SchedulerRepository first.