quartz jobs in a cluster running on wrong node - spring

uI have an application that is clustered in a 2+2 setup where 2 machines are used in alpha product phase, and the other two are used for real customers. all machines are looking at the same database,
I need a job to run at midnight for each of the groups. one of the two alpha machines should get a job, and the other two should get another job.
I'm using Spring 3.0.5 with Quartz 1.8.5 with the following properties
phase=alpha
quartz.job.name=MY_JOB_${phase}
<bean id="quartzPropertiesFactoryBean" class="com.liveperson.kwo.quartz.QuartzPropertiesFactory">
<constructor-arg value="AUTO"/>
<constructor-arg value="MY_CLUSTER"/>
<constructor-arg value="JobStoreTX"/>
<constructor-arg value="StdJDBCDelegate"/>
<constructor-arg value="true"/> //isClustered
<constructor-arg value="false"/> //useProperties
</bean>
<bean name="runJobBean" class="org.quartz.JobDetail">
<property name="name" value="${quartz.job.name}"/>
<property name="jobClass" value="CLASS1"/>
<property name="group" value="JOB-for-${quartz.job.name}"/>
<property name="jobDataMap">
<bean class="org.quartz.JobDataMap">
</bean>
</property>
</bean>
<bean name="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="quartzProperties" ref="quartzPropertiesFactoryBean"/>
<property name="dataSource" ref="mySqlConnectorBean"/>
<property name="overwriteExistingJobs" value="true"/>
<property name="jobDetails">
<list>
<ref bean="runJobBean"/>
</list>
</property>
<property name="triggers">
<list>
<ref bean="cronTriggerBean"/>
</list>
</property>
</bean>
I define two jobs, one for alpha and another for production throw the phase property, and the problem I'm having is that the job I define for alpha phase runs on the node defined to production, how can I make the job defined for the alpha phase run only on machines defined for alpha?
Thanks!

Related

JdbcCursorItemReader - Stored Procedure call

Currently I am using JdbcCursorItemReader and FlatFileItemWriter in a job step.
Due to performance issue we have to use stored procedure.
Is there a way to make a call to stored proc in Spring Batch 2.0.. RELEASE?
<bean id="jdbcCursorItemReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="dataSource"/>
<property name="sql"
value="SELECT X,Y,Z
FROM V_VIEW "/>
<property name="mapper">
<bean class="com.mapping.SomeMapper"/>
</property>
</bean>
<bean class="org.springframework.batch.item.file.FlatFil eItemWriter" id="flatFileItemWriter">
<property name="resource" ref="resource"/>
<property name="fieldSetCreator">
<bean class="org.springframework.batch.item.file.mapping .PassThroughFieldSetMapper"/>
</property>
</bean>
... Other config
how to write a custom database reader wih callable statement...
Sample code is appreciated.. Thank You,.
There is a StoredProceedureItemReader that is built just for this use case. You can read more about it in the documentation here: https://docs.spring.io/spring-batch/apidocs/org/springframework/batch/item/database/StoredProcedureItemReader.html
I could find my answer:
<bean id="jdbcCursorItemReader" class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="dataSource"/>
<property name="sql" value="Call schema.StoredProcName"/>
<property name="mapper">
<bean class="com.mapping.SomeMapper"/>
</property>
</bean>

spring bean optional property

I am using a data source defined in tomcat in my spring configuration as shown in the below xml.
It can happen sometimes that this data source may not be defined in the context.xml of tomcat.
In such cases , the context initialization fails since myDS is not found.
Is it possible to configure the datasource as optional so that application initialisation is not impacted ?
There can be a run time error when this data source is accessed , which is acceptable
<bean id="myDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jdbc/myDS"/>
</bean>
<bean id="myEntityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="myDataSource" />
<property name="persistenceXmlLocation" value="classpath:META-INF/persistence.xml" />
<property name="packagesToScan" value="com..XX.XX" />
<property name="persistenceUnitName" value="myPU" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="#{systemProperties['showSql'] == null ? 'true' : systemProperties['showSql'] }" />
</bean>
</property>
<property name="persistenceUnitPostProcessors">
<list>
<ref bean="wrkflw-punitpostprocessor" />
</list>
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.dialect">#{systemProperties['dbDialect']}</prop>
</props>
</property>
</bean>
Thanks
Muhad
You may check the DelegatingDataSource, you could encapsulate the logic to load the datasource from JNDI within its instantiation. For your application there will be always a DataSource there, but in some cases (whenever its not able to load the DataSource from JNDI) there is no delegation.

Spring Batch - Load properties from database table

I have a requirement in my spring batch where I have to load few key value properties from a database table. Is this possible? The job runs in a stand alone environment and not in a container.
Please let me know if you have a solution for this. `I am in a secured environment and that is the reason I have not shared the code initially. Anyways below is what I have. Below are my properties
<bean id="properties"
class="org.springframework.beans.factory.config.PropertiesFactoryBean">
<property name="locations">
<list>
<value>classpath:EnvConfig.properties</value>
<value>classpath:DatabaseConfig.properties</value>
<value>classpath:WebServiceConfig.properties</value>
</list>
</property>
</bean>
<bean id="propertyPlaceholderConfigurer"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties" ref="properties" />
</bean>
<bean
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="properties">
<bean class="org.apache.commons.configuration.ConfigurationConverter"
factory-method="getProperties">
<constructor-arg>
<bean class="org.apache.commons.configuration.DatabaseConfiguration">
<constructor-arg>
<ref bean="sam-datasource" />
</constructor-arg>
<constructor-arg value="PTTMCDB.PROPERTY" /> <!-- DB Table -->
<constructor-arg value="PROPERTYNAME" /> <!-- DB Key Column -->
<constructor-arg value="PROPERTYVALUE" /> <!-- DB Value Column -->
</bean>
</constructor-arg>
</bean>
</property>
</bean>
This is where I am reading the properties fetched from DB.
<bean id="emailReaderUtil"
class="mailreader.pop3.EmailReaderUtil">
<property name="popServerHost" value="${pop3.popServerHost}"/>
</bean>

Creating multiple SchedulerFactoryBean in Quartz

I have run into a problem where I have two classes extending QuartzJobBean. The problem I am facing is to create two SchedulerFactoryBean. I did my research and found that setting the property schedulerName solves the problem. This did not work for me. I used #Qualifier also. If I create the two beans of SchedulerFactoryBean , Spring doesn't know which scheduler to refer to. I have two CronTriggers.
Code:
<!--
<bean name="quartzSchedulerR" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.task.QuartzScheduler" />
<property name="jobDataAsMap">
<map>
<entry key="rRSImpl" value-ref="rRSService" />
<entry key="SRObject" value-ref="SRObject"/>
</map>
</property>
</bean>
<bean id="cronTriggerR"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="quartzSchedulerR" />
<property name="cronExpression" value="0 30 12 ? * MON *" />
</bean>
<bean id="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
<property name="schedulerName" value="scheduleOne"/>
<property name="schedulerContextAsMap">
<map>
<entry key="rSchedulerServiceImpl" value-ref="rSchedulerServiceImpl"></entry>
</map>
</property>
<property name="jobDetails">
<list>
<ref bean="quartzSchedulerR" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="cronTriggerR" />
</list>
</property>
</bean> -->
<bean id ="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
<property name="schedulerContextAsMap">
<map>
<entry key="rSSImpl" value-ref="rSSImpl"></entry>
</map>
</property>
</bean>
<bean id="jobDetailFactory" class="org.springframework.beans.factory.config.ObjectFactoryCreatingFactoryBean">
<property name="targetBeanName">
<idref local="jobDetail" />
</property>
</bean>
<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean" scope="prototype">
<property name="jobClass" value="com.scheduler.SMTPMailJob " />
<property name="jobDataAsMap">
<map>
<entry key="rSSeImpl" value-ref="rSSImpl" />
<entry key="fUtil" value-ref="fUtil" />
<entry key="rService" value-ref="rService" />
<entry key="fusion" value-ref="fusion"/>
<entry key="fcproperties" value-ref="fcproperties"/>
</map>
</property>
</bean>
<bean id="jobTriggerFactory"
class="org.springframework.beans.factory.config.ObjectFactoryCreatingFactoryBean">
<property name="targetBeanName">
<idref local="jobTrigger" />
</property>
</bean>
<bean id="jobTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean"
scope="prototype">
</bean>
Currently I comment out the first scheduler and the application works as expected. But if I uncomment it, the second scheduler stops working. Any workaround for this issue.. ? Any help is appreciated.
EDIT: There is no error but I know that the job isn't scheduled. The error is basically that spring doesn't find a unique bean for com.quartz.Scheduler when both the SchedulerFactoryBean are defined. Basically how to configure multiple SchedulerFactoryBean for totally isolated classes.. ?
Please can you explain why do you need two scheduarfactorybeans.
As per spring doc, FactoryBean that creates and configures a Quartz Scheduler, manages its lifecycle as part of the Spring application context, and exposes the Scheduler as bean reference for dependency injection.
This means you cant have multiple instances of the bean.
Also as per your code you are trying to schedule multiple jobs which can be done using one schedularFactoryBean. Create Multiple job beans and their corresponding triggers and add them as list to the schedular factoryBean. All the triggers will be invoked as per configuration irrespective of whether it is a cron trigger or simple trigger.

Heritrix: how to exclude everything but pdf from mirroring?

I found this topic How do i exclude everything but text/html from a heritrix crawl?
I have changed bean to this
<property name="shouldProcessRule">
<bean class="org.archive.modules.deciderules.ContentTypeMatchesRegexDecideRule">
<property name="decision" value="ACCEPT" />
<property name="regex" value="^application/pdf.*"/>
</bean>
</property>
</bean>
But heritrix still saves every file to mirror dir.
I believe you are missing a reject rule above your accept rule. I have the following that works:
<property name="shouldProcessRule">
<bean class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<bean class="org.archive.modules.deciderules.RejectDecideRule">
</bean>
<bean class="org.archive.modules.deciderules.ContentTypeMatchesRegexDecideRule">
<property name="decision" value="ACCEPT" />
<property name="regex" value="^application/pdf.*"/>
</bean>
</list>
</property>
</bean>
</property>
This rejects everything, then accepts everything listed in the following rules.

Resources