Creating multiple SchedulerFactoryBean in Quartz - spring

I have run into a problem where I have two classes extending QuartzJobBean. The problem I am facing is to create two SchedulerFactoryBean. I did my research and found that setting the property schedulerName solves the problem. This did not work for me. I used #Qualifier also. If I create the two beans of SchedulerFactoryBean , Spring doesn't know which scheduler to refer to. I have two CronTriggers.
Code:
<!--
<bean name="quartzSchedulerR" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.task.QuartzScheduler" />
<property name="jobDataAsMap">
<map>
<entry key="rRSImpl" value-ref="rRSService" />
<entry key="SRObject" value-ref="SRObject"/>
</map>
</property>
</bean>
<bean id="cronTriggerR"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="quartzSchedulerR" />
<property name="cronExpression" value="0 30 12 ? * MON *" />
</bean>
<bean id="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
<property name="schedulerName" value="scheduleOne"/>
<property name="schedulerContextAsMap">
<map>
<entry key="rSchedulerServiceImpl" value-ref="rSchedulerServiceImpl"></entry>
</map>
</property>
<property name="jobDetails">
<list>
<ref bean="quartzSchedulerR" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="cronTriggerR" />
</list>
</property>
</bean> -->
<bean id ="quartzScheduler" class="org.springframework.scheduling.quartz.SchedulerFactoryBean" lazy-init="false">
<property name="schedulerContextAsMap">
<map>
<entry key="rSSImpl" value-ref="rSSImpl"></entry>
</map>
</property>
</bean>
<bean id="jobDetailFactory" class="org.springframework.beans.factory.config.ObjectFactoryCreatingFactoryBean">
<property name="targetBeanName">
<idref local="jobDetail" />
</property>
</bean>
<bean id="jobDetail" class="org.springframework.scheduling.quartz.JobDetailBean" scope="prototype">
<property name="jobClass" value="com.scheduler.SMTPMailJob " />
<property name="jobDataAsMap">
<map>
<entry key="rSSeImpl" value-ref="rSSImpl" />
<entry key="fUtil" value-ref="fUtil" />
<entry key="rService" value-ref="rService" />
<entry key="fusion" value-ref="fusion"/>
<entry key="fcproperties" value-ref="fcproperties"/>
</map>
</property>
</bean>
<bean id="jobTriggerFactory"
class="org.springframework.beans.factory.config.ObjectFactoryCreatingFactoryBean">
<property name="targetBeanName">
<idref local="jobTrigger" />
</property>
</bean>
<bean id="jobTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean"
scope="prototype">
</bean>
Currently I comment out the first scheduler and the application works as expected. But if I uncomment it, the second scheduler stops working. Any workaround for this issue.. ? Any help is appreciated.
EDIT: There is no error but I know that the job isn't scheduled. The error is basically that spring doesn't find a unique bean for com.quartz.Scheduler when both the SchedulerFactoryBean are defined. Basically how to configure multiple SchedulerFactoryBean for totally isolated classes.. ?

Please can you explain why do you need two scheduarfactorybeans.
As per spring doc, FactoryBean that creates and configures a Quartz Scheduler, manages its lifecycle as part of the Spring application context, and exposes the Scheduler as bean reference for dependency injection.
This means you cant have multiple instances of the bean.
Also as per your code you are trying to schedule multiple jobs which can be done using one schedularFactoryBean. Create Multiple job beans and their corresponding triggers and add them as list to the schedular factoryBean. All the triggers will be invoked as per configuration irrespective of whether it is a cron trigger or simple trigger.

Related

Use of Heritrix's HtmlFormCredential and CredentialStore

I am attempting to add authentication to my Heritrix configuration. My .cxml file has the following:
<bean id="preconditions" class="org.archive.crawler.prefetch.PreconditionEnforcer">
<property name="credentialStore">
<ref bean="credentialStore" />
</property>
</bean>
<bean id="fetchHttp" class="org.archive.modules.fetcher.FetchHTTP">
<property name="credentialStore">
<ref bean="credentialStore" />
</property>
<property name="shouldProcessRule">
<bean class="org.archive.modules.deciderules.DecideRuleSequence">
<property name="rules">
<list>
<bean class="org.archive.modules.deciderules.recrawl.IdenticalDigestDecideRule">
<property name="decision" value="REJECT" />
</bean>
<bean class="org.archive.modules.deciderules.ResourceNoLongerThanDecideRule">
<property name="contentLengthThreshold" value="54" />
<property name="useHeaderLength" value="true" />
<property name="decision" value="REJECT" />
</bean>
</list>
</property>
</bean>
</property>
</bean>
<bean id="exampleCredential" class="org.archive.modules.credential.HtmlFormCredential">
<property name="domain" value="example.com" />
<property name="loginUri" value="https://example.com/user?destination=%2f" />
<property name="formItems">
<map>
<!-- username/password -->
<entry key="name" value="something#something.com"/>
<entry key="pass" value="genericpassword"/>
<!-- hidden inputs -->
<entry key="form_build_id" value="form-asdf" />
<entry key="form_id" value="user_login" />
<!-- submit -->
<entry key="op" value="submit"/>
</map>
</property>
</bean>
<bean id="credentialStore" class="org.archive.modules.credential.CredentialStore">
<property name="credentials">
<map>
<entry key="exampleCredential" value-ref="exampleCredential" />
</map>
</property>
</bean>
I also set the logging for FetchHTTP and PreconditionEnforcer to FINE, but nothing seems to be happening. No logging output is appearing from either modules and the pages that are pulled down clearly are those of an un-authenticated view.
I find it somewhat unclear as to how to use the CredentialStore, considering that I've spent a good amount of time reading through the specifications, which are patchy at best when it comes to authentication and websites.
Anyone know how to set up authentication in Heritrix, please help.
Update:
Logging didn't work because eclipse didn't know about my HERITRIX_HOME variable, so it never even read the logging configuration file.
I changed the bean exampleCredential's domain property from:
<property name="domain" value="example.com" />
to:
<property name="domain" value="www.example.com" />
and now the login page is enqueued, but now the logger spits out the following for all queued files:
org.archive.crawler.prefetch.PreconditionEnforcer.innerProcessResult() PolitenessEnforcer doesn't understand uri's of type dns (ignoring)
org.archive.modules.deciderules.ResourceNoLongerThanDecideRule.evaluate() Error: Missing HttpMethod object in CrawlURI. dns:secure.www.example.com
and none of the files are downloaded or crawled. So though I made progress, it didn't lead me anywhere. There is not much logging information to go off of.
I also asked this question on the Heritrix forms: http://tech.groups.yahoo.com/group/archive-crawler/message/8235 and Noah Levitt had the idea to add the login page as a seed to my crawl. Everything now seems to be working without much issue.
My conclusion is that I had everything set up correctly in my config file, but was missing the actual page seed that I needed.

How to force beans in quartz scheduler to be initialized on start up?

<bean name="MyJob" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="jobClass" value="com.package.scheduler.MyJob"/>
<property name="jobDataAsMap">
<map>
<entry key="numArr" value="10,20,30"/>
</map>
</property>
</bean>
I have a quartz job defined that runs this at 1:30am. The problem is this array definition does not work
<entry key="numArr" value="10,20,30"/>
We had to use list instead. Since the beans do not get initialized until the quartz job kicks in and it's unlikely we would be coding at 1:30am, we cannot catch this error locally. The only way right now is to change the quartz job's running time to force it to happen. We want to force these beans to be initialized on startup when we are running on local dev. Is there a way to do this?
If possible I would decouple your bean from the Quartz job. You could then intialise your bean separately in your spring configuration and it does not need to be made aware of Quartz. I would then use the spring MethodInvokingJobDetailFactoryBean to call a method on your decoupled object.
For example something similar to this:
<bean id="myBean class="...">
<property name="numArr" value="10,20,30"/>
</bean>
<bean id="myJob"
class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="myBean" />
<property name="targetMethod" value="doSomething" />
</bean>
<bean id="staleTraderRatesTrigger"
class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="myJob" />
<property name="cronExpression" value="0 30 1 * * ?" />
</bean>

quartz jobs in a cluster running on wrong node

uI have an application that is clustered in a 2+2 setup where 2 machines are used in alpha product phase, and the other two are used for real customers. all machines are looking at the same database,
I need a job to run at midnight for each of the groups. one of the two alpha machines should get a job, and the other two should get another job.
I'm using Spring 3.0.5 with Quartz 1.8.5 with the following properties
phase=alpha
quartz.job.name=MY_JOB_${phase}
<bean id="quartzPropertiesFactoryBean" class="com.liveperson.kwo.quartz.QuartzPropertiesFactory">
<constructor-arg value="AUTO"/>
<constructor-arg value="MY_CLUSTER"/>
<constructor-arg value="JobStoreTX"/>
<constructor-arg value="StdJDBCDelegate"/>
<constructor-arg value="true"/> //isClustered
<constructor-arg value="false"/> //useProperties
</bean>
<bean name="runJobBean" class="org.quartz.JobDetail">
<property name="name" value="${quartz.job.name}"/>
<property name="jobClass" value="CLASS1"/>
<property name="group" value="JOB-for-${quartz.job.name}"/>
<property name="jobDataMap">
<bean class="org.quartz.JobDataMap">
</bean>
</property>
</bean>
<bean name="schedulerFactoryBean" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="quartzProperties" ref="quartzPropertiesFactoryBean"/>
<property name="dataSource" ref="mySqlConnectorBean"/>
<property name="overwriteExistingJobs" value="true"/>
<property name="jobDetails">
<list>
<ref bean="runJobBean"/>
</list>
</property>
<property name="triggers">
<list>
<ref bean="cronTriggerBean"/>
</list>
</property>
</bean>
I define two jobs, one for alpha and another for production throw the phase property, and the problem I'm having is that the job I define for alpha phase runs on the node defined to production, how can I make the job defined for the alpha phase run only on machines defined for alpha?
Thanks!

Tomcat deploy my project twice

Tomcat deploy my project twice. is anybody faced and solved this issue?. This is spring project and I am using quartz scheduler. because of this problem quartz running twice.
Using Quartz-2.1.1 and Spring framework 3.1.1
The quartz configuration is as follows :
<bean id="simpleTrigger" class="org.springframework.scheduling.quartz.SimpleTriggerFactoryBean">
<property name="jobDetail" ref="jobDetail" />
<property name="startDelay" value="10000" />
<property name="repeatInterval" value="1000" />
</bean>
<bean id="sc" class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="simpleTrigger" />
</list>
</property>
</bean>
<bean id="jobDetail" class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
<property name="targetObject" ref="staFromInQObj" />
<property name="targetMethod" value="processInQueueData" />
<property name="concurrent" value="false" />
</bean>
I think this is the correct answer for the problem.
Spring 3 MVC dispatcher xml and applicationContext xml
Double deployments can happen due to spring configs as well !

Spring/JTA/JPA DAO integration test doesn't rollback?

My DAO integration tests are failing because entities created during the tests are still in the database at the start of the next test. The exact same behavior is seen from both MySQL 5 and H2.
The test classes are annotated with:
#Transactional
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration( { "/testPersist-applicationContext.xml" })
The transaction bean configuration in the test application context is as follows:
<tx:annotation-driven />
<bean id="transactionManager"
class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="transactionManager" ref="atomikosTransactionManager" />
<property name="userTransaction" ref="atomikosUserTransaction" />
</bean>
<bean id="atomikosTransactionManager" class="com.atomikos.icatch.jta.UserTransactionManager"
init-method="init" destroy-method="close">
<property name="forceShutdown" value="false" />
</bean>
<bean id="atomikosUserTransaction" class="com.atomikos.icatch.jta.UserTransactionImp">
<property name="transactionTimeout" value="300" />
</bean>
The entity manager is configured as follows:
<bean id="myappTestLocalEmf"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="myapp" />
<property name="persistenceUnitPostProcessors">
<bean class="com.myapp.core.persist.util.JtaPersistenceUnitPostProcessor">
<property name="jtaDataSource" ref="myappPersistTestJdbcDataSource" />
</bean>
</property>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="false" />
<property name="database" value="$DS{hibernate.database}" />
<property name="databasePlatform" value="$DS{hibernate.dialect}" />
</bean>
</property>
<property name="jpaProperties">
<props>
<prop key="hibernate.transaction.manager_lookup_class">com.atomikos.icatch.jta.hibernate3.TransactionManagerLookup
</prop>
<prop key="hibernate.format_sql">true"</prop>
<prop key="hibernate.use_sql_comments">true</prop>
</props>
</property>
</bean>
<context:annotation-config />
Everything in the log files seem to be fine...I can see messages from Spring about rollback and from Atomikos about rollback as well. But frankly, the logs are so huge and so complex, I could easily be missing something...
Yet the inserted test data remains! Any clues?
It turned out that my C3P0 JDBC data source was not XA aware and was therefore not participating in the transaction. Why I did not get an error, nor a warning in the log file, I do not know. Nonetheless, the reason you cannot use an XA aware data source is explained very nicely here. Note that the data source does not need to be XA capable...just XA aware.
Replacing the C3P0 data source with the following one solved the problem.
<bean id="myappJTANonXADataSource" class="com.atomikos.jdbc.nonxa.AtomikosNonXADataSourceBean">
<property name="uniqueResourceName" value="myappDatabase" />
<property name="driverClassName" value="$DS{hibernate.connection.driver_class}" />
<property name="url" value="$DS{hibernate.connection.url}" />
<property name="user" value="$DS{hibernate.connection.username}" />
<property name="password" value="$DS{hibernate.connection.password}" />
<property name="maxPoolSize" value="20" />
<property name="reapTimeout" value="300" />
</bean>
I think you will need to go through the logs in details. It could be that the rollbacks you are seeing are working, except that something else has executed a commit first. I also cannot see anything in your code which indicates automated rollback. And that it should occur at the end of each test. If you are depending on a timeout based rollback it could be that the second test is running before the timeout occurs, therefore it sees the data before it is rolled back.
Many options here there is :-)

Resources