I have written a small Spring Boot service using Quartz. I am trying to schedule a job using the code below. However, I am repeatedly getting the following error:
ERROR[my-service] [2017-01-22 17:38:37,328] [] [main] [SpringApplication]: Application startup failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'schedulerFactoryBean' defined in class path resource [com/myservice/configuration/QuartzConfiguration.class]: Invocation of init method failed; nested exception is org.quartz.JobPersistenceException: Couldn't store trigger 'group1.trigger1' for 'group1.job1' job:The job (group1.job1) referenced by the trigger does not exist. [See nested exception: org.quartz.JobPersistenceException: The job (group1.job1) referenced by the trigger does not exist.]
Needless to say, the job never gets scheduled. Here is the relevant code:
#Slf4j
#NoArgsConstructor
public class TimedJob implements Job {
#Override
public void execute(JobExecutionContext context) throws JobExecutionException {
log.debug("**********timed job****************");
}
}
#Configuration
public class QuartzConfiguration {
...
#Inject
MyDao myDao;
#Bean
public SchedulerFactoryBean schedulerFactoryBean(ApplicationContext applicationContext) throws SchedulerException {
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setAutoStartup(true);
factory.setOverwriteExistingJobs(true);
QuartzJobFactory jobFactory = new QuartzJobFactory();
jobFactory.setApplicationContext(applicationContext);
factory.setJobFactory(jobFactory);
factory.setDataSource(dataSource());
factory.setSchedulerContextAsMap(Collections.singletonMap("my_dao", myDao));
JobDetail jobDetail = JobBuilder.newJob()
.ofType(TimedJob.class)
.storeDurably()
.withDescription("desc")
.withIdentity("job1", "group1")
.build();
Calendar calendar = Calendar.getInstance();
calendar.add(Calendar.SECOND, 10);
Trigger trigger = TriggerBuilder
.newTrigger()
.startAt(calendar.getTime())
.withSchedule(
SimpleScheduleBuilder.simpleSchedule().withIntervalInSeconds(1).repeatForever()
)
.forJob(jobDetail)
.withIdentity("trigger1", "group1")
.build();
factory.setTriggers(trigger);
Properties quartzProperties = new Properties();
quartzProperties.setProperty("org.quartz.scheduler.instanceName", instanceName);
quartzProperties.setProperty("org.quartz.scheduler.instanceId", instanceId);
quartzProperties.setProperty("org.quartz.threadPool.threadCount", threadCount);
quartzProperties.setProperty("org.quartz.jobStore.driverDelegateClass", driverDelegateClassName);
factory.setQuartzProperties(quartzProperties);
factory.start();
return factory;
}
This is almost exactly copied from the Quartz tutorials here. What am I doing wrong?
I guess you need setting your jobDetail to your factory instance.
The below URL might help you.
http://www.baeldung.com/spring-quartz-schedule
Related
I am able to run my Springbatch application when Datasource was configured in my SpringbatchConfiguration class. But i dont want Datasource to be configured. SO i used ResourcelessTransactionManager. See below my configuration class. Some one guide me how i can launch Jobs without configuring Datasource as part of Batchjob configurations.
#Configuration
#EnableBatchProcessing
#EnableAutoConfiguration
//#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
public class SprintgBatchConfiguration {
/*#Autowired
private DBConfiguration dbConfig;*/
/*#Autowired
private DataSource dataSource;
#Autowired
private DataSourceTransactionManager transactionManager;
*/
//Tomcat relaated configuration//
#Bean
public MultipartConfigElement multipartConfigElement() {
MultipartConfigFactory factory = new MultipartConfigFactory();
factory.setMaxFileSize("124MB");
factory.setMaxRequestSize("124MB");
return factory.createMultipartConfig();
}
#Bean(name="csvjob")
public Job job(JobBuilderFactory jobBuilderFactory,StepBuilderFactory stepBuilderFactory,ItemReader<List<CSVPojo>> itemReader,ItemProcessor<List<CSVPojo>,CsvWrapperPojo> itemProcessor,AmqpItemWriter<CsvWrapperPojo> itemWriter){
Step step=stepBuilderFactory.get("ETL-CSV").<List<CSVPojo>,CsvWrapperPojo>chunk(100)
.reader(itemReader)
.processor(itemProcessor)
.writer(itemWriter)
.build();
Job csvJob= jobBuilderFactory.get("ETL").incrementer(new RunIdIncrementer())
.start(step).build();
return csvJob;
}
#Bean(name="exceljob")
public Job jobExcel(JobBuilderFactory jobBuilderFactory,StepBuilderFactory stepBuilderFactory,ItemReader<List<ExcelPojo>> itemReader,ItemProcessor<List<ExcelPojo>,ExcelWrapperPojo> itemProcessor,AmqpItemWriter<ExcelWrapperPojo> itemWriter){
Step step=stepBuilderFactory.get("ETL-Excel").<List<ExcelPojo>,ExcelWrapperPojo>chunk(100)
.reader(itemReader)
.processor(itemProcessor)
.writer(itemWriter)
.build();
Job ExcelJob= jobBuilderFactory.get("ETL-Excel").incrementer(new RunIdIncrementer())
.start(step).build();
return ExcelJob;
}
/*#Override
public void setDataSource(DataSource dataSource){
System.out.println("overriden");
}*/
/*#Bean
public FlatFileItemReader<CSVPojo> fileItemReader(Resource resource){
return null;
}*/
/*#Bean(name="dataSource")
public DataSource dataSource() throws SQLException
{
//BasicDataSource dataSource = new BasicDataSource();
return dataSource;
}*/
#Bean(name="transactionManager")
public ResourcelessTransactionManager transactionManager() throws SQLException{
return new ResourcelessTransactionManager();
}
/*#Bean(name="transactionManager")
public DataSourceTransactionManager transactionManager() throws SQLException{
DataSourceTransactionManager transactionManager = new DataSourceTransactionManager(this.dataSource());
return transactionManager;
}*/
/*#Bean
public JobRepository jobRepository() throws Exception{
JobRepositoryFactoryBean factoryBean = new JobRepositoryFactoryBean();
factoryBean.setDatabaseType("ORACLE");
factoryBean.setDataSource(dataSource);
factoryBean.setTransactionManager(transactionManager);
factoryBean.setIsolationLevelForCreate("ISOLATION_READ_UNCOMMITTED");
return factoryBean.getObject();
}*/
#Bean
public JobRepository jobRepository(ResourcelessTransactionManager transactionManager) throws Exception {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
return mapJobRepositoryFactoryBean.getObject();
}
}
But i am getting below exception when i am running Application.
ationConfigEmbeddedWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'fileProcessController': Unsatisfied dependency expressed through field 'jobLauncher'; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration': Unsatisfied dependency expressed through field 'dataSources'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [org/springframework/boot/autoconfigure/jdbc/DataSourceConfiguration$Tomcat.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.apache.tomcat.jdbc.pool.DataSource]: Factory method 'dataSource' threw exception; nested exception is org.springframework.boot.autoconfigure.jdbc.DataSourceProperties$DataSourceBeanCreationException: Cannot determine embedded database driver class for database type NONE. If you want an embedded database please put a supported one on the classpath. If you have database settings to be loaded from a particular profile you may need to active it (no profiles are currently active).
Thanks in advance!!!!
Spring Boot is intended for building production grade applications. When it is used to build a Spring Batch application, it requires a data source to persist Spring Batch meta-data (See BATCH-2704).
But you can always use either:
an embedded datasource supported by Spring Boot (H2, HSQL or Derby) by just adding it to the classpath. This data source will be picked up automatically by Spring Batch
or provide a custom BatchConfigurer and use the MapJobRepository (See here)
Hope this helps.
I am developing Spring Boot Batch Example. In this example, I have created BatchJPA core module which has Entities, JPARepository and DB configurations.
This module adding into another Spring Module as dependency and in this module, I am adding code related specific batch jobs (like custom repository etc). I have total 15 batch jobs and I will be creating separate Spring Boot project with BatchJPA dependency.
10-08-2018 14:54:11.853 [main] WARN org.springframework.context.support.ClassPathXmlApplicationContext.refresh - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'MyTestTasklet': Initialization of bean failed; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No bean named 'transactionManager' available
10-08-2018 14:54:11.855 [main] INFO org.springframework.boot.autoconfigure.logging.ConditionEvaluationReportLoggingListener.logAutoConfigurationReport -
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
10-08-2018 14:54:11.919 [main] ERROR org.springframework.boot.diagnostics.LoggingFailureAnalysisReporter.report -
***************************
APPLICATION FAILED TO START
***************************
Description:
A component required a bean named 'transactionManager' that could not be found.
Action:
Consider defining a bean named 'transactionManager' in your configuration.
Code below:
#Service
public class MyTaskTasklet implements Tasklet {
#Autowired
private MyCustomerCustomRepository myCustomerCustomRepository;
#Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
List<MyTest> mydata = myCustomerCustomRepository.getElligibleData();
if (!mydata.isEmpty()) {
System.out.println("XXXXXX = " + mydata.size());
}
chunkContext.getStepContext().getStepExecution().getJobExecution().getExecutionContext()
.put("listOfData", mydata);
return RepeatStatus.FINISHED;
}
}
Another file:
#Component
#EnableBatchProcessing
public class MyJobLauncher {
public void executeJob() {
String[] springConfig = { "jobs/ABC.xml"};
ApplicationContext context = new ClassPathXmlApplicationContext(springConfig);
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
Job job = (Job) context.getBean("MyJobId");
try {
JobParameters jobParameters = new JobParametersBuilder().addString("runMode", "DATA")
.addDate("date", new Date()).addLong("time", System.currentTimeMillis()).toJobParameters();
jobLauncher.run(job, jobParameters);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
and another file
MainApplication
#SpringBootApplication
#EnableAutoConfiguration(exclude = {DataSourceAutoConfiguration.class,HibernateJpaAutoConfiguration.class})
public class MainApplication implements CommandLineRunner {
#Autowired
private MyJobLauncher jobLauncher;
public static void main(String[] args) {
SpringApplication.run(MyJobLauncher.class, args);
}
#Override
public void run(String... args) throws Exception {
jobLauncher.executeJob();
}
}
I originally asked a question at [Getting "Scope 'step' is not active for the current thread" while creating spring batch beans, and based on the suggestion provided at Spring batch scope issue while using spring boot, I tried to replace #StepScope annotation and instead defined StepScope bean in configuration as below
#Bean
#Qualifier("stepScope")
public org.springframework.batch.core.scope.StepScope stepScope() {
final org.springframework.batch.core.scope.StepScope stepScope = new org.springframework.batch.core.scope.StepScope();
stepScope.setAutoProxy(true);
return stepScope;
}
with this change, I'm not able to pass job parameters while creating beans as, it is throwing
'jobParameters' cannot be found on object of type 'org.springframework.beans.factory.config.BeanExpressionContext'
My configuration is like
#Configuration
#EnableBatchProcessing
public class MyConfig{
#Bean
#Qualifier("partitionJob")
public Job partitionJob() throws Exception {
return jobBuilderFactory
.get("partitionJob")
.incrementer(new RunIdIncrementer())
.start(partitionStep(null))
.build();
}
#Bean
#StepScope
public Step partitionStep(
#Value("#{jobParameters[gridSize]}") String gridSize)
throws Exception {
return stepBuilderFactory
.get("partitionStep")
.partitioner("slaveStep", partitioner())
.gridSize(gridZize)
.step(slaveStep(chunkSize))
.taskExecutor(threadPoolTaskExecutor()).build();
}
#Bean
#StepScope
public Step slaveStep(int chunkSize) throws Exception {
return stepBuilderFactory
.get("slaveStep")
......
........
}
I read that the bean should be annotated with #StepScope,if job Parameters needs to be accessed like my example. But I'm getting exceptions as explained above.
This is how you can pass job parameters
GenericApplicationContext context = new AnnotationConfigApplicationContext(MyConfiguration.class);
JobLauncher jobLauncher = (JobLauncher) context.getBean("jobLauncher");
JobParameters jobParameters = new JobParametersBuilder()
.addString("paramter1", "Test")
.toJobParameters();
Job job = (Job) context.getBean("myJob");
JobExecution execution = jobLauncher.run(job, jobParameters);
Access Job parameters
Bean
#StepScope
public Step partitionStep(
#Value("#{jobParameters['paramter1']}") String gridSize)
throws Exception {
return stepBuilderFactory
.get("partitionStep")
.partitioner("slaveStep", partitioner())
.gridSize(gridZize)
.step(slaveStep(chunkSize))
.taskExecutor(threadPoolTaskExecutor()).build();
}
For my application, we are using the spring's
org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource
The target dataSources are configured and chosen based on request's domain URL.
Eg:
qa.example.com ==> target datasource = DB1
qa-test.example.com ==> target datasource = DB2
Following is the configuration for the same
#Bean(name = "dataSource")
public DataSource dataSource() throws PropertyVetoException, ConfigurationException {
EERoutingDatabase routingDB = new EERoutingDatabase();
Map<Object, Object> targetDataSources = datasourceList();
routingDB.setTargetDataSources(targetDataSources);
return routingDB;
}
public class EERoutingDatabase extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
// This is derived from the request's URL/Domain
return SessionUtil.getDataSourceHolder();
}
}
The task is now using Quartz JDBCJobStore to store the quartz jobs/triggers.
The preferred option is using JobStoreCMT.
We used the following config
#Configuration
public class QuartzConfig {
private static final Logger LOG = LoggerFactory.getLogger(QuartzConfig.class);
private static final String QUARTZ_CONFIG_FILE = "ee-quartz.properties";
#Autowired
private DataSource dataSource;
#Autowired
private PlatformTransactionManager transactionManager;
#Autowired
private ApplicationContext applicationContext;
/**
* Spring wrapper over Quartz Scheduler bean
*/
#Bean(name="quartzRealTimeScheduler")
SchedulerFactoryBean schedulerFactoryBean() {
LOG.info("Creating QUARTZ Scheduler for real time Job invocation");
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setConfigLocation(new ClassPathResource(QUARTZ_CONFIG_FILE));
factory.setDataSource(dataSource);
factory.setTransactionManager(transactionManager);
factory.setJobFactory(springBeanJobFactory());
factory.setWaitForJobsToCompleteOnShutdown(true);
factory.setApplicationContextSchedulerContextKey("applicationContext");
return factory;
}
#Bean
public SpringBeanJobFactory springBeanJobFactory() {
AutoWiringSpringBeanJobFactory jobFactory = new AutoWiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
jobFactory.setIgnoredUnknownProperties("applicationContext");
return jobFactory;
}
}
and following is the config in quartz properties file (ee-quartz.properties)
org.quartz.scheduler.instanceId=AUTO
org.quartz.jobStore.useProperties=false
org.quartz.jobStore.misfireThreshold: 60000
org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
On starting the application, following exception occurs
Caused by: java.lang.IllegalStateException: Cannot determine target DataSource for lookup key [null]
at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.determineTargetDataSource(AbstractRoutingDataSource.java:202) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at com.expertly.config.EERoutingDatabase.determineTargetDataSource(EERoutingDatabase.java:60) ~[EERoutingDatabase.class:na]
at org.springframework.jdbc.datasource.lookup.AbstractRoutingDataSource.getConnection(AbstractRoutingDataSource.java:164) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.doGetConnection(DataSourceUtils.java:111) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:77) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:289) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:329) ~[spring-jdbc-4.1.6.RELEASE.jar:4.1.6.RELEASE]
at org.springframework.scheduling.quartz.LocalDataSourceJobStore.initialize(LocalDataSourceJobStore.java:149) ~[spring-context-support-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.quartz.impl.StdSchedulerFactory.instantiate(StdSchedulerFactory.java:1321) ~[quartz-2.2.2.jar:na]
at org.quartz.impl.StdSchedulerFactory.getScheduler(StdSchedulerFactory.java:1525) ~[quartz-2.2.2.jar:na]
at org.springframework.scheduling.quartz.SchedulerFactoryBean.createScheduler(SchedulerFactoryBean.java:599) ~[spring-context-support-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.scheduling.quartz.SchedulerFactoryBean.afterPropertiesSet(SchedulerFactoryBean.java:482) ~[spring-context-support-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1612) ~[spring-beans-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1549) ~[spring-
beans-4.0.1.RELEASE.jar:4.0.1.RELEASE]
It seems that
Quartz is trying to create connections with my datasource upfront.
Since my dataSource isn't concrete one (its routing dataSource) and in addition doesn't have knowledge to which target Db to connect (at config time), it fails
Do we have any provision, where quartz can be used with RoutingDataSource? If Not, what would be the next best thing?
Ideally you can try making SchedulerFactoryBean as #Lazy.
But It seems lazy initialization will not work bug, there is also a work around listed in the comments.
Create schedulerFactory bean dynamically after
ContextRefreshedEvent received on root context.
Let us know, If this works.
To all Spring enthusiasts, here's a good challenge. Hope someone can crack it!!!
I use Spring to batch the extraction process. I have two classes 'ExtractBatchJob' and 'TaskletImpl'
public class ExtractBatchJob {
/** Logger for current class */
private static Log log = LogFactory.getLog(Extractor.class);
public static void main(String[] args)
throws IOrganizationServiceRetrieveMultipleOrganizationServiceFaultFaultFaultMessage,
IOException {
ApplicationContext context = new ClassPathXmlApplicationContext(
"/META-INF/cxf/batch-context.xml");
SpringBusFactory factory = new SpringBusFactory(context);
Bus bus = factory.createBus();
SpringBusFactory.setDefaultBus(bus);
IOrganizationService service = (IOrganizationService) factory
.getApplicationContext().getBean("service");
JobLauncher jobLauncher = (JobLauncher)context.getBean("jobLauncher");
Job job = (Job) context.getBean("firstBatchJob");
try {
JobExecution execution = jobLauncher.run(job, new JobParameters());
}catch (Exception e){
e.printStackTrace();
}
}
The second class TaskletImpl implements the Spring Tasklet interface.
public class TaskletImpl implements Tasklet {
/** Logger for current class */
private static Log log = LogFactory.getLog(CRMExtractor.class);
/* (non-Javadoc)
* #see org.springframework.batch.core.step.tasklet.Tasklet#execute(org.springframework.batch.core.StepContribution, org.springframework.batch.core.scope.context.ChunkContext)
*/
#Overridepublic RepeatStatus execute(StepContribution arg0, ChunkContext arg1)
throws Exception {
// TODO Auto-generated method stub
log.info("************ CRM Extraction Batch Job is executing!!! *******");
//QUESTION: To Extract Entity from third party
// web service need object reference for
//factory and service from ExtractBatchjob class
List<Entity> orderEntities = getEntities("orderQueryImpl", factory, service);
OrderDao orderDao = (OrderDao) factory.getApplicationContext()
.getBean("orderDao");
orderDao.batchInsert(orderEntities);*/
return RepeatStatus.FINISHED;
}
public static List<Entity> getEntities(String queryImpl, SpringBusFactory factory,
IOrganizationService service)
throws IOrganizationServiceRetrieveMultipleOrganizationServiceFaultFaultFaultMessage,
IOException {
QueryBuilder queryBuilder = (QueryBuilderTemplate) factory
.getApplicationContext().getBean(queryImpl);
QueryExpression queryExpr = queryBuilder.getQuery();
EntityCollection result = service
.retrieveMultiple(queryExpr);
return result.getEntities().getEntities();
}
}
Below is the snippet of the context file
`<import resource="cxf.xml" />
<bean id="firstBatch" class="com.abc.model.TaskletImpl" />
<batch:step id="firstBatchStepOne">
<batch:tasklet ref="firstBatch" />
</batch:step>
<batch:job id="firstBatchJob">
<batch:step id="stepOne" parent="firstBatchStepOne" />
</batch:job>`
My question is quite straightforward, how do I pass the two variables/objects 'service' and 'factory' to the TaskletImpl class from the ExtractBatchJob class.
The cleanest solution is to wire service and factory using Spring injection mechanism.
You have two solution:
Create SpringBusFactory as Spring bean and wire it into tasklet
Define a ContextBean (as singleton) for you job, create SpringBusFactory and set it as property of ContextBean; wire this bean to your tasklet
If you want to use object created outside Spring context (with new I meant) must be injected into Spring context.