How do i pass values which are fetched from database in one job as parameters to another job in Spring framework?PLese provide a example code.
I'm guessing by jobs you mean scheduled-tasks. I was in the same situation where 2 jobs were dependent on one another eg:
<task:scheduled-tasks>
<task:scheduled ref="deleteExpiredAccountsJob" method="launch" fixed-delay="100000" />
</task:scheduled-tasks>
<task:scheduled-tasks>
<task:scheduled ref="emailDeletedAccountsConfirmationJob" method="launch" fixed-delay="100000" />
</task:scheduled-tasks>
What I did was to set a DELETE flag on the ACCOUNTS table to true and have the emailDeletedAccountsConfirmationJob read only the ACCOUNTS with DELETE = true
Another solution, if you want to share data between ItemReaders would be to be to have a Java Spring configuration class "#Configuration" and declare your ItemReaders in this class with #Bean annotation and also have a private synchronized Map or List which will be shared between all the threads/jobs or your spring batch steps and you access the synchronized list in your readers.
#Configuration
public class MySpringConfig {
private List<String> list = Collections.synchronizedList(new ArrayList<String>());
#Bean
#Scope("step")
public JdbcCursorItemReader readerOne(){
//add data to list
}
#Bean
#Scope("step")
public JdbcCursorItemReader readerTwo(){
//check for certain data in the list, if exists ... do something
}
in Spring Batch Jobs are considered distinct use cases with no conceptual relationship imposed by the framework. you could see a job as a standalone processing application.
a typical batch system consists of data sources and data sinks such as a message queue, database or file system, so one job is producing data, that is intended to be process by another part of your application.
you may rework your design to use tasks instead of jobs. this is useful if you have several interdependent process steps. there is a way to access the JobExcecution from StepExecution.
I recommend reading the excellent articles as well as "Spring Batch in Action".
Related
I am new to Spring Batch and I would like to seek the advice of the experienced programmer on how to implement the following:
I am writing an API that triggers a batch task to generate a report from a list of transactions from JPA repo.
In spring batch, there is ItemReader, ItemProcessor, ItemWriter for each step.
How should I implement my ItemReader such that it can execute the following:
Generate a custom query from the parameters set in the jobParameters (obtained from the API query)
Using the custom query in 2 to obtain the list of transactions from the Transactions JPA repo, which will be processed in the ItemProcessor and subsequently generate reports through ItemWriter.
Question: How should I write the ItemReader, I looking at JpaCursorItemReader (dk if it is the right one) but could not find examples of implementation online to refer to. Any help is appreciated. Thank you.
I am at the stage of trying to understand how Spring batch works, I hope to seek proper guidance from experts in this field of the direction to go to accomplished the mentioned task.
Generate a custom query from the parameters set in the jobParameters (obtained from the API query)
You can define a step-scoped bean and inject job parameters to use them in your query:
#Bean
#StepScope
public JpaCursorItemReader jpaCursorItemReader(#Value("#{jobParameters['parameter']}") String parameter) {
return new JpaCursorItemReaderBuilder()
.queryString("..") // use job parameter in your query here
// set other properties on the item reader
.build();
}
You can find more details on step-scoped components in the documentation here: Late Binding of Job and Step Attributes.
I need to create a spring batch wherein i need to fetch data from 2 different tables.
This will be the reader part.
The problem is i need to have 2 reader queries. And only one of them will be called based on the condition.
But the writer will be same.
So tasklet becomes same basically.
Can i have 2 readers inside one single step which will be called according to the condition..??
Something like inside the spring batch xml:
if (condition)
then reader1
else reader2
...
....
......
<reader1 id=".." class="..">
</reader1>
.....
........
<reader2 id=".." class="..">
</reader2>
Have you considered using the conditional control flow patterns Spring Batch offers? I think it could lead to simpler construction and adhere to some of the core patterns spring batch encourages.
You basically program the "condition" that you want a step to be called under. So define two steps, say step1 which has a reader, processor, writer for one object type and step2 which has a reader, processor, writer for the other type. Perhaps the writers are shared between types as well. You then can define control flow like so:
#Bean
public Job myJob() {
JobBuilder jobBuilder = jobs.get("Spring Batch: conditional steps");
return jobBuilder.start(determineRoute())
.on("item1").to(flow1())
.on("item2").to(flow2()).end()
.build();
}
In the example determineRoute() is a tasklet that returns custom ExitStatus values item1 or item2 & flow1 and flow2 are different flows (or steps) to handle each object.
See here in their docs: https://docs.spring.io/spring-batch/docs/current/reference/html/step.html#conditionalFlow
Edit: you can also do something similar with a JobExecutionDecider https://www.baeldung.com/spring-batch-conditional-flow#2-programmatic-branching-withjobexecutiondecider
UPDATE: I just published this question also here, I might have done a better work phrasing it there.
How can I explicitly define an order in which Spring's out-of-the-box process of reading properties out of an available-in-classpath application.yml will take place BEFORE my #Configuration annotated class which reads configuration data from zookeeper and places them as system properties which are later easily read and injected into members using #Value?
I have a #Configuration class, which defines a creation of a #Bean, in a which configuration data from zookeeper is read and placed as system properties, in a way that they can easily be read and injected into members using #Value.
#Profile("prod")
#Configuration
public class ZookeeperConfigurationReader {
#Value("${zookeeper.url}")
static String zkUrl;
#Bean
public static PropertySourcesPlaceholderConfigurer zkPropertySourcesPlaceHolderConfigurer() {
PropertySourcesConfigurerAdapter propertiesAdapter = new PropertySourcesConfigurerAdapter();
new ConfigurationBuilder().populateAdapterWithDataFromZk(propertiesAdapter);
return propertiesAdapter.getConfigurer();
}
public void populateAdapterWithDataFromZk(ConfigurerAdapter ca) {
...
}
}
Right now I pass the zookeeper.url into the executed program using a -Dzookeeper.url which is added to the execution line. Right now I read it by calling directly System.getProperty("zookeeper.url").
Since I'm using Spring-Boot application, I also have a application.yml configuration file.
I would like to be able to set the zookeeper.url in the application.yml, and keep my execution line clean as possible from explicit properties.
The mission turns out to be harder than I thought.
As you can see in the above code sniplet of ZookeeperConfigurationReader, I'm trying to inject that value using #Value("${zookeeper.url}") into a member in the class which performs the actual read of data from zookeeper, but at the time the code that needs that value accesses it, it is still null. The reason for that is that in spring life cycle wise, I'm still in the phase of "configuration" as I'm a #Configuration annotated class myself, and the spring's code which reads the application.yml data and places them as system properties, hasn't been executed yet.
So bottom line, what I'm looking for is a way to control the order and tell spring to first read application.yml into system properties, and then load ZookeeperConfigurationReader class.
You can try to use Spring Cloud Zookeeper. I posted a brief example of use here
I'm using Spring and have created a web application. In my web app I've a rest service. I have one rest method called process which takes in a user's details (from an angular ui) and saves the users details to a database (SQLite)
Basically what I want to do is that when a user initiates a rest call. I want to start a separate thread (of only which one will exist). This thread will poll a database for certain values and emails an administrator if certain values are found. I need the class to be thread safe. See the below. Am I correct in using something like this ?. Do I need an #Async annotation ? Or should i use a TimerTask instead ?
#EnableScheduling
public class DBPoller {
#Scheduled(fixedRate = 5000)
public void checkDatabase() {
//checks the db for certain values
}
}
You must write
#EnableScheduling
in the Main Class of the Application
and the DBPoller class must be an Component of the Spring Framework.
So you must add the Annotation #Component (or #Service) to the Head of the DBPoller Class
It sounds like you do want to use an #Async annotation. #Scheduled won't really achieve the effect you are trying to achieve. #Scheduled would work if you were trying to run this check/email scenario on fixed time intervals, rather than on user request. Luckily the config for both is the same.
<task:annotation-driven scheduler="scheduler"
executor="asyncMethodTaskExecutor" />
<task:scheduler id="scheduler" pool-size="x" />
<bean id="asyncMethodTaskExecutor"
class="org.springframework.scheduling.quartz.SimpleThreadPoolTaskExecutor">
<property name="threadCount" value="y"/>
<property name="threadNamePrefix" value="AsyncMethodWorkerThread"/>
</bean>
If you have the #EnableScheduling annotation, you don't need to define <task:scheduler id="scheduler" pool-size="x" />, but personally I prefer the XML configuration because if you want to change your thread pool size you only have to edit the XML values and restart your application, not recompile and redeploy the whole thing.
Make sure you change x and y to suitable values. This will depend on how many concurrent users you may have on your system.
You also need to make sure that your class is discoverable to the Spring context, and that this method is implementing an interface so that Spring can generate a proxy of it to actually invoke asynchronously like the below example
public interface AsyncService {
public void checkDatabase();
}
#Service
public class AsyncServiceImpl implements AsyncService {
#Override
#Async
public void checkDatabase(){
//Do your database check here.
}
}
You also need to make sure that the package your service is in can be found by Spring, double check your <context:component-scan> value.
Happy asynchronous execution.
I'm a beginner in hibernate 4 & Spring 3.2 stuffs.
I have read some tutorials and discussion on stack but i don't find a clear answer to my questions. And i think the best way to understand is to ask and share knowledges !
Here we go!
So you create each time a Pojo, a Dao , a Service class, with methods annotated transactionnal. That's ok. I'm using Sessionfactory to handle my transaction. I'm looking for good practices.
1- If you want to use Delete Method and Save Method from the same Service, how will you do to make it works in a same transaction. When i look at the log, each method are executed in different transactions.
This SampleServiceImpl:
#Transactional
public void save(Sample sample){
sampleDao.save(sample);
}
#Transactional
public void delete(Sample sample){
sampleDao.delete(sample);
}
// A solution could be that , but not very clean...there should be an another way, no?
#Transactional
public void action(Sample sample){
sampleDao.save(sample);
sampleDao.delete(sample);
}
2- If you want to use Delete Method and Save Method from different Services class, how will you do to make it works in a same transaction. Because each method in each service class is handled by a Transactionnal annotation. Do you create a global Service calling all subservice in one method annoted Transactional
SampleServiceImpl:
#Transactional
public void save(Sample sample){
sampleDao.save(sample);
}
ParcicipantServiceImpl
#Transactional
public void save(Participant participant){
participantDao.save(participant);
}
// A solution could be that , but not very clean...there should be an another way, no?
GlobalServiceImpl
#Transactional
public void save(Participant participant,Sample sample){
participantDao.save(participant);
sampleDao.save(sample);
}
3- And the last question but not the least .If you want to use several Methods from severals service in one global transaction. Imagine you want to fill up 5 or more table in one execution of a standalone program. How is it possible because each Service to have his proper transactional method, so each time you called this method, there is a transaction.
a- I have successfully arrive to fill up two tables in a sample transaction using Mkyong tutorial and cascade property in the mapping. So i see how to make it works for one table directly joined to one another or more tables.
b- But if you have a 3 tables Participant -> Samples -> Derived Products. How will you fill up the three tables in a same transaction.
I don't know if i'm clear. But i would appreciated some help or example on that from advanced users.
Thanks a lot for you time.
Your solution is fine, maybe this works if you want to using nested transactional methods(note I saw this solution couple days ago and didn't test it):
< tx:annotation-driven mode="aspectj" / >
< context:load-time-weaver aspectj-weaving="on"/ >
#Transactional
public void action(Sample sample){
save(sample);
delete(sample);
}
Transaction should propagate.
GlobalServiceImpl
#Transactional
public void save(Participant participant,Sample sample){
participantDao.save(participant);
sampleServiceImpl.save(sample);
}
The approch you are following is cleaner approch,
ServiceOpjects are ment to contain business logic. Hence they will always manuplate through data objects.
What we do in practise is create a another layer that uses dataObjects and and other functional call of same layer. Then this all business layer is called via service layer having annotation #transactional.
Can you please mention why you think this approch is dirty??