I am using JSF-2, Spring 4, hibernate 4 in my application. I have Spring type service layer, Dao Layers , Models and other thing. I want to schedule some of the services which should be automatically executed or called at specified time, usually these services or business logic would perform some kind of data-mapping from excel-file to database.
I want to perform these task without user-intervention and scheduler should take care all these data-mapping.
Note : I am calling these services from my view as well as these services also should be used in scheduler to perform data-mapping.
I am newbie at utmost level, never used any kind of scheduler or anything. So my question :
1)what should I have to use to schedule these task?
2)I am confused regarding Spring Batch and Spring-sheduler? are they both perform scheduling ,if no then what is actual use of sping-batch?
3)Can spring-scheduler itself sufficient enough to perform these scheduling
Any help would be highly considerable.
1)what should I have to schedule these task?
Basically you need the classes that support the operations that you want to do (excel creation from database queries), spring in both cases.
2)I am confused regarding Spring Batch and Spring-sheduler? are they both perform scheduling ,if know then what is actual use of sping-batch?
Spring Batch provides reusable functions that are essential in
processing large volumes of records, including logging/tracing,
transaction management, job processing statistics, job restart, skip,
and resource management. It also provides more advanced technical
services and features that will enable extremely high-volume and high
performance batch jobs though optimization and partitioning techniques
Spring scheduler just run any method at certain time, it is not so robust, and only execute the logic involve on a process, not statistic, not job restart, just start a process during predefined period of time (calling a method of a class)
3)Can spring-scheduler itself sufficient enough to perform these scheduling?
Yes it is, if you are not very related with spring-batch this will take more time that just call the methods you already have.
Scheduler A scheduler is a software product that allows an enterprise
to schedule and track computer batch tasks
Scheduler just ran the process.
Related
I have a requirement to have a schedule batch that will identify what are the batches I need to restart or re-submit(as new job instance). Schedule batch will identify and call SCDF API to launch tasks. Is it really good design pattern to have a such batch ?
I can implement the above require pattern but is is good practice or anyone can suggest what is alternate way of doing it.
I see nothing wrong in implementing that requirement with Spring Batch (it would be a single step job with a simple tasklet). The question you should be asking is that do you really need a Spring Batch job for that? What would you gain from using Spring Batch? Most interesting features won't be used for that job (restartability, fault-tolerance, transaction management, etc), so the benefit/cost ratio is low IMO.
I have seen folks putting similar logic (db query + rest call) in a shell script scheduled with crontab. And I see nothing wrong with this approach neither.
So it really depends on how you want to implement that requirement.
I was reading about Spring Batch and I read the below:
Spring Batch is not a scheduling framework. There are many good
enterprise schedulers (such as Quartz, Tivoli, Control-M, etc.)
available in both the commercial and open source spaces. It is
intended to work in conjunction with a scheduler, not replace a
scheduler.
Source: https://docs.spring.io/spring-batch/docs/current/reference/html/spring-batch-intro.html#springBatchBackground
So what is the difference between Spring Batch and Tivoli?
Spring Batch is mainly designed to provide a runtime for java batch workload.
IBM Workload Scheduler (Tivoli) / HCL Workload Automation, like other schedulers, doesn't run the workload directly, but is used to triggers any kind of workload (jobs), including Spring Batch, on on-prem or hybrid and multi cloud environments, including Kubernetes.
It can trigger jobs based on calendar, time, considering free/working day, complex runcycles (e.g. 3 working days before the end of each month).
In addition it can trigger workload based on dependencies on other jobs, so that they can start as soon as the previous job (running on any other system) has completed successfully, or run jobs only if predecessor has completed with a specific RC or result. Or you can use logic resources and limits to control how many jobs using the same machine or resource can run at the same time.
It can be also used to trigger workload based on events, e.g. when a new file is uploaded.
In recent releases IBM Workload Scheduler / HCL Workload Automation also added built-in capabilities to transfer files.
IBM Workload Scheduler / HCL Workload Automation is also key to have a centralized monitoring and recovery of failures, to centralize security granting access to different teams only on their jobs, to have a centralized governance (e.g. auditing any change and recovery on jobs).
It's also able to forecast the job durations and when every job will run, and generate alerts if they are running too long or if based on predecessors they are expected to miss their deadline.
I am reading different posts and books on Microservice Architecture in the hunt to answer my question which is related to the Decomposition Strategies. The question is, should we create a new microservice specifically to handle the batch job?
To my context, the nature of the batch job is to read the data from the database and make REST calls to external system if the data is in the particular state. Additionally, the batch job is suppose to run only once a day.
My questions related to this are
Is this an industry norm/practice that when we have to run BATCH job, it should be a new microservice because batch job consumes resources which can hinder the incoming traffic and increases latency.
Does running a batch job effect the latency of the APIs exposed towards client?
I would say yes, it makes sense. Usually batch jobs have very different development lifecycle and deployment frequency.
I've done something similar by myself and I'm totally sure it's worth it.
Also It would then possible to spin instance to run job once a day - which can save money in cloud environments.
Latency: it depends on that other system. You might want to come with throttling your requests to other system, to not put it down under the heavy load.
How can I configure a frequency based schedule on Azure Batch Service (ex. hourly/daily/weekly job)?
I suppose Azure Batch Service has job scheduling features, but couldn't find a time based scheduling descriptions, although I found this page that describes dependency based scheduling based on task dependency graph.
Preface: since you didn't specify an SDK language for context, I will reference the REST API documentation for the answer.
You will want to use Job Schedules to schedule recurrences. You will need to define, at the minimum, the following:
Specify a Schedule which will determine how often a job will run along with constraints on when it can run.
Specify a Job Specification as per normal (which can include what the job's target pool or autopool, constraints, job manager task if required, job prep/release if required, etc).
We have a Spring + JPA web application.
We use two tomcat servers that run both application and uses the same DB.
One of our application requirmemnt is to preform cron \ scheduled tasks.
After a short research we found that spring framework delivers a very straight forward solution to cron jobs,
(Annotation based solution)
However since both tomcats running the same webapp - if we will use this spring's solution we will create a very problematic scenario where 2 crons are running at the same time (each on a different tomcat)
Is there any way to solve this issue? maybe this alternative is not good for our purpose?
thanks!
As a general rule, you're going to want to save a setting to indicate that a job is running. Similar to how "Spring Batch" does the trick, you might want to create a table in your database simply for storing a job execution. You can choose to implement this however you'd like, but ultimately, your scheduled tasks should check the database to see if an identical task is already running, and if not, proceed with the task execution. Once the task has completed, update the database appropriately so that a future execution will be able to proceed.
#kungfuters solution is certainly a better end goal, but as a simple first implementation, you could use a property to enable/disable the tasks, and only have the tasks run on one of the servers.