We have a web application written on SpringMVC. Also, we have some need of running code through batch application (Spring scheduler). Code re-usability view, we thought to have batch code as well part of the application and then generated JAR out of which.
Is it best way to do so having batch and application code part of the same application. Or do we need separate batch application?
Please advise the best design approach here.
Is both are part of same business? As you have to run Spring Batch in background to do some process, it would be better to keep it separate from Spring MVC Application. Each application should be part of one type of business.
First find out how much dependencies are there for making it separate.
Related
I'm working on a web application that accepts a csv file, parses its contents and stores them in a db.
I'm using spring boot.
I read this tutorial about spring batch.
Question #1
The purpose of Spring Batch is to implement only stand alone programs or could it be efficiently integrated into a web application?
I'm wondering if, for my use case, a combination of FileInputStream and InputStreamReader to parse the file maybe more simple and straight forward, while using Spring Batch could be a little overkill.
Question #2
I didn't find any example, or tutorial, or documentation page that explains how to call (run) a "batch job" from a web application (for example from a method of the controller). In the aforementioned tutorial the job is "hooked" from the application, there's nothing like a job.run(), it just executes when the batch demo application is run. How could it be made? Is there any place where something like my specific use case is explained?
Question #1
Using Spring Batch from a web application may be a good idea because you get all the benefits from Spring Batch.
Question #2
Inject the JobLauncher instance. JobLauncher is the class used to run batch jobs.
Please read the documentation:
https://docs.spring.io/spring-batch/docs/current/reference/html/job.html#runningJobsFromWebContainer
I'm dipping my toes into the microservices, is spring boot batch applicable to the following requirements?
Files of one or multiple are read from a specific directory in Linux.
Several operations like regex, build new files, write the file and ftp to a location
Send email during a process fail
Using spring boot is confirmed, now the question is
Should I use spring batch or just core spring framework?
I need to integrate with Control-M to trigger the job. Can the Control-M be completely removed by using Spring batch library? As we don't know when to expect the files in the directory.
I've not seen a POC with these requirements. Would someone provide an example POC or an affirmation this could be achieved with Spring batch?
I would use Spring Batch for that use case. Not only does it provide out of the box components for reading, processing, and writing files, it adds a lot more for error handling, scalability, etc. All of those things you'd probably end up wiring up by yourself if you go without Spring Batch.
As for being launched via Control-M, yes MANY large customers use Control-M to launch their jobs. Unfortunately, I've never done it myself so I cannot provide any details on the mechanics, but if Control-M can either launch a script or call a REST API, you can launch a job with it.
I would suggest you, go for spring batch as it has much-inbuilt functionality which will be provided to you for file reading and writing to your required location. Even you will be able to handle record skipping requirement. Your mail triggering requirement will be handled by Control M. You just need to decide one exit code for your handled exception and on the basis of that exit code you can trigger the mail to respective members. And there are many other features which will be helpful if you go for spring batch.
Just looking for some information if others have solved this pattern. I want to use Spring Integration and Spring Batch together. Both of these are SpringBoot applications and ideally I'd like to keep them and their respective configuration separated, so they are both their own executable jar. I'm having problems executing them in their own process space and I believe I want, unless someone can convince me otherwise, each to run like they are their own Spring Boot app and initialize themselves with their own profiles and properties. What I'm having trouble with though is the invocation of the job in my SpringBatch project from my SpringIntegration project. At first I couldn't get the properties loaded from the batch project, so I realized I need to pass the spring.active.profiles as a Job Parameter and that seemed to solve that. But there are other things in the Spring Boot Batch application that aren't loading correctly like the schema-platform.sql file and the database isn't getting initialized, etc.
On this initial launch of the job I might want the response to go back to Spring Integration for some messaging on Job Status. There might be times when I want to run a job without Spring Integration kicking off the job, but still take advantage of sending statuses back to the Spring Integration project providing its listening on a channel or something.
I've reviewed quite a few Spring samples and have yet to find my exact scenario, most are with the two dependencies in the same project, so maybe I'm doing something that's not possible, but I'm sure I'm just missing a little something in the Spring configuration.
My questions/issues are:
I don't want the Spring Integration project to know anything about the SpringBatch configuration other than the job its kicking off. I have found a good way to do that reference to the Job Bean without getting my entire batch configuration loading.
Should I keep these two projects separated or would it be better to combine them since I have two-way communication between both.
How should the Job be launch from the integration project. We're using the spring-batch-integration project with JobLaunchRequest and JobLauncher. This seems to run it in the same process as the Spring Integration project and I'm missing a lot of my SpringBootBatch projects initialization
Should I be using a CommandLineRunner instead to force it to another process.
Is SpringApplication.run(BatchConfiguration.class) the answer?
Looking for some general project configuration setup to meet these requirements.
Spring Cloud Data Flow in combination with Spring Cloud Task does exactly what you're asking. It launches Spring Cloud Task applications (which can contain batch jobs) as new processes on the platform you choose. I'd encourage you to check out that project here: http://cloud.spring.io/spring-cloud-dataflow/
I am reading Spring user guide. I came across below statement. I confused by statement "let the framework take care of infrastructure". I mean infrastructure means any Hardware..Nw in Spring Batch is framework, where does infrastructure came in picture
Batch developers use the Spring programming model: concentrate on business logic; let the
framework take care of infrastructure
Please help me in understanding/
If you will read the complete documentation, you will get:
Figure: Spring Batch Layered Architecture
This layered architecture highlights three major high level
components: Application, Core, and Infrastructure. The application
contains all batch jobs and custom code written by developers using
Spring Batch. The Batch Core contains the core runtime classes
necessary to launch and control a batch job. It includes things such
as a JobLauncher, Job, and Step implementations. Both Application and
Core are built on top of a common infrastructure. This infrastructure
contains common readers and writers, and services such as the
RetryTemplate, which are used both by application
developers(ItemReader and ItemWriter) and the core framework itself.
(retry)
spring-batch reference
The Spring Batch framework is designed to cater to batch applications that run on a daily basis in enterprise organizations. It helps to leverage the benefits of the Spring framework along with the advance services. Spring Batch is mainly used to process huge volume of data. It offers better performance and is highly scalable using different optimization and partition techniques. It also provides advantage over logging/tracing, transaction management, job processing statistics, job restart, steps, and resource management. By using the Spring programming model, I can write the business logic and let the framework take care of infrastructure.
Spring Batch includes three components: batch application, batch execution environment and batch infrastructure.
The Application component contains all the batch jobs and custom code written using Spring Batch.
The Core component contains the core runtime classes necessary to launch and control a batch job. It includes things such as a JobLauncher, Job, and Step implementations. Both Application and Core are built on top of a common infrastructure.
The Infrastructure contains readers, writers and services which are used both by application and the core framework itself. They include things like ItemReader, ItemWriter and MongoTemplate. To use the Spring Batch framework, you need only to configure and customize the XML files. All existing core services should be easy to replace or extend, without any impact to the infrastructure layer.
-from Devx
I hope this would help you understand how it works.
Currently I am working on some application which would later require batch functionality module since I have not yet started that module I need to know which (Struts or Springs) has the best Framework for batch processing.
Definitely Springs is better for batch processing as it provide complete spring batch framework for batch processing. Apart from this whatever you are running in struts can be easily integrated to spring frameworks.