I'm working on a web application that accepts a csv file, parses its contents and stores them in a db.
I'm using spring boot.
I read this tutorial about spring batch.
Question #1
The purpose of Spring Batch is to implement only stand alone programs or could it be efficiently integrated into a web application?
I'm wondering if, for my use case, a combination of FileInputStream and InputStreamReader to parse the file maybe more simple and straight forward, while using Spring Batch could be a little overkill.
Question #2
I didn't find any example, or tutorial, or documentation page that explains how to call (run) a "batch job" from a web application (for example from a method of the controller). In the aforementioned tutorial the job is "hooked" from the application, there's nothing like a job.run(), it just executes when the batch demo application is run. How could it be made? Is there any place where something like my specific use case is explained?
Question #1
Using Spring Batch from a web application may be a good idea because you get all the benefits from Spring Batch.
Question #2
Inject the JobLauncher instance. JobLauncher is the class used to run batch jobs.
Please read the documentation:
https://docs.spring.io/spring-batch/docs/current/reference/html/job.html#runningJobsFromWebContainer
Related
We have a web application written on SpringMVC. Also, we have some need of running code through batch application (Spring scheduler). Code re-usability view, we thought to have batch code as well part of the application and then generated JAR out of which.
Is it best way to do so having batch and application code part of the same application. Or do we need separate batch application?
Please advise the best design approach here.
Is both are part of same business? As you have to run Spring Batch in background to do some process, it would be better to keep it separate from Spring MVC Application. Each application should be part of one type of business.
First find out how much dependencies are there for making it separate.
I'm dipping my toes into the microservices, is spring boot batch applicable to the following requirements?
Files of one or multiple are read from a specific directory in Linux.
Several operations like regex, build new files, write the file and ftp to a location
Send email during a process fail
Using spring boot is confirmed, now the question is
Should I use spring batch or just core spring framework?
I need to integrate with Control-M to trigger the job. Can the Control-M be completely removed by using Spring batch library? As we don't know when to expect the files in the directory.
I've not seen a POC with these requirements. Would someone provide an example POC or an affirmation this could be achieved with Spring batch?
I would use Spring Batch for that use case. Not only does it provide out of the box components for reading, processing, and writing files, it adds a lot more for error handling, scalability, etc. All of those things you'd probably end up wiring up by yourself if you go without Spring Batch.
As for being launched via Control-M, yes MANY large customers use Control-M to launch their jobs. Unfortunately, I've never done it myself so I cannot provide any details on the mechanics, but if Control-M can either launch a script or call a REST API, you can launch a job with it.
I would suggest you, go for spring batch as it has much-inbuilt functionality which will be provided to you for file reading and writing to your required location. Even you will be able to handle record skipping requirement. Your mail triggering requirement will be handled by Control M. You just need to decide one exit code for your handled exception and on the basis of that exit code you can trigger the mail to respective members. And there are many other features which will be helpful if you go for spring batch.
We have Spring4 and Spring Batch 3 and our app consumes CSV files as input file. Currently we kick off the jobs manually from the command line, using CommandLineJobRunner with parms, including the name of the file to process.
I want to kick off a job to process asynchronously just as soon as the input file arrives in a monitored directory. How can we do that?
You may use java.nio.file.WatchService to monitor directory for a file.
Once file appears you may start (or kick off a job to process asynchronously) actual processing.
You may also use FileReadingMessageSource.WatchServiceDirectoryScanner from Spring Integration (https://docs.spring.io/spring-integration/reference/html/files.html#watch-service-directory-scanner)
Comparing release notes Spring Batch https://github.com/spring-projects/spring-batch/releases
to Spring Integration https://github.com/spring-projects/spring-integration/releases it looks that Spring Integration is released more often. It also has more features and Integration points.
In this case it looks like a overkill to bring Spring Integration if you just need to watch a directory for a file.
I would recommend using the powerful combination of Spring Batch with Spring Integration. For example, you can use a FileInboundChannelAdapter from Spring Integration to monitor a directory and start a Spring Batch Job as soon as the input file arrives.
There is a code example for this typical use case in the reference documentation of Spring Batch here: https://docs.spring.io/spring-batch/4.0.x/reference/html/spring-batch-integration.html#launching-batch-jobs-through-messages
I hope this helps.
I am a newbie in Java and trying to implement a Spring Boot batch application.
My requirement is like to check some data in database (one part) and delete if found (another part).
I am planning to implement Spring Boot batch for this.
I will have one job which will have 2 steps. If Step 1 find some data then only execute step 2? Can I achieve in Spring Boot Batch? Or what is the best way to achieve this keeping in mind I have to schedule this to run weekly.
With just the scheduled job for find and delete records from DB, I don't suggest using Spring Batch. Spring has nice good way of doing it without Batch using scheduling-tasks. You can see example here. Use Spring Batch only if you need to run jobs in batch that can't be handled with normal operation.
If you need complex scheduler, you can use Spring Quartz scheduler.
Currently I am working on some application which would later require batch functionality module since I have not yet started that module I need to know which (Struts or Springs) has the best Framework for batch processing.
Definitely Springs is better for batch processing as it provide complete spring batch framework for batch processing. Apart from this whatever you are running in struts can be easily integrated to spring frameworks.