I have a requirement where I have to call a spring batch job from a rest endpoint. We are using an API which has all the boilerplate code for running a job and it also adds an endpoint to our service and upon calling that endpoint it will run the requested job. I have attached a snip of that endpoint.
Please suggest a best approach to run the job from another rest endpoint within the same service should I call /Jobs endpoint or should I implement the logic of running the required job? TIA.
There is no best approaches, it depends on the use case. I would not make REST endpoints call each others, I think it would be simpler to inject the JobLauncher in your controller and call your job accordingly. You can find more details and a code example in the reference documentation here: Running Jobs from within a Web Container.
Related
I am working on the micro-service developed using Spring Boot . I have implemented following layers:
Controller layer: Invoked when user sends API request
Service layer: Processes the request. Either sends request to third-part service or sends request to database
Repository layer: Used to interact with the
database
.
Methods in all of above layers returns the CompletableFuture. I have following questions related to this setup:
Is it good practice to return Completable future from all methods across all layers?
Is it always recommended to use #Async annotation when using CompletableFuture? what happens when I use default fork-join pool to process the requests?
How can I configure the threads for above methods? Will it be a good idea to configure the thread pool per layer? what are other configurations I can consider here?
Which metrics I should focus while optimizing performance for this micro-service?
If the work your application is doing can be done on the request thread without too much latency, I would recommend it. You can always move to an async model if you find that your web server is running out of worker threads.
The #Async annotation is basically helping with scheduling. If you can, use it - it can keep the code free of the references to the thread pool on which the work will be scheduled. As for what thread actually does your async work, that's really up to you. If you can, use your own pool. That will make sure you can add instrumentation and expose configuration options that you may need once your service is running.
Technically you will have two pools in play. One that Spring will use to consume the result of your future, and another that you will use to do the async work. If I recall correctly, Spring Boot will configure its pool if you don't already have one, and will log a warning if you didn't explicitly configure one. As for your worker threads, start simple. Consider using Spring's ThreadPoolTaskExecutor.
Regarding which metrics to monitor, start first by choosing how you will monitor. Using something like Spring Sleuth coupled with Spring Actuator will give you a lot of information out of the box. There are a lot of services that can collect all the metrics actuator generates into time-based databases that you can then use to analyze performance and get some ideas on what to tweak.
One final recommendation is that Spring's Web Flux is designed from the start to be async. It has a learning curve for sure since reactive code is very different from the usual MVC stuff. However, that framework is also thinking about all the questions you are asking so it might be better suited for your application, specially if you want to make everything async by default.
I have a case where I want to start Spring Scheduled job from Linux Terminal. Is there some way to trigger it from the terminal?
For example can I use Spring Shell to start the Scheduled job?
can I use Spring Shell to start the Scheduled job?
Yes, theoretically you can (you can see a simple example here), but I am not sure it will fit your usecase if an application server is involved - invoking spring shell commands outside of the running shell process is more complex.
I would look into exposing start/stop functionality to Spring Scheduled Jobs as a REST api. You can still execute it from command line using curl and you can implement some authentication protocol around spring-security (or not if it's not needed).
Just wrap your start/stop methods with controller methods and expose those as REST api.
I have been working with Spring Cloud DataFlow and I have created my Composed Task which involves 3 Tasks.
My question is: Is possible to share information between those tasks? What would be a good pattern to do this?
I know that Spring Dataflow executes composed tasks as Spring Batch Jobs, so I was thinking that might be feasible share information using the Context.
As responded in the other thread, we don't provide out-of-the-box opinions because of the said reason.
However, if you have a requirement for it, you could use a shared database or a pub/sub broker for sharing information.
My application is written in Spring Framework 3.x.x. In my application I want to integrate the JBPM5.x processes using its REST API. I have done this thing but I want to send feedback like simple message to my spring application so that I'm able to know what will be the status of my process on the exit of the process. I'm not able to find any way to send this kind of feedback using REST API.
Please give the right direction for it, or give any other way to integrate Spring and JBPM so that my Spring application and JBPM process can be run in different application container instances (same application server but two different instances).
Not sure the REST api would be capable of handling something like that, it is basically some stateless API to get / send information, but doesn't handle async notifications.
What I would recommend is registering a custom process listener to the engine that will be notified when a process is completed, at that point you can do whatever you want, like for example send a JMS message or any other type of async message that could be picked up by your application.
This information is probably already stored in the history log as well. So if your Spring application could take advantage of that, that might be an option as well.
I am new to spring batch. I want to run spring batch jobs on server a and want to launch those jobs from server b using spring batch admin.is it possible? I have searched the following two ways:
1.JMX way: i could convert spring batch beans into mbeans but i cant read them from spring batch admin.can you tell how to read mbeans from spring batch admin and launch them?
2.common repository: i think if i use the same db repository for both spring batch and spring batch admin then i can launch remote jobs from spring batch admin (from server b).but in the job xml file in spring batch admin what should be the classpath for tasklet?
can you help in the above or tell me if any new way exists?
we ended up implementing a framework using mq communication to handle this. each 'batch node' registers itself and any 'batch class' parameters such as 'nodeType=A' or 'jobSizeiCanHandle=BIG' (these are fictitious but you get the point). The client console reads this information and queries the nodes via MQ for the job list. It then submits job requests with parameters via a rudimentary text based protocol (property file format).
command=START_JOB
job=JobABC
param1=x
param2=y
One of the batch nodes will pick up the message and start the job, it will return success/fail status in the same manner with a message with the same correlation id. so the client can show response to the user.
this allows us to do what you're talking about AND spark the jobs via an external scheduler (Control-M) . The 'nodeType=A' mentioned above allows us to query individual nodes (the nodes listen where 'nodeType=A or nodeType=*'. This allows commands to be 'targeted' to specific nodes if that is necessary.
Keep in mind, this is our own console, not the spring batch admin console. So perhaps that doesn't help you, but building up a simple console doesn't take that long using the spring batch APIs (4 or 5 asps).
The batch nodes could also have started up simple services like HTTP REST services or 'whatever' but we use MQ heavily and i liked the idea of not having to preregister nodes (the framework code doesn't know/care that it's in an HTTP container, so it couldn't register the endpoint easily). With MQ, the channel is preconfigured and all apps just 'use it' so it seemed easier.
Good luck.
I am trying to do the same thing. But it seems that in order to launch job directly from Spring batch admin, all the job resource has to be added to the spring batch web app. May be try restful job submission with spring MVC
#chau
One way to use Spring batch admin as is, but "discover" and "invoke" remote jobs is to provide your own implementations for org.springframework.batch.admin.service.JobService and org.springframework.batch.core.launch.JobOperator that can query and invoke jobs from remote job registry/repository.
You can find custom implementation for JobService and JMX enabled Job administrator in : https://github.com/regunathb/Trooper/tree/master/batch-core as: org.trpr.platform.batch.impl.spring.admin.SimpleJobService and org.trpr.platform.batch.impl.spring.jmx.JobAdministrator
Spring beans XML that uses these beans are here : https://github.com/regunathb/Trooper/blob/master/batch-core/src/main/resources/packaged/common-batch-config.xml