Currently, when I launch a task in spring cloud dataflow it starts a pod inside which the task and the inherent jobs run. That pod has a naming convention of task name followed by a random ID. I wanted to know if I can maybe map it to the task execution ID or Job Execution ID so that its easier for me to locate a pod in case a job fails and look at the logs.
In Spring Cloud Data Flow, this is the expected behavior.
The task execution ID and job execution ID are generated only after the task launch request is sent to the corresponding target deployment environment (local, CF, k8s).
Feel free to create a feature request (would be great if you have any other ideas) in SCDF Github and we can track it from there.
Related
We have Spring Cloud Data Flow local setup and the task is running the Spring Batch Job which reads from a Database and writes to AWS S3, all of this works fine.
When it comes to stopping the JOB, the task stops but resuming the job is not possible since the status is in "STARTED", this I think we can handle in code, by setting the batch status to 'STOPPED' when the stop is triggered, correct me if this can't be handled?
Also when trying to stop an individual slave task, there's an error:
2020-03-27 10:48:48.140 INFO 11258 --- [nio-9393-exec-7]
.s.c.d.s.s.i.DefaultTaskExecutionService : Task execution stop request
for id 192 for platform default has been submitted 2020-03-27
10:48:48.144 ERROR 11258 --- [nio-9393-exec-7]
o.s.c.d.s.c.RestControllerAdvice : Caught exception while
handling a request
java.lang.NullPointerException: null at
org.springframework.cloud.dataflow.server.service.impl.DefaultTaskExecutionService.cancelTaskExecution(DefaultTaskExecutionService.java:669)
~[spring-cloud-dataflow-server-core-2.3.0.RELEASE.jar!/:2.3.0.RELEASE]
at
org.springframework.cloud.dataflow.server.service.impl.DefaultTaskExecutionService.lambda$stopTaskExecution$0(DefaultTaskExecutionService.java:583)
~[spring-cloud-dataflow-server-core-2.3.0.RELEASE.jar!/:2.3.0.RELEASE]
How do we implement this is in distributed environment where we have a master server which can start the master on the master server and start the workers on respective slave servers?
1) You are correct you will need to change your status from STARTED to FAILED.
2) Since remote partitioning uses Spring Cloud Deployer (not Spring Cloud Data Flow) to launch the worker tasks, SCDF does not have a way to determine platform information to properly stop the the worker task. I've added GH Issue spring-cloud/spring-cloud-dataflow#3857 to resolve this problem.
3) The current implementation prevents a user from launching on multiple servers, rather lets the platform (Kubernetes, Cloud Foundry) distribute the worker tasks. You can implement your own deployer to add this feature.
How can I have a declarative Jenkins pipeline that is able to manage Jenkins server itself? I.e:
a pipeline that is able to query what Jobs I have in a folder and then disable/enable those jobs
Query what agents are available and trigger a job on that agent
A pipeline global variable currentBuild has a property called rawBuild that provides access to the Jenkins model for the current build. From there you can get to many of the Jenkins internals.
I'm not sure what you can find in the way of agent and job triggering - have a look there are/were plugins that offered alternatives to the default model.
I have a spring scheduler task configured with either of fixedDelay or cron, and have multiple instances of this app running on multiple JVMs.
The default behavior is all the instances are executing the scheduler task.
Is there a way by which we can control this behavior so that only one instance will execute the scheduler task and others don't.
Please let me know if you know any approaches.
Thank you
We had similar problem. We fixed it like this:
Removed all #Scheduled beans from our Spring Boot services.
Created AWS Lambda function scheduled with desired schedule.
Lambda function hits our top level domain with scheduling request.
Load balancer forwards this request to one of the service instances.
This way we are sure that scheduled task is executed only once across the cluster of our services.
I have faced similar problem where same scheduled batch job was running on two server where it was intended to be running on one node at a time. But later on I found a solution to not to execute the job if it is already running on other server.
Job someJob = ...
Set<JobExecution> jobs = jobExplorer.findRunningJobExecutions("someJobName");
if (jobs == null || jobs.isEmpty()) {
jobLauncher.run(someJob, jobParametersBuilder.toJobParameters());
}
}
So before launching the job, a check is needed if the job is already in execution on other node.
Please note that this approach will work only with DB based job repository.
We had the same problem our three instance were running same job and doing the tasks three times every day. We solved it by making use of Spring batch. Spring batch can have only unique job id so if you start the job with a job id like date it will restricts duplicate jobs to start with same id. In our case we used date like '2020-1-1' (since it runs only once a day) . All three instance tries to start the job with id '2020-1-1' but spring rejects two duplicate job stating already job '2020-1-1' is running.
If my understanding is correct on your question, that you want to run this scheduled job on a single instance, then i think you should look at ShedLock
ShedLock makes sure that your scheduled tasks are executed at most once at the same time. If a task is being executed on one node, it acquires a lock which prevents execution of the same task from another node (or thread). Please note, that if one task is already being executed on one node, execution on other nodes does not wait, it is simply skipped.
I didn't find info whether it is possible to define something like an Event-hook upon up/down-scaling or deletion of an App in the Marathon Rest API docs at https://mesosphere.github.io/marathon/docs/rest-api.html
What I'd like to achieve is that I'm able to backup some data from a running Docker container before be is destroyed. For example, I run a cluster of Elasticsearch nodes on Marathon, and I would like to delay the deletion of the app until the then triggered "Create snapshot to external disk resource" process is finished.
Is there currently something I could use?
Marathon provides an Event Bus covering some phases of the lifecycle. Beyond that, currently the only other option I see is to go for Mesos Modules/Hooks.
I can't get the information out of the documentation. Can anyone tell me how Spring-XD executes jobs? Does it assign a job to a certain container and is this job only executed on the container it is deployed to, or is each job execution assigned to another container? Can I somehow control that a certain job may be executed in parallel (with different arguments) and others may not ?
Thanks!
Peter
I am sure you would have seen some of the documentation here:
https://github.com/spring-projects/spring-xd/wiki/Batch-Jobs
To answer your questions:
Can anyone tell me how Spring-XD executes jobs? Does it assign a job to a certain container and is this job only executed on the container it is deployed to, or is each job execution assigned to another container?
After you create a new job definition using this:
xd>job create dailyfeedjob --definition "myfeedjobmodule" --deploy
the batch job module myfeedjobmodule gets deployed into the XD container. Once deployed, there is a job launching queue setup in the message broker: redis, rabbit or local. The name of the queue is job:dailyfeedjob in the message broker. Since this queue is bound to the job module deployed in the XD container, a request message sent to this queue is picked by the job module deployed inside that specific container.
Now, you can send the job launching request message (with job parameters) into the job:dailyfeedjob queue by simply setting up a stream that sends a message into this queue. For example: a trigger (fixed-delay, cron, date triggers) could do that. This also a job launch command from the shell which launches job only once.
This section would explain it more: https://github.com/spring-projects/spring-xd/wiki/Batch-Jobs#launching-a-job
Hence, the job is launched (every time it receives the job launching request) only inside the container where the job module is deployed and you can expect original the spring batch flow when the job is executed. (refer to shell doc for all the job related commands)
Can I somehow control that a certain job may be executed in parallel (with different arguments) and others may not ?
If it is for the different job parameters for the same job definition, then it would go to the same container where the job module is deployed.
But, you can still create a new job definition with the same batch job module.
xd>job create myotherdailyfeedjob --definition "myfeedjobmodule" --deploy
The only difference being it will be under that namespace. and, the job launching queue name would job:myotherdailyfeedjob. It all depends on how do you want to organize running your batch jobs.
Also, for parallel processing batch jobs you can use:
http://docs.spring.io/spring-batch/reference/html/scalability.html
and, XD provides single step partitioning support for running batch jobs:
Include this in your job module:
<import resource="classpath:/META-INF/spring-xd/batch/singlestep-partition-support.xml"/>
with partitioner and tasklet beans defined.
You can try out some of the XD batch samples from here:
https://github.com/spring-projects/spring-xd-samples