I'm working on a Springboot app that includes a task that's executed on a schedule. It typically takes about two to three minutes to run.
#Scheduled(cron = "* */30 * * * *")
public void stageOfferUpdates() throws SQLException {
...
}
We have a requirement to be able to kick off the execution of that task at any time by calling a rest endpoint. Is there a way my #GET method can programmatically kick this off and immediately return an http 200 OK?
So you just want to trigger an async task without waiting for results. Because you are using Spring, the #Async annotation is an easy way to achieve the goal.
#Async
public void asyncTask() {
stageOfferUpdates();
}
Couldn't you just run the method in another Thread:
executor.execute(() -> {
stageOfferUpdates();
}
and then procede and return 200?
Related
I'm using jobrunr 5.1.4 in my spring boot application. I have a simple service declaring a recurring job which allows for some retries. A single failing job run is not that relevant for me. Instead, I'm interested in getting notified after all jobs, i.e. the initial job including all the retries, have failed.
I thought JobRunr's JobServerFilter would be a good idea. But the onProcessed() method never gets triggered in case of an exception only in case of a successful job run. And the ApplyStateFilter gets triggered on every state change. Far too often for my requirement. Leaving me clueless, if a change to a FAILED state was the last in a series of jobs belonging together (initial job + allowed retried jobs).
A simple example would look like this:
#Service
public class JobScheduler {
#Job(name = "My Recurring Job", retries = 2, jobFilters = ExceptionFilter.class)
#Recurring(id = "my-recurring-job", cron = "*/10 * * * *")
public void recurringJob() {
throw new RuntimeException("foo");
}
}
A basic implementation of my JobFilter looks like this:
#Component
public class ExceptionFilter implements JobServerFilter, ApplyStateFilter {
#Override
public void onProcessing(Job job) {
log.info("onProcessing: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onProcessed(Job job) {
log.info("onProcessed: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onStateApplied(Job job, JobState jobState1, JobState jobState2) {
log.info("onStateApplied: {}", job.getJobName());
log.info("jobState1: {}", jobState1.getName().name());
log.info("jobState2: {}", jobState2.getName().name());
}
}
Is this use case even possible with JobRunr? Or does anyone have an idea how to solve this issue in a different way?
Thank you very much in advance for you support.
I think you're on the right track with onStateApplied from ApplyStateFilter.
You can use the following approach:
#Override
public void onStateApplied(Job job, JobState oldState, JobState newState) {
if (isFailed(newState) && maxAmountOfRetriesReached(job)) {
// your logic here
}
}
OnProcessed is not triggered as your job was not processed (due to the failure).
I am creating a simple integration service with AWS S3. I am facing some difficulties when an exception occurs.
My requirement is to poll an S3 Bucket periodically and to apply some transformation whenever a file is newly placed into S3 Bucket. The below code snippet works fine, but when an exception occurs it continues to retry again and again. I do not want that to happen. Can someone help me here.,
The IntegrationFlow is defined as below.,
#Configuration
public class S3Routes {
#Bean
public IntegrationFlow downloadFlow(MessageSource<InputStream> s3InboundStreamingMessageSource) {
return IntegrationFlows.from(s3InboundStreamingMessageSource)
.channel("s3Channel")
.handle("QueryServiceImpl", "processFile")
.get();
}
}
Configuration file is as below.,
#Service
public class S3AppConfiguration {
#Bean
#InboundChannelAdapter(value = "s3Channel")
public MessageSource<InputStream> s3InboundStreamingMessageSource(S3RemoteFileTemplate template) {
S3StreamingMessageSource messageSource = new S3StreamingMessageSource(template);
messageSource.setRemoteDirectory("my-bucket-name");
messageSource.setFilter(new S3PersistentAcceptOnceFileListFilter(new SimpleMetadataStore(),
"streaming"));
return messageSource;
}
#Bean
public PollableChannel s3Channel() {
return new QueueChannel();
}
#Bean
public S3RemoteFileTemplate template(AmazonS3 amazonS3) {
return new S3RemoteFileTemplate(new S3SessionFactory(amazonS3));
}
#Bean(name = "amazonS3")
public AmazonS3 nonProdAmazonS3(BasicAWSCredentials basicAWSCredentials) {
ClientConfiguration config = new ClientConfiguration();
config.setProxyHost("localhost");
config.setProxyPort(3128);
return AmazonS3ClientBuilder.standard().withRegion(Regions.fromName("ap-southeast-1"))
.withCredentials(new AWSStaticCredentialsProvider(basicAWSCredentials))
.withClientConfiguration(config)
.build();
}
#Bean
public BasicAWSCredentials basicAWSCredentials() {
return new BasicAWSCredentials("access_key", "secret_key");
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata nonProdPoller() {
return Pollers.cron("* */2 * * * *")
.get();
}
}
AcceptOnceFileList filter that I have used here, helps me to prevent handling the same file for continuous retries. But, I do not want to use AcceptOnceFileList filter, because when a file is not processed on 1st attempt, I wish to retry on next Poll (usually it happens every 1 hour in Prod region). I tried to use filter.remove() method whenever the processing fails(in case of any exception), it again results in continuous retries.
I am not sure how to disable the continuous retries on failure. Where should I configure it?
I took a look at Spring Integration ( Retry Strategy). Same scenario, but a different integration. I am not sure how to set up this for my IntegrationFlow. Can someone help here? Thanks in advance
That story is different: it talks about a listener container for AMQP. You use a source polling channel adapter - the approach might be different.
You create two source polling channel adapters: one via that #InboundChannelAdapter, another via IntegrationFlows.from(s3InboundStreamingMessageSource). Both of them produces data to the same channel. Not sure if that is really intentional.
It is not clear what is that retry in your case unless you really do that manual filter.remove() call. In this case indeed it is going to retry. But this is a single, not controlled retry. It is going to retry again only if you call that filter.remove() again. So, if you do everything yourself, why is the question?
Consider to use a RequestHandlerRetryAdvice configured for that your handle() instead: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-endpoints.html#message-handler-advice-chain. This way you really going to pull the remote file only once and retry is going to be managed by the Spring Retry API.
UPDATE
So, after some Cron Expression learning I realized that your one is wrong:
* */2 * * * * - means every second of every even minute
Must be like this:
0 */2 * * * * - at the beginning of every even minute
Perhaps something similar is with your hourly cron expression on the prod...
I am trying to schedule the following function to run every 10seconds. But it tells that it needs a void function. Is there any way to schedule the following function?
#Scheduled(fixedRate = 10000)
public FtpOutboundGateway getFiles() {
FtpOutboundGateway gateway = new FtpOutboundGateway(sf(), "mget", "payload");
gateway.setAutoCreateDirectory(true);
gateway.setLocalDirectory(new File("./downloads/"));
gateway.setFileExistsMode(FileExistsMode.REPLACE_IF_MODIFIED);
gateway.setFilter(new AcceptOnceFileListFilter<>());
gateway.setOutputChannelName("fileResults");
return gateway;
}
Since #Scheduled method is called internally by the threads in thread pool in a given interval or time, the annotation expect it to be void because it doesn't make sense to return values. if you want to track down the response the extract it to another method and call it from the #Scheduled method as well as from other services.
For this, we need to do the following things.
First put this Annotation #EnableScheduling at the class level.
then at the function which needs to be scheduled should be marked with #Scheduled.
The function that we are trying to schedule should not return anything.
#EnableScheduling
class <ClassName>{
//def
#Scheduled(fixedDelayString = "10000")
public void fxnName() {
//def
}
}
I'm running a spring boot scheduled process that takes 5-10 seconds to complete. After it completes, 60 seconds elapse before the process begins again (Note that I'm not using fixedRate):
#Scheduled(fixedDelay=60_000)
Now, I want to limit it to run every minute Mon-Fri 9am to 5pm. I can accomplish this with
#Scheduled(cron="0 * 9-16 ? * MON-FRI")
Problem here is that this acts similar to fixedRate - the process triggers EVERY 60 seconds regardless of the amount of time it took to complete the previous run...
Any way to to combine the two techniques?
it worked for me like this
I created a bean that returns a specific task executor and allowed only 1 thread.
#Configuration
#EnableAsync
public class AsyncConfig implements AsyncConfigurer {
#Bean(name = "movProcTPTE")
public TaskExecutor movProcessualThreadPoolTaskExecutor() {
ThreadPoolTaskExecutor exec = new ThreadPoolTaskExecutor();
exec.setMaxPoolSize(1);
exec.initialize();
return exec;
}
}
In my service, I injected my task executor and wrapped my logic with it, so even though my schedule runs every minute, my logic will only run when the task executor is free.
#Service
#EnableScheduling
public class ScheduledService {
#Autowired
private ReportDataService reportDataService;
#Autowired
private AsyncService async;
#Autowired
#Qualifier("movProcTPTE")
private TaskExecutor movProcTaskExecutor;
#Scheduled(cron = "0 * * 1-7 * SAT,SUN")
public void agendamentoImportacaoMovProcessual(){
movProcTaskExecutor.execute(
() -> {
reportDataService.importDataFromSaj();
}
);
}
}
try this:
#Schedules({
#Scheduled(fixedRate = 1000),
#Scheduled(cron = "* * * * * *")
})
You can try this one:
#Scheduled(cron="1 9-16 * * MON-FRI")
Also you can try write correct on this site https://crontab.guru/
You can pass fixed delay (and any other number of optional parameters) to the annotation, like so:
#Scheduled(cron="0 * 9-16 ? * MON-FRI", fixedDelay=60_000)
From the documentation: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/scheduling/annotation/Scheduled.html
In dropwizard, I need to implement asynchronous jobs and poll their status.
I have 2 endpoints for this in resource:
#Path("/jobs")
#Component
public class MyController {
#POST
#Produces(MediaType.APPLICATION_JSON)
public String startJob(#Valid MyRequest request) {
return 1111;
}
#GET
#Path("/{jobId}")
#Produces(MediaType.APPLICATION_JSON)
public JobStatus getJobStatus(#PathParam("id") String jobId) {
return JobStatus.READY;
}
}
I am considering to use quartz to start job, but only single time and without repeating. And when requesting status, I will get trigger status. But the idea of using quartz for none-scheduled usage looks weird.
Is there any better approaches for this? Maybe dropwizard provides better tools itself? Will appriciate any advices.
UPDATE: I also looking at https://github.com/gresrun/jesque, but can not find any way to poll the status of running job.
You can use the Managed interface. In the snippet below I am using the ScheduledExecutorService to exuecute jobs, but you can use Quartz instead if you like. I prefer working with ScheduledExecutorService as it is simpler and easier...
first step is to register your managed service.
environment.lifecycle().manage(new JobExecutionService());
Second step is to write it.
/**
* A wrapper around the ScheduledExecutorService so all jobs can start when the server starts, and
* automatically shutdown when the server stops.
* #author Nasir Rasul {#literal nasir#rasul.ca}
*/
public class JobExecutionService implements Managed {
private final ScheduledExecutorService service = Executors.newScheduledThreadPool(2);
#Override
public void start() throws Exception {
System.out.println("Starting jobs");
service.scheduleAtFixedRate(new HelloWorldJob(), 1, 1, TimeUnit.SECONDS);
}
#Override
public void stop() throws Exception {
System.out.println("Shutting down");
service.shutdown();
}
}
and the job itself
/**
* A very simple job which just prints the current time in millisecods
* #author Nasir Rasul {#literal nasir#rasul.ca}
*/
public class HelloWorldJob implements Runnable {
/**
* When an object implementing interface <code>Runnable</code> is used
* to create a thread, starting the thread causes the object's
* <code>run</code> method to be called in that separately executing
* thread.
* <p>
* The general contract of the method <code>run</code> is that it may
* take any action whatsoever.
*
* #see Thread#run()
*/
#Override
public void run() {
System.out.println(System.currentTimeMillis());
}
}
As mentioned in the comment below, if you use Runnable, you can Thread.getState(). Please refer to Get a List of all Threads currently running in Java. You may still need some intermediary pieces depending on how you're wiring you application.