How to configure graceful shutdown using DelegatingSecurityContextScheduledExecutorService with Spring - spring

I'm trying to use the new options to do graceful shutdown with spring introduced in version 2.3, but I'm struggling to make my scheduled task to behave the same way.
As I need a valid user in the context during scheduled tasks execution, I am using DelegatingSecurityContextScheduledExecutorService to achieve this goal.
Here is a sample of my implementation of SchedulingConfigurer:
#Configuration
#EnableScheduling
public class ContextSchedulingConfiguration implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
}
#Bean
public TaskSchedulerCustomizer taskSchedulerCustomizer() {
return taskScheduler -> {
taskScheduler.setAwaitTerminationSeconds(120);
taskScheduler.setWaitForTasksToCompleteOnShutdown(true);
taskScheduler.setPoolSize(2);
};
}
#Bean
public Executor taskExecutor() {
ThreadPoolTaskScheduler threadPool = new ThreadPoolTaskScheduler();
taskSchedulerCustomizer().customize(threadPool);
threadPool.initialize();
threadPool.setThreadNamePrefix("XXXXXXXXX");
SecurityContext schedulerContext = createSchedulerSecurityContext();
return new DelegatingSecurityContextScheduledExecutorService(threadPool.getScheduledExecutor(), schedulerContext);
}
private SecurityContext createSchedulerSecurityContext() {
//This is just an example, the actual code makes several changes to the context.
return SecurityContextHolder.createEmptyContext();
}
#Scheduled(initialDelay = 5000, fixedDelay = 15000)
public void run() throws InterruptedException {
System.out.println("Started at: " + LocalDateTime.now().toString());
long until = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(30);
while (System.currentTimeMillis() < until) {}
System.out.println("Ended at: " + LocalDateTime.now().toString());
}
}
But when I send a termination signal while the sheduled task is running, the application does not wait for the task.
If in my bean taskExecutor I replace the last two lines, returning the ThreadPoolTaskScheduler without a context, everything work as expected. It only doesn't work when I return the DelegatingSecurityContextScheduledExecutorService.
How can I set the context for the taskExecutor and at the same time configure to wait for tasks to complete on shutdown?
I alredy tried several variations of this code, using another implementations of the interfaces TaskScheduler and TaskExecutor, but without success.

For starters cleanup your code and use the proper return types in the bean methods (be specific) and expose both as beans (marking one as #Primary!).
#Configuration
#EnableScheduling
public class ContextSchedulingConfiguration implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(securitytaskScheduler());
}
#Bean
public ThreadPoolTaskScheduler taskScheduler() {
ThreadPoolTaskScheduler taskScheduler= new ThreadPoolTaskScheduler();
taskScheduler.setAwaitTerminationSeconds(120);
taskScheduler.setWaitForTasksToCompleteOnShutdown(true);
taskScheduler.setPoolSize(2);
taskScheduler.setThreadNamePrefix("XXXXXXXXX");
return taskScheduler;
}
#Bean
#Primary
public DelegatingSecurityContextScheduledExecutorService securitytaskScheduler() {
SecurityContext schedulerContext = createSchedulerSecurityContext();
return new DelegatingSecurityContextScheduledExecutorService(taskScheduler().getScheduledExecutor(), schedulerContext);
}
private SecurityContext createSchedulerSecurityContext() {
//This is just an example, the actual code makes several changes to the context.
return SecurityContextHolder.createEmptyContext();
}
#Scheduled(initialDelay = 5000, fixedDelay = 15000)
public void run() throws InterruptedException {
System.out.println("Started at: " + LocalDateTime.now().toString());
long until = System.currentTimeMillis() + TimeUnit.SECONDS.toMillis(30);
while (System.currentTimeMillis() < until) {}
System.out.println("Ended at: " + LocalDateTime.now().toString());
}
}
Important is to be as specific in your return types as possible. Configuration classes are detected early on and the return types are checked to determine the callbacks to be made. Now ThradPoolTaskScheduler is a DisposableBean an Executor is not and will not receive callbacks as such!.

Related

Spring #Schleduled annotation does not work

I want a spring scheduled task, that runs every 10 seconds, however for some reason the task runs only once and is never repeated again.
Please do not suggest me to use other types of tasks, because I need specifically to use spring tasks.
#Scheduled(fixedRate = 10000, initialDelay = 1000)
public void myTask() {
...
}
In my main config class I have #EnableScheduling added as well.
Scheduling is a process of executing the tasks for a specific time period, but you looking to make it asynchrounslly so there will be a couple of changes
create a config class that will manage the Async operations so you make use of ThreadPoolTaskExecutor:
#EnableScheduling
#Configuration
public class TaskConfig implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar scheduledTaskRegistrar)
{
ThreadPoolTaskScheduler threadPoolTaskScheduler = new ThreadPoolTaskScheduler();
threadPoolTaskScheduler.setPoolSize(10);
threadPoolTaskScheduler.setThreadNamePrefix("your-scheduler-");
threadPoolTaskScheduler.initialize();
scheduledTaskRegistrar.setTaskScheduler(threadPoolTaskScheduler);
}
}
then you can run jobs asynchrounslly as the below :
#Component
public class HelloSender {
#Scheduled(fixedRate = 10000)
public void myTask() {
System.out.println("im running asynchronous with Worker : " + Thread.currentThread().getName());
}
}
for further information about ThreadPoolTaskExecutor please have look here: https://docs.spring.io/spring-framework/docs/3.0.x/spring-framework-reference/html/scheduling.html

How to use WatchService (NIO Package) & #Scheduled same time in single project of Spring-Boot?

I have written a script in while I have a schedular class that does something in after every certain time interval and another class in which I am watching a folder continuously for the occurance of any new file. And these both jobs (Schedular + WatchService) has to be endless.
But they are not getting called concurrently.
Called schedular class by - #Scheduled & #ComponentScan(basePackages = "com.project.schedular")
Calling WatchService by - #PostConstruct on method
Already tried Putting #PostConstruct on both and putting both packages in #ComponentScan({"com.project.schedular","com.project.watcher"})
Also tried putting #Async on both the methods.
Main Class:
#SpringBootApplication
#EnableScheduling
#ComponentScan(basePackages = "com.aprstc.schedular")
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Component
public class SchedularClass {
#PostConstruct
#Scheduled(fixedRate = 30000)
public void execute() {
//logic of scheduling method
}
Watcher Class:
#Component
public class WaybillReadScript {
#PostConstruct
public void watchFolder() throws InterruptedException, IOException {
System.out.println("Into the watch Folder.");
WatchService watchService = FileSystems.getDefault().newWatchService();
System.out.println(2);
Path path = Paths.get("/home/mypc-630/Work/abc");
System.out.println(3);
try {
path.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);
} catch (IOException e) {
e.printStackTrace();
}
WatchKey key;
while ((key = watchService.take()) != null) {
for (WatchEvent<?> event : key.pollEvents()) {
if (event.context().toString().equalsIgnoreCase("wbill.txt"))
processWaybillFile();
}
key.reset();
}
}
}
I expect that both classes must run concurrently.
Watcher Must do continuous watching.
And the scheduler must do a continuous scheduled job.
I think the PostConstruct is the wrong place. PostConstruct is used to initialize your beans/components. And if you make a blocking call with watchService.take(), this PostContruct will never be left and if not all beans are completely created than your application with the scheduler will not start.

Shutting down TaskScheduler does not stop running #Scheduled method

I have a class with a method annotated #Scheduled
#Component
#Slf4j
public class MyScheduler {
#Scheduled(cron = "${polling-job-cron}") //each minute
public void pollingJob() {
log.info("starting polling job...");
//some work
log.info("polling job finished.");
}
}
and a configuration for taskScheduler:
#Bean
public ThreadPoolTaskScheduler taskScheduler() {
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.setPoolSize(5);
scheduler.setThreadNamePrefix("mynameofscheduler");
scheduler.setWaitForTasksToCompleteOnShutdown(true);
scheduler.setAwaitTerminationSeconds(30);
scheduler.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy());
return scheduler;
}
I'm trying to use graceful shutdown by using class which waits for ContextClosedEvent :
#Component
#Slf4j
public class GracefulShutdown implements ApplicationListener<ContextClosedEvent> {
private final ApplicationContext context;
private final ThreadPoolTaskScheduler taskScheduler;
public GracefulShutdown(ApplicationContext context,
ThreadPoolTaskScheduler taskScheduler) {
this.context = context;
this.taskScheduler = taskScheduler;
}
#Override
public void onApplicationEvent(ContextClosedEvent event) {
log.info("Graceful shutdown - start");
log.info("Closing task scheduler");
taskScheduler.shutdown(); //1
taskScheduler.getScheduledThreadPoolExecutor().shutdown(); //2
log.error("Closed task scheduler");
//give k8s a chance to hit in readinessProbe and stop sending requests to this pod
try {
Thread.sleep(80000); //3
} catch (InterruptedException error) {
log.info("error while trying to sleep");
error.printStackTrace();
}
log.info("Closing spring context with startup date, {}, parent: {}, id: {}, name: {}",
context.getStartupDate(), context.getParent(), context.getId(), context.getDisplayName());
((ConfigurableApplicationContext) context).close();
log.info("Graceful shutdown - end");
}
and even though I'm closing taskScheduler and underlying taskExecutor new tasks are still ran by #Scheduled. Code of GracefulShutdown is ran when SIGTERM is send, and other that closing taskScheduler it works fine.
Graceful shutdown - start
Closing task scheduler
Closed task scheduler
starting polling job...
polling job finished
starting polling job...
polling job finished.
threadPoolPrefix is logged in front of those lines (I've cut that above as line were too long to read):
{"timeMillis":1534234560001,"thread":"mynameofscheduler","level":"INFO","loggerName":"myclassr","message":"starting polling job..."
I thought that maybe some other taskScheduler is used and I'm shutting down wrong one, but its all mynameofscheduler which is configured in #Bean
thx to M. Deinum. I was messing up with spring shutting down flow. I've fixed that by registering shutdown hook:
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(AgileStreamApplication.class, args);
Runtime.getRuntime().addShutdownHook(new Thread(new GracefulShutdownHook(context)));
}
and now I don't have to shut down taskSchedulers explicitly. It's done by spring.
Because by default the ScheduledThreadPoolExecutor will wait for all delayed scheduled tasks to finish executing, even if scheduled tasks aren't running at that time.
Try this below:
#Bean
public ThreadPoolTaskScheduler taskScheduler() {
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler() {
private static final long serialVersionUID = -1L;
#Override
public void destroy() {
this.getScheduledThreadPoolExecutor().setExecuteExistingDelayedTasksAfterShutdownPolicy(false);
super.destroy();
}
};
scheduler.setPoolSize(5);
scheduler.setThreadNamePrefix("mynameofscheduler");
scheduler.setWaitForTasksToCompleteOnShutdown(true);
scheduler.setAwaitTerminationSeconds(30);
scheduler.setRejectedExecutionHandler(new ThreadPoolExecutor.AbortPolicy());
return scheduler;
}
Then the ScheduledThreadPoolExecutor will only wait for scheduled tasks which are currently running to finish executing.

How to trigger a thread to check the system time say 5pm est in a spring integration project using java dsl and spring boot

Main application:-
#SpringBootApplication
#Configuration
public class Application {
public static void main(String[] args) {
new SpringApplicationBuilder()
.web(false)
.sources(Application.class)
.run(args);
}
}
FilePollingIntegrationFlow class:-
#Configuration
class FilePollingIntegrationFlow {
#Autowired
private ApplicationContext applicationContext;
//It does the file polling part:-
#Bean
public IntegrationFlow inboundFileIntegration(#Value("${inbound.file.poller.fixed.delay}") long period,
#Value("${inbound.file.poller.max.messages.per.poll}") int maxMessagesPerPoll,
TaskExecutor taskExecutor,
MessageSource < File > fileReadingMessageSource) {
// JobLaunchingGateway jobLaunchingGateway
return IntegrationFlows.from(fileReadingMessageSource,
c - > c.poller(Pollers.fixedDelay(period)
.taskExecutor(taskExecutor)
.maxMessagesPerPoll(maxMessagesPerPoll)))
.transform(Transformers.fileToString())
.channel(ApplicationConfiguration.INBOUND_CHANNEL)
.channel(MessageChannels.queue("fileReadingResultChannel"))
.get();
}
#Bean
TaskExecutor taskExecutor(#Value("${inbound.file.poller.thread.pool.size}") int poolSize) {
ThreadPoolTaskExecutor taskExecutor = new ThreadPoolTaskExecutor();
taskExecutor.setCorePoolSize(poolSize);
return taskExecutor;
}
#Bean
public FileReadingMessageSource fileReadingMessageSource(#Value("${inbound.filename.regex}") String regex, #Value("${inbound.directory.location}") String directory) {
FileReadingMessageSource source = new FileReadingMessageSource();
source.setDirectory(new File(directory));
source.setAutoCreateDirectory(true);
CompositeFileListFilter < File > filter = new CompositeFileListFilter < > (
Arrays.asList(new AcceptOnceFileListFilter < File > (),
new RegexPatternFileListFilter(regex))
);
source.setFilter(filter);
return source;
}
}
How to trigger a thread to check the system time with a fixed time in a day(say 5pm est), if sysyem time equals fixed time then System.exit(1) in a spring integration project using java dsl and spring boot.
I am not able to figure
how to exit from Integrationflows and move to another program or method.
Run a separate thread or on top of the same thread which does
polling to achieve my functionality.
Please advice.
Seems a bit brutal to pull the plug regardless of what's in process but...
#Component
public class Foo {
#Scheduled(cron = "0 0 17 * * *")
public void killer() {
System.exit(1);
}
}
with #EnableScheduling on the Application class.

spring-boot-starter-jta-atomikos and spring-boot-starter-batch

Is it possible to use both these starters in a single application?
I want to load records from a CSV file into a database table. The Spring Batch tables are stored in a different database, so I assume I need to use JTA to handle the transaction.
Whenever I add #EnableBatchProcessing to my #Configuration class it configures a PlatformTransactionManager, which stops this being auto-configured by Atomikos.
Are there any spring boot + batch + jta samples out there that show how to do this?
Many Thanks,
James
I just went through this and I found something that seems to work. As you note, #EnableBatchProcessing causes a DataSourceTransactionManager to be created, which messes up everything. I'm using modular=true in #EnableBatchProcessing, so the ModularBatchConfiguration class is activated.
What I did was to stop using #EnableBatchProcessing and instead copy the entire ModularBatchConfiguration class into my project. Then I commented out the transactionManager() method, since the Atomikos configuration creates the JtaTransactionManager. I also had to override the jobRepository() method, because that was hardcoded to use the DataSourceTransactionManager created inside DefaultBatchConfiguration.
I also had to explicitly import the JtaAutoConfiguration class. This wires everything up correctly (according to the Actuator's "beans" endpoint - thank god for that). But when you run it the transaction manager throws an exception because something somewhere sets an explicit transaction isolation level. So I also wrote a BeanPostProcessor to find the transaction manager and call txnMgr.setAllowCustomIsolationLevels(true);
Now everything works, but while the job is running, I cannot fetch the current data from batch_step_execution table using JdbcTemplate, even though I can see the data in SQLYog. This must have something to do with transaction isolation, but I haven't been able to understand it yet.
Here is what I have for my configuration class, copied from Spring and modified as noted above. PS, I have my DataSource that points to the database with the batch tables annotated as #Primary. Also, I changed my DataSource beans to be instances of org.apache.tomcat.jdbc.pool.XADataSource; I'm not sure if that's necessary.
#Configuration
#Import(ScopeConfiguration.class)
public class ModularJtaBatchConfiguration implements ImportAware
{
#Autowired(required = false)
private Collection<DataSource> dataSources;
private BatchConfigurer configurer;
#Autowired
private ApplicationContext context;
#Autowired(required = false)
private Collection<BatchConfigurer> configurers;
private AutomaticJobRegistrar registrar = new AutomaticJobRegistrar();
#Bean
public JobRepository jobRepository(DataSource batchDataSource, JtaTransactionManager jtaTransactionManager) throws Exception
{
JobRepositoryFactoryBean factory = new JobRepositoryFactoryBean();
factory.setDataSource(batchDataSource);
factory.setTransactionManager(jtaTransactionManager);
factory.afterPropertiesSet();
return factory.getObject();
}
#Bean
public JobLauncher jobLauncher() throws Exception {
return getConfigurer(configurers).getJobLauncher();
}
// #Bean
// public PlatformTransactionManager transactionManager() throws Exception {
// return getConfigurer(configurers).getTransactionManager();
// }
#Bean
public JobExplorer jobExplorer() throws Exception {
return getConfigurer(configurers).getJobExplorer();
}
#Bean
public AutomaticJobRegistrar jobRegistrar() throws Exception {
registrar.setJobLoader(new DefaultJobLoader(jobRegistry()));
for (ApplicationContextFactory factory : context.getBeansOfType(ApplicationContextFactory.class).values()) {
registrar.addApplicationContextFactory(factory);
}
return registrar;
}
#Bean
public JobBuilderFactory jobBuilders(JobRepository jobRepository) throws Exception {
return new JobBuilderFactory(jobRepository);
}
#Bean
// hopefully this will autowire the Atomikos JTA txn manager
public StepBuilderFactory stepBuilders(JobRepository jobRepository, JtaTransactionManager ptm) throws Exception {
return new StepBuilderFactory(jobRepository, ptm);
}
#Bean
public JobRegistry jobRegistry() throws Exception {
return new MapJobRegistry();
}
#Override
public void setImportMetadata(AnnotationMetadata importMetadata) {
AnnotationAttributes enabled = AnnotationAttributes.fromMap(importMetadata.getAnnotationAttributes(
EnableBatchProcessing.class.getName(), false));
Assert.notNull(enabled,
"#EnableBatchProcessing is not present on importing class " + importMetadata.getClassName());
}
protected BatchConfigurer getConfigurer(Collection<BatchConfigurer> configurers) throws Exception {
if (this.configurer != null) {
return this.configurer;
}
if (configurers == null || configurers.isEmpty()) {
if (dataSources == null || dataSources.isEmpty()) {
throw new UnsupportedOperationException("You are screwed");
} else if(dataSources != null && dataSources.size() == 1) {
DataSource dataSource = dataSources.iterator().next();
DefaultBatchConfigurer configurer = new DefaultBatchConfigurer(dataSource);
configurer.initialize();
this.configurer = configurer;
return configurer;
} else {
throw new IllegalStateException("To use the default BatchConfigurer the context must contain no more than" +
"one DataSource, found " + dataSources.size());
}
}
if (configurers.size() > 1) {
throw new IllegalStateException(
"To use a custom BatchConfigurer the context must contain precisely one, found "
+ configurers.size());
}
this.configurer = configurers.iterator().next();
return this.configurer;
}
}
#Configuration
class ScopeConfiguration {
private StepScope stepScope = new StepScope();
private JobScope jobScope = new JobScope();
#Bean
public StepScope stepScope() {
stepScope.setAutoProxy(false);
return stepScope;
}
#Bean
public JobScope jobScope() {
jobScope.setAutoProxy(false);
return jobScope;
}
}
I found a solution where I was able to keep #EnableBatchProcessing but had to implement BatchConfigurer and atomikos beans, see my full answer in this so answer.

Resources