I have 2 jobs running asynchronously one which triggers everymin and another with a fixed delay.
#Scheduled(fixedDelay=30000)
public void runJob() {
try {
JobParameters jobParameters = new JobParametersBuilder().addLong("time",System.currentTimeMillis()).toJobParameters();
JobExecution execution=jobLauncher.run(job,jobParameters);
LOGGER.info(execution.getExitStatus());}
catch (Exception e) {
try {
throw new SystemException("Scheduler ERROR :: Error coocured during Job run "+e);
} catch (SystemException e1) {
LOGGER.error("Scheduler ERROR :: Error coocured during Job run "+e);
}
}
}
#Scheduled(cron = "0 0/1 * * * ?")
public void runJob2() {
try {
JobParameters jobParameters = new JobParametersBuilder().addLong("time",System.currentTimeMillis()).toJobParameters();
JobExecution execution=jobLauncher.run(job2,jobParameters);
LOGGER.info(execution.getExitStatus());}
catch (Exception e) {
try {
throw new SystemException("ERROR:: Exception occured"+e);
} catch (SystemException e1) {
LOGGER.error("ERROR:: JOB Launching exception happened"+e);
}
}
}
as fixedDelay says "the duration between the end of last execution and the start of next execution is fixed" but for me its triggering with a fixed delay between the starts of last and next execution.
2018-05-11 **12:48:00.016** INFO 2112 --- [ taskExecutor-5] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=systemStartJob]] launched with the following parameters: [{time=1526023080016}]
2018-05-11 12:48:00.016 INFO 2112 --- [ taskExecutor-5] org.sapient.t1automation.SystemListener : Intercepting system Job Execution - Before Job!
2018-05-11 12:48:00.017 INFO 2112 --- [ taskExecutor-5] o.s.batch.core.job.SimpleStepHandler : Executing step: [systemStartStep]
.
.
.
.
2018-05-11 **12:48:24.721** INFO 2112 --- [ taskExecutor-6] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=sendMailJob]] launched with the following parameters: [{time=1526023104706}]
2018-05-11 12:48:24.737 INFO 2112 --- [ taskExecutor-6] org.sapient.t1automation.MailListener : Intercepting Job Excution - Before Job!
2018-05-11 12:48:24.737 INFO 2112 --- [ taskExecutor-6] o.s.batch.core.job.SimpleStepHandler : Executing step: [sendMailStep1]
.
.
.
2018-05-11 12:48:44.533 INFO 2112 --- [ taskExecutor-6] org.sapient.t1automation.MailListener : Intercepting Job Excution - After Job!
2018-05-11 12:48:44.533 INFO 2112 --- [ taskExecutor-6] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=sendMailJob]] completed with the following parameters: [{time=1526023104706}] and the following status: [COMPLETED]
2018-05-11 12:48:45.001 INFO 2112 --- [ taskExecutor-3] o.s.t.service.mail.MailReader : Mail:: Mails to process. 1
2018-05-11 12:48:45.017 INFO 2112 --- [ taskExecutor-3] org.sapient.t1automation.MailListener : Intercepting Job Excution - After Job!
2018-05-11 12:48:45.017 INFO 2112 --- [ taskExecutor-3] o.s.b.c.l.support.SimpleJobLauncher : Job: [FlowJob: [name=sendMailJob]] completed with the following parameters: [{time=1526023044672}] and the following status: [COMPLETED]
here the time between start of 2 executions is 30s instead of between end of last and start of next execution.
Check the execution time of you code. It may be possible your code executes within a second, hence you cant see the time difference.
Sample example:
#Scheduled(fixedDelay = 3000)
private void test() {
System.out.println("test -> " + new Date());
try {
Thread.sleep(2000);
}
catch (Exception e) {
System.out.println("error");
}
}
Output
test -> Fri May 11 13:45:35 IST 2018
test -> Fri May 11 13:45:40 IST 2018
test -> Fri May 11 13:45:45 IST 2018
test -> Fri May 11 13:45:51 IST 2018
test -> Fri May 11 13:45:56 IST 2018
Here you can see, difference between every print is 5 seconds instead of 3 seconds.
For debugging, you can add logs at start and end of the code. Also, Thread.sleep() for delay.
Related
I am using QuickFix/J 2.3.1 (same results with 2.3.0). I have a rather straightforward spring boot application, where a FIX service is one of the beans. It creates an initiator. Until recently everything worked fine. Suddenly I stumbled into the following issue - quickfix doesn't seem to even attempt to open a connection to the specified host:port. I do suspect that this can be something to do with my code, but so far I don't have a clue on how to figure out what is going on.
Here is the initialisation code (Kotlin):
#PostConstruct
override fun start() {
logger.info("Using config file {}", config.tradingServiceConfig.quickFixConfigFile)
val sessionSettings = SessionSettings(config.tradingServiceConfig.quickFixConfigFile)
val messageStoreFactory = FileStoreFactory(sessionSettings)
val messageFactory = DefaultMessageFactory()
initiator = SocketInitiator(
this,
messageStoreFactory,
sessionSettings,
SLF4JLogFactory(sessionSettings),
messageFactory
)
logger.info("Calling initiator start")
initiator?.start()
logger.info("Initiator startup finished")
}
Here is the corresponding piece of log:
2021-12-12 22:20:48.962 INFO 94182 --- [ restartedMain] i.s.trading.gateway.service.FixService : Calling initiator start
2021-12-12 22:20:49.157 INFO 94182 --- [ restartedMain] quickfix.DefaultSessionSchedule : [FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT] daily, 08:00:00-UTC - 08:45:00-UTC
2021-12-12 22:20:49.180 INFO 94182 --- [ restartedMain] quickfixj.event : FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT: Session FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT schedule is daily, 08:00:00-UTC - 08:45:00-UTC
2021-12-12 22:20:49.181 INFO 94182 --- [ restartedMain] quickfixj.event : FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT: Session state is not current; resetting FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT
2021-12-12 22:20:49.185 INFO 94182 --- [ restartedMain] quickfixj.event : FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT: Created session: FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT
2021-12-12 22:20:49.186 INFO 94182 --- [ restartedMain] i.s.t.gateway.service.FixServiceBase : New session started: FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT}
2021-12-12 22:20:49.193 INFO 94182 --- [ restartedMain] quickfix.mina.NetworkingOptions : Socket option: SocketTcpNoDelay=true
2021-12-12 22:20:49.194 INFO 94182 --- [ restartedMain] quickfix.mina.NetworkingOptions : Socket option: SocketSynchronousWrites=false
2021-12-12 22:20:49.194 INFO 94182 --- [ restartedMain] quickfix.mina.NetworkingOptions : Socket option: SocketSynchronousWriteTimeout=30000
2021-12-12 22:20:49.276 INFO 94182 --- [ restartedMain] quickfixj.event : FIX.4.2:XXX_STAGE_UAT->YYY_XXX_STAGE_UAT: Configured socket addresses for session: [localhost/127.0.0.1:10669]
2021-12-12 22:20:49.277 INFO 94182 --- [ restartedMain] quickfix.SocketInitiator : SessionTimer started
2021-12-12 22:20:49.280 INFO 94182 --- [ restartedMain] i.s.trading.gateway.service.FixService : Initiator startup finished
2021-12-12 22:20:49.280 INFO 94182 --- [ssage Processor] quickfix.SocketInitiator : Started QFJ Message Processor
No other FIX, including quickfix, messages appear in the log. And I can see via netstat that not even an attempt is made to connect to the specified socket. I tried stopping the process in debugger to see what was going on, but couldn't see anything obvious.
As I said before, this used to work just fine a week or so ago when I last tried, that's why I'm so puzzled.
Any thoughts on how to debug the issue?
You seem to have configured the initiator to connect to the acceptor on a daily basis, between 08:00:00-UTC and 08:45:00-UTC.
Try increasing the date range (i. e. 08:00:00 to 18:00:00) and see if you get connected.
PS: If you're using quickfixj and Spring, have a look at QuickFixJ Spring Boot starter in Github https://github.com/esanchezros/quickfixj-spring-boot-starter
I have couple of routes (route 1 and route 2) in my Spring Boot application. I have been researching how to gracefully shutdown the application after processing both the routes. I have referred the documentation (https://camel.apache.org/manual/latest/graceful-shutdown.html) but couldn't successfully achieve what I needed. Maybe my understanding is wrong.
Below are my two routes
Route 1
from("timer://runOnce?repeatCount=1")
.to("{{sql.selectAll}}")
......... SOME PROCESSING
.to("direct:checkStatus")
Route 2
from("direct:checkStatus")
.delay(5000)
.loopDoWhile(CONDITION)
.process(DO_SOMETHING)
.end()
.to("jpa:com.pqr.MyClass)
.stop();
I have tried all these options
1. Automatic shutdown after 60 seconds
camel.springboot.duration-max-seconds = 60
It does GRACEFULLY shutdown the 2 routes but then WARNs about FORCEFUL shutdown ExecutorsService and also it doesn't stop the main thread to stop the application.
2020-03-01 18:28:25.507 WARN 30279 --- [otTerminateTask] o.a.c.i.e.BaseExecutorServiceManager : Forcing shutdown of ExecutorService: org.apache.camel.util.concurrent.SizedScheduledExecutorService#17fbfb02[CamelSpringBootTerminateTask] due first await termination elapsed.
2020-03-01 18:28:25.507 WARN 30279 --- [otTerminateTask] o.a.c.i.e.BaseExecutorServiceManager : Forcing shutdown of ExecutorService: org.apache.camel.util.concurrent.SizedScheduledExecutorService#17fbfb02[CamelSpringBootTerminateTask] due interrupted.
2020-03-01 18:28:25.508 INFO 30279 --- [otTerminateTask] o.a.c.i.e.BaseExecutorServiceManager : Shutdown of ExecutorService: org.apache.camel.util.concurrent.SizedScheduledExecutorService#17fbfb02[CamelSpringBootTerminateTask] is shutdown: true and terminated: false took: 10.004 seconds.
2020-03-01 18:28:25.508 WARN 30279 --- [otTerminateTask] o.a.c.i.e.BaseExecutorServiceManager : Forced shutdown of 1 ExecutorService's which has not been shutdown properly (acting as fail-safe)
2020-03-01 18:28:25.508 WARN 30279 --- [otTerminateTask] o.a.c.i.e.BaseExecutorServiceManager : forced -> org.apache.camel.util.concurrent.SizedScheduledExecutorService#17fbfb02[CamelSpringBootTerminateTask]
2. Initiate shutdown from the Route2
from("direct:checkStatus")
.delay(5000)
.loopDoWhile(CONDITION)
.process(DO_SOMETHING)
.end()
.to("jpa:com.pqr.MyClass)
.process(exchange -> {
exchange.getContext().getRouteController().stopRoute("route1");
exchange.getContext().getRouteController().stopRoute("route2");
System.out.println("Route1 -->"+exchange.getContext().getRouteController().getRouteStatus("route1"));
System.out.println("Route2 -->"+exchange.getContext().getRouteController().getRouteStatus("route2"));
exchange.getContext().shutdown();
});
"route1" is gracefully stopped but "route2" fails to be gracefully stopped with below message and waits for default timeout (300s).
2020-03-01 18:35:29.113 INFO 30504 --- [read #4 - Delay] o.a.c.i.engine.DefaultShutdownStrategy : Starting to graceful shutdown 1 routes (timeout 300 seconds)
2020-03-01 18:35:29.116 INFO 30504 --- [ - ShutdownTask] o.a.c.i.engine.DefaultShutdownStrategy : Route: route1 shutdown complete, was consuming from: timer://runOnce?repeatCount=1
2020-03-01 18:35:29.116 INFO 30504 --- [read #4 - Delay] o.a.c.i.engine.DefaultShutdownStrategy : Graceful shutdown of 1 routes completed in 0 seconds
2020-03-01 18:35:29.117 INFO 30504 --- [read #4 - Delay] o.a.c.s.boot.SpringBootCamelContext : Route: route1 is stopped, was consuming from: timer://runOnce?repeatCount=1
2020-03-01 18:35:29.117 INFO 30504 --- [read #4 - Delay] o.a.c.i.engine.DefaultShutdownStrategy : Starting to graceful shutdown 1 routes (timeout 300 seconds)
2020-03-01 18:35:29.118 INFO 30504 --- [ - ShutdownTask] o.a.c.i.engine.DefaultShutdownStrategy : Waiting as there are still 1 inflight and pending exchanges to complete, timeout in 300 seconds. Inflights per route: [route2 = 1]
It looks like there is a pending exchange message to be consumed. Do I need to manually clear/consume the exchange message in order to clear and facilitate a graceful shutdown?
Either option doesn't stop the main application. Do I have to write a custom Shutdown strategy instead of DefaultShutdownStrategy to achieve this? Can someone kindly point to an example to shut down the Spring Boot application after completion of the routes? Thanks in advance!!!
Did you try to use exchange.getContext().stop() to stop main application?
To force stop route without waiting for default timeout you can use exchange.getContext().stopRoute(routeId, 1L, TimeUnit.SECONDS); or set your timeout in seconds context.getShutdownStrategy().setTimeout(30);
You have to stop the currently running route from a new thread. The onCompletion() DSL is to make sure every message has been processed.
The attached code is in Kotlin, but it should be easy to transfer it to Java:
fromF(route).id(routeId)
.process(someProcessor)
.to("jdbc:dataSource")
.onCompletion()
.choice().`when`(exchangeProperty("CamelBatchComplete"))
.process(object : Processor {
override fun process(exchange: Exchange) {
Thread {
try {
exchange.context.routeController.stopRoute(routeId)
exchange.context.stop()
} catch (e: Exception) {
throw RuntimeException(e)
}
}.start()
}
}
)
// must use end to denote the end of the onCompletion route
.end()
If you want to stop the entire application, you can use this class and add a call of shutdownManager.initiateShutdown() after the exchange.context.stop().
#Component
class ShutdownManager {
companion object {
val logger = LoggerFactory.getLogger(ShutdownManager::class.java)
}
#Autowired
private val appContext: ApplicationContext? = null
fun initiateShutdown(returnCode: Int) {
logger.info("Shutting down with a Shutdown manager")
SpringApplication.exit(appContext, ExitCodeGenerator { returnCode })
System.exit(returnCode)
}
}
I'm trying to get the file count with my filter from my streaming inbound ftp apdater, so after i process all files, I want to launch a remote shell, or is there any other way to know that the adapter finished sending messages?
I tried already with CompositeFileListFilter overriding the public List filterFiles(F[] files) method, but it never gets called.
for now I'm using a fixed file count, but it should be dynamic.
I made an override of this method on the CompositeFileListFilter
#Override
public List<F> filterFiles(F[] files) {
log.info("received {} files", files.length);
return super.filterFiles(files);
}
I have the following integration flow, using an atomic counter until 3, it should be 3.:
AtomicInteger messageCounter = new AtomicInteger(0);
return IntegrationFlows.from(Ftp.inboundStreamingAdapter(goldv5template())
.remoteDirectory("/inputFolder")
.filter(new CompositeFileListFilterWithCount<>() {{
addFilter(new FtpSimplePatternFileListFilter("pattern1.*"));
addFilter(new FtpSimplePatternFileListFilter("pattern2.*"));
addFilter(new FtpSimplePatternFileListFilter("pattern3.*"));
}})
, pollerConfiguration)
.transform(Transformers.fromStream(StandardCharsets.UTF_8.toString()))
.log(message -> "process file " + message.getHeaders().get(FileHeaders.REMOTE_FILE))
.handle(message -> {
int numericValue = messageCounter.incrementAndGet();
log.info("numeric value: {}", numericValue);
if (numericValue == 3) {
messageCounter.set(0);
log.info("launch remote shell here now"));
}
}, e -> e.advice(after()))
.get();
if I don't use the counter, I would get a remote shell call for every file and I only need it to be called once, only when the flow finished, it's scheduled based on a cronjob, so I want to call it only one time at the end.
I'm using 1s fixed delay for test, but it would only run three times a day, I have to fetch all times at every clock.
this is my pollerConfiguration for test:
sourcePollingChannelAdapterSpec -> sourcePollingChannelAdapterSpec.poller(pollerFactory -> pollerFactory.fixedRate(1000L))
UPDATE
I tried what was suggested by Artem, but I'm having a weird behavior, I'm trying to fetch all files in a certain ftp folder in one poll, so reading the docs:
if the max-messages-per-poll is set to 1 (the default), it processes only one file at a time with intervals as defined by your trigger, essentially working as “one-poll === one-file”.
For typical file-transfer use cases, you most likely want the opposite behavior: to process all the files you can for each poll and only then wait for the next poll. If that is the case, set max-messages-per-poll to -1. Then, on each poll, the adapter tries to generate as many messages as it possibly can...
so i have set max-message-per-poll to -1 so every poll gives me every file.
I added a Filter to only take .xml files and to prevent duplicates, an acceptOnceFilter, but the ftp streaming adapter is giving me unlimited times the same files which doesn't make sense, I used for this test a FixedDelay of 10s.
2019-07-23 10:32:04.308 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process2 file sample1.xml
2019-07-23 10:32:04.312 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample1.xml
2019-07-23 10:32:04.313 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.313 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.315 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample2.xml
2019-07-23 10:32:04.324 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample2.xml
2019-07-23 10:32:04.324 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.324 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.326 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample3.xml
2019-07-23 10:32:04.330 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample3.xml
2019-07-23 10:32:04.331 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.331 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.333 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample4.xml
2019-07-23 10:32:04.337 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample4.xml
2019-07-23 10:32:04.338 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.338 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.341 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample1.xml
2019-07-23 10:32:04.345 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample1.xml
2019-07-23 10:32:04.346 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.346 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.347 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample2.xml
2019-07-23 10:32:04.351 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample2.xml
2019-07-23 10:32:04.351 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.351 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.353 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample3.xml
2019-07-23 10:32:04.356 INFO 9008 --- [ scheduling-1] o.s.integration.ftp.session.FtpSession : File has been successfully transferred to: /output/sample3.xml
2019-07-23 10:32:04.356 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : Advice after handle.
2019-07-23 10:32:04.357 INFO 9008 --- [ scheduling-1] i.d.e.v.job.factory.TestFlowFactory : ________________________________
2019-07-23 10:32:04.358 INFO 9008 --- [ scheduling-1] o.s.integration.handler.LoggingHandler : process file sample4.xml
...............................
return IntegrationFlows
.from(Ftp.inboundStreamingAdapter(testFlowTemplate())
.remoteDirectory("/inputTestFlow")
.filter(new CompositeFileListFilter<>() {{
addFilter(new AcceptOnceFileListFilter<>());
addFilter(new FtpSimplePatternFileListFilter("*.xml"));
}})
, sourcePollingChannelAdapterSpec -> sourcePollingChannelAdapterSpec.poller(pollerConfiguration.maxMessagesPerPoll(-1)))
.transform(Transformers.fromStream(StandardCharsets.UTF_8.toString()))
.log(message -> {
execution.setStartDate(new Date());
return "process file " + message.getHeaders().get(FileHeaders.REMOTE_FILE);
})
.handle(Ftp.outboundAdapter(FTPServers.PC_LOCAL.getFactory(), FileExistsMode.REPLACE)
.useTemporaryFileName(false)
.fileNameExpression("headers['" + FileHeaders.REMOTE_FILE + "']")
.remoteDirectory("/output/")
, e -> e.advice(testFlowAfter())
)
.get();
Update 2
I achieved what I needed creating this custom filter:
.filter(new FileListFilter<>() {
private final Set<String> seenSet = new HashSet<>();
private Date lastExecution;
#Override
public List<FTPFile> filterFiles(FTPFile[] files) {
return Arrays.stream(files).filter(ftpFile -> {
if (lastExecution!= null && TimeUnit.MILLISECONDS.toSeconds(new Date().getTime() - lastExecution.getTime()) >= 10L) {
this.seenSet.clear();
}
lastExecution = new Date();
if (ftpFile.getName().endsWith(".xml")) {
return this.seenSet.add(ftpFile.getRawListing());
}
return false;
}).collect(Collectors.toList());
}
})
but I used a handmade 10 Seconds Interval which is okay for my need, any other smart way to make this code better depending on the trigger ?
I think cron trigger is not a right solution here since you really would like to have a single process for all the fetched files.
I think your logic in the filterFiles() is wrong. You really would like to set a counter to the number of files it is going to process, but not the original amount:
#Override
public List<F> filterFiles(F[] files) {
List<F> filteredFiles = super.filterFiles(files);
log.info("received {} files", filteredFiles.size());
return filteredFiles;
}
and here you indeed can set a value into that messageCounter.
UPDATE
There is this functionality on filter:
/**
* Indicates that this filter supports filtering a single file.
* Filters that return true <b>must</b> override {#link #accept(Object)}.
* Default false.
* #return true to allow external calls to {#link #accept(Object)}.
* #since 5.2
* #see #accept(Object)
*/
default boolean supportsSingleFileFiltering() {
return false;
}
I think when you override it to an explicit false in your CompositeFileListFilterWithCount, you should be good. Otherwise you are indeed right: only a plain accept() is called for each file by default. Just because all your FtpSimplePatternFileListFilter comes with true by default and all of them are contribution to true on the FtpSimplePatternFileListFilter level.
Nevertheless all of that says to us that you are using already Spring Integration 5.2 :-)...
UPDATE 2
Try ChainFileListFilter isntead. Place an AcceptOnceFileListFilter in the end of the chain. Although it might be better to use a FtpPersistentAcceptOnceFileListFilter instead: it takes into account a lastmodified for the file. Also consider to include into chain some LastModifiedFileListFilter variant for the FTPFile. Something similar you have in your custom one, but as a separate filter.
Not sure, though, what you mean about making it based on trigger. There is just no any relationship between filter and trigger. You may, of course, have some common interval property and adjust it into the last modified filter value.
By the way: this your story went far away from the original at once request. An Inbound Channel Adapter is really about one file per message, so you definitely can't have a list of file in one message, like it is possible with the FtpOutboundGateway and its LS or MGET commands as I mentioned in comments below.
Regarding "how can I achieve wether all files in one message or all the messages together?" you can try property "max-messages-per-poll". It means:
"The maximum number of messages that will be produced for each poll. Defaults to
infinity (indicated by -1) for polling consumers, and 1 for polled inbound channel adapters.
I work on a Spring-boot project and use Flyway for database migration.
While working in dev-profile I want to fill the database with dummy data.
In order to have the exact same initial data values I overrode the FlywayMigrationStrategy Bean so that it performs a flyway.clean() before the migration starts:
#Bean
#Profile("dev")
public FlywayMigrationStrategy cleanMigrateStrategy() {
FlywayMigrationStrategy strategy = new FlywayMigrationStrategy() {
#Override
public void migrate(Flyway flyway) {
flyway.clean();
flyway.migrate();
}
};
return strategy;
}
My Migration folder contains several versioned migration scripts and ONE afterMigrate callback script which adds data to the created Tables.
The problem now is, the afterMigrate.sql script gets called two times as you can see from the following log:
2017-07-03 13:12:42.332 INFO 23222 --- [ main] o.f.core.internal.command.DbClean : Successfully cleaned schema "PUBLIC" (execution time 00:00.031s)
2017-07-03 13:12:42.397 INFO 23222 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 4 migrations (execution time 00:00.044s)
2017-07-03 13:12:42.413 INFO 23222 --- [ main] o.f.c.i.metadatatable.MetaDataTableImpl : Creating Metadata table: "PUBLIC"."schema_version"
2017-07-03 13:12:42.428 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema "PUBLIC": << Empty Schema >>
2017-07-03 13:12:42.430 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema "PUBLIC" to version 1 - create users
2017-07-03 13:12:42.449 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema "PUBLIC" to version 2 - create address
2017-07-03 13:12:42.464 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema "PUBLIC" to version 3 - create patient case
2017-07-03 13:12:42.475 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Migrating schema "PUBLIC" to version 4 - state machine
2017-07-03 13:12:42.498 INFO 23222 --- [ main] o.f.core.internal.command.DbMigrate : Successfully applied 4 migrations to schema "PUBLIC" (execution time 00:00.086s).
2017-07-03 13:12:42.499 INFO 23222 --- [ main] o.f.c.i.c.SqlScriptFlywayCallback : Executing SQL callback: afterMigrate
2017-07-03 13:12:42.502 INFO 23222 --- [ main] o.f.c.i.c.SqlScriptFlywayCallback : Executing SQL callback: afterMigrate
2017-07-03 13:12:42.917 INFO 23222 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup
If I remove the flyway.clean() function call it gets called only once.
Can somebody tell me why its called twice when I call flyway.clean() and flyway.migrate() and how to prevent the second call?
It's a known issue which will be fixed in Flyway 5.0. See https://github.com/flyway/flyway/issues/1653
I was trying to create a websocket based application where the server needs to keep the connection alive with the clients using heartbeat.
I checked the server ServerWebSocketContainer.SockJsServiceOptions class for the same, but could not use it. I am using the code from the spring-integration sample
#Bean
ServerWebSocketContainer serverWebSocketContainer() {
return new ServerWebSocketContainer("/messages").withSockJs();
}
#Bean
MessageHandler webSocketOutboundAdapter() {
return new WebSocketOutboundMessageHandler(serverWebSocketContainer());
}
#Bean(name = "webSocketFlow.input")
MessageChannel requestChannel() {
return new DirectChannel();
}
#Bean
IntegrationFlow webSocketFlow() {
return f -> {
Function<Message , Object> splitter = m -> serverWebSocketContainer()
.getSessions()
.keySet()
.stream()
.map(s -> MessageBuilder.fromMessage(m)
.setHeader(SimpMessageHeaderAccessor.SESSION_ID_HEADER, s)
.build())
.collect(Collectors.toList());
f.split( Message.class, splitter)
.channel(c -> c.executor(Executors.newCachedThreadPool()))
.handle(webSocketOutboundAdapter());
};
}
#RequestMapping("/hi/{name}")
public void send(#PathVariable String name) {
requestChannel().send(MessageBuilder.withPayload(name).build());
}
Please let me know how can I set the heartbeat options ensure the connection is kept alive unless the client de-registers itself.
Thanks
Actually you got it right, but missed a bit of convenience :-).
You can configure it like this:
#Bean
ServerWebSocketContainer serverWebSocketContainer() {
return new ServerWebSocketContainer("/messages")
.withSockJs(new ServerWebSocketContainer.SockJsServiceOptions()
.setHeartbeatTime(60_000));
}
Although it isn't clear for me why you need to configure it at all because of this:
/**
* The amount of time in milliseconds when the server has not sent any
* messages and after which the server should send a heartbeat frame to the
* client in order to keep the connection from breaking.
* <p>The default value is 25,000 (25 seconds).
*/
public SockJsServiceRegistration setHeartbeatTime(long heartbeatTime) {
this.heartbeatTime = heartbeatTime;
return this;
}
UPDATE
In the Spring Integration Samples we have something like stomp-chat application.
I have done there something like this to the stomp-server.xml:
<int-websocket:server-container id="serverWebSocketContainer" path="/chat">
<int-websocket:sockjs heartbeat-time="10000"/>
</int-websocket:server-container>
Added this to the application.properties:
logging.level.org.springframework.web.socket.sockjs.transport.session=trace
And this to the index.html:
sock.onheartbeat = function() {
console.log('heartbeat');
};
After connecting the client I see this in the server log:
2015-10-13 19:03:06.574 TRACE 7960 --- [ SockJS-3] s.w.s.s.t.s.WebSocketServerSockJsSession : Writing SockJsFrame content='h'
2015-10-13 19:03:06.574 TRACE 7960 --- [ SockJS-3] s.w.s.s.t.s.WebSocketServerSockJsSession : Cancelling heartbeat in session sogfe2dn
2015-10-13 19:03:06.574 TRACE 7960 --- [ SockJS-3] s.w.s.s.t.s.WebSocketServerSockJsSession : Scheduled heartbeat in session sogfe2dn
2015-10-13 19:03:16.576 TRACE 7960 --- [ SockJS-8] s.w.s.s.t.s.WebSocketServerSockJsSession : Preparing to write SockJsFrame content='h'
2015-10-13 19:03:16.576 TRACE 7960 --- [ SockJS-8] s.w.s.s.t.s.WebSocketServerSockJsSession : Writing SockJsFrame content='h'
2015-10-13 19:03:16.576 TRACE 7960 --- [ SockJS-8] s.w.s.s.t.s.WebSocketServerSockJsSession : Cancelling heartbeat in session sogfe2dn
2015-10-13 19:03:16.576 TRACE 7960 --- [ SockJS-8] s.w.s.s.t.s.WebSocketServerSockJsSession : Scheduled heartbeat in session sogfe2dn
In the browser's console I see this after:
So, looks like heart-beat feature works well...