Spring-xd sink:file rollover support - spring-xd

I found request for sink:file rollover support, but it seems idle for more than a year now. What's the chance of adding this to spring-xd? In the mean time, is there a way to accomplish this without installing/deploying HDFS?

You can write a custom file sink; also make a comment and/or vote for that JIRA issue.

Related

Spring Integration `RotatingServerAdvice` Polling

When RotatingServerAdvice is added as an advice to a Poller, as in
PollerSpec pollerSpec = Pollers.cron(cronExpression)
.advice(rotatingServerAdvice(sftpConfig, proxyConfig))
.maxMessagesPerPoll(3)
.errorChannel("errorChannel");
will the poller rotate through each RotationPolicy.KeyDirectory at the scheduled time, or will it check one directory per poll? I've checked the examples in the Spring Integration Github repo and the reference documentation but I'm not able to get clarity on this. I'm guessing it should be the first, but I'd like to confirm.
Please, clarify why do you see a difference between scheduled time an poll? The poll really happens only when scheduler comes to the task to perform.
There is a fair option for you to consider. See docs: https://docs.spring.io/spring-integration/docs/current/reference/html/ftp.html#ftp-rotating-server-advice

Stackdriver monitoring - exclude specific time range in metric

I want to create a policy to check whether new data is published or not for X minutes. The issue I'm facing is, that we don't get any data in the night. Therefore it's necessary to exclude a specific time range in the metric otherwise we're getting an alert every night.
Is there any way to exclude a specific time range e.g. 9 PM until 9 AM in the policy/metric?
Alert Policy management currently does not include scheduling... but I'm thinking that using Cloud Scheduler, you may be able to achieve the results you are looking for, you may give this a try.
I also found that there's a Public Issue Tracker about Enable alerts only on certain hours. It is mentioned that a Feature Request was created, any update on this request will be published in the Public Issue Tracker or you can check the Feature Request directly as well.
You can use any gmail account to access the Public Issue Tracker.

Alternative to Timeout notification node in IBM Integration Bus

I probably have found a similar question (and answer) but I wanted to know if any better alternative is available.
Link to similar question:
http://mqseries.net/phpBB/viewtopic.php?t=72601&sid=f62d9730d61ee2ee2a59986dd79defd1
I want to schedule a particular message flow every 5 seconds (or so). I'm using IIB 10 and it's not associated with MQ. So, Timer nodes are non functional.
I've read about scheduling it with cronjob but again it's getting dependent on the OS which is not my preference. Is there any alternative to the timeout notification node?
Can we use java.util.TimerTask or something similar to to it? Any helping hands please?
I don't know of any solution that does not require a cron job or other external scheduler.
Many organisations use a distributed scheduler like Ctrl-M for a wide range of tasks, and adding a couple of jobs to support the integration layer is not seen as a problem.
You can write you own timer flow using an infinite WHILE loop with SLEEP and PROPAGATE TO TERMINAL functions and sending HTTP requests or configure a "CallableFlow".

how to avoid filling up hadoop logs on nodes?

When our Cascading jobs encounter an error in data, they throw various exceptions… These end up in the logs, and if the logs fill up, the cluster stops working. do we have any config file to be edited/configured to avoid such scenarios?
we are using MapR 3.1.0, and we are looking for a way to limit the log use (syslogs/userlogs), without using centralized logging, without adjusting the logging level, and we are less bothered about whether it keeps the first N bytes, or the last N bytes of logs and discords remain part.
We don't really care about the logs, and we only need the first (or last) few Megs to figure out what went wrong. We don't wan't to use centralized logging, because we don't really want to keep the logs/ don't care to spend the perf overhead of replicating them. Also, correct me if I'm wrong: user_log.retain-size, has issues when JVM re-use is used.
Any clue/answer will be greatly appreciated !!
Thanks,
Srinivas
This should probably be in a different StackExchange site as it's a more of a DevOps question than a programming question.
Anyway, what you need is your DevOps to setup logrotate and configure it according to your needs. Or edit the log4j xml files on the cluster to change the way logging is done.

Scheduled Tasks/plugin on CRM 4.0

Is it possible to schedule pulgin on CRM.
I ‘m trying to create a reminder task activity for an opportunity entity if no there has been no activities in the last week. I would like this to run every week.
Thanks in Advance.
Though there is no such thing as a recurring workflow in Dynamics CRM, you can simulate this behaviour by creating a Workflow Rule that
Gets fired upon creation of the record
Checks a condition
Conditionally does something
Waits for n days/hours/minutes
Calls itself again
Don't forget to mark the rule as "Run as a child workflow".
I think perhaps a better approach would be to build that functionality into a workflow? I have not been doing much CRM lately, but I have implemented what you described, in Crm 1.2 back in the day. This turned out to be very useful for reporting as well, pulling reports of what sales staff makes it a habit to not follow up on opportunities.
Best Regards
Rihan
I would recommend to utilize SQL Server Scheduled Job. Although workflow can be used for this problem, but waiting-state-workflow is considered application consuming.
The reason is because the waiting-state-workflow is held in the memory, and it needs to be validated against certain timing condition periodically. In your case, it needs to periodically calculated whether it has exceed one week or not, and if it's yes, it will upload the waiting-state-workflow from the database to be executed.
Regards,
hadi

Resources