I'm trying to build a control flow in ADF pipeline where an If Condition should only run on specific hours of the day. The pipeline has a trigger that runs every hour, but because of run limitations in the external system, this system should only be called at 6, 8, 10, 12 and so on during the day.
In SQL I would do something like CASE WHEN DATEPART(HOUR, GETDATE()) IN (6,8,10,12) THEN 1 ELSE 0
I fully believe there is a way to construct such an expression in "dynamic content", but I really don't know where to being. Should the hours be there written in an variable array or directly in the expression?
Thanks in advance!
I think your requirement could be implemented with built-in expression supported by ADF.
Please try configure Dynamic content in the If-condition Activity as below:
#contains('06081012',substring(formatDateTime(utcnow()),11,2))
Related
Environment:
I have around 100 pipelines that run on a number of triggers.
Outcome: I want to create a master pipeline that calls those 100 pipelines.
Currently, I've created a list of pipeline names and put them to an array. Then I was hoping to use forEach and execute pipeline activities to pass those names.
Issue, it seems that execute pipeline activity does not take variables or it is not obvious how to do it.
I do not want to create master pipeline manually as it can change often and I hope there must be better way to do it than manually.
You are correct that the "Invoked pipeline" setting of the Execute Pipeline activity does not support a variable value: the Pipeline name must be known at design time. This makes sense when you consider parameter handling.
One way around this is to create an Azure Function to execute the pipeline. This answer has the .Net code I leverage in my pipeline management work. It's a couple years old, so probably needs an update. If you need them to run sequentially, you'll need to build a larger framework to monitor and manage the executions, which is also discussed in that answer. There is a concurrency limit (~40 per pipeline, I believe), so you couldn't run all 100 simultaneously.
If I cannot, is there a scope that will make the same cfldap query available to all users? I'm just trying to cut down on the number of times it is called, since it's a cross-server call.
Step 1 - write some code that gets your ldap data and persists it somewhere. This could be a database, text file, or perhaps even a ColdFusion variable in the server scope.
Step 2 - Schedule that code to run at whatever frequency is good enough for your requirements.
The reason I mentioned server scope instead of application is that I'm not sure how hard it is to write application variables using scheduled jobs. I've never had a reason to try.
I have a build with a finish build trigger and i want to limit it to run only in specific time domain (e.g. between 4 AM to 5 AM) regardless to the finish build trigger, any suggestions?
Another solution is to pause build configuration. You can pause build configuration using REST API.
You could remove the finish build trigger and add an extra step to the triggering build which calls the rest api.
By adding a bit of logic around the call the check the time is in the required range, you should be able to achieve what you are after.
You can use cron-like expressions to write schedule triggers that will trigger your build at customized times. You can read the teamcity specs here
For example , if you want to run a target everyday at 4.30 AM , you would have to define it in the following syntax.
Seconds 0
Minutes 30
Hours 4
Day-of-month *
Month *
Day-of-week ?
Is it possible to insert the java code to run a previous activity in the process definition flow ?
For Example: A process definition contains the following items.
Start--> ReadFile-> SoapRequestReply -->end
In the above example I want to retry the SoapRequestReply activity with the help of java code if the execution of that activity contains any error.
I want to implement the logic in a generic way... I know the said concept can be implemented with the help of "REPEAT ON ERROR UNTILL TRUE" Group but I want to do it with the help of java code. so the new process definition would look like this.
Start--> ReadFile-> SoapRequestReply --exception-->RetryOnce(Java Code) --> end..
The Java Code will execute the Previous activity one more time.
Please suggest...
This is, indeed, a perfect fit for an error group. But if you really can't afford to use one, you could create a SubProcess that calls back your MainProcess on error and hold the retry count in a Job Shared Variable. Please note that this is a quick and dirty workaround.
You could do it by simply surround the SoapRequestReply with a Group.
This could be either a "Repeat on Error until true" group that repeats
by condition x times if an error occurs or a "while true" loop with individual
handling (error transition) e.g. for logging purposes.
No Java Coding / Activities needed.
With best regards
Seb
Is it possible to schedule a receive location to run once a week? I can only see to start it between certain dates and at certain times. Seems weird there is no way to run it at 10:00 on Fridays.
Am I missing an option here or is that just not possible? would the only possible way be to use the pollingstatement and do a query that results in 1 if the date is Friday?
you can try use this scheduled-task-adpater like this scheduled-task-adpater
or
if you like more complex way you can try to do somthing with the sdk
In the SDK directory of your BizTalk installation there is a script called EnableRecLoc.vbs
i prefer the first method