I have a git repository(hosted on Github) where I create my workflow-template YAMLs, and upload them to Argo via UI or REST-api. Now whenever I update any workflow-template, I have to manually update it in 2 places, the git-repo and Argo. There is a chance of either place being missed in this process.
How can I automate the process of updating workflow-templates in Argo-service, whenever the workflow-templates in git repository change?
you can do it in many ways.
you have ArgoCD to do CD on GitOps. It will make sure synced with desired state from GitHub(preferred)
you can use Argo workflow event binding to trigger workflow on GitHub action and get the change from GitHub and apply it. https://github.com/argoproj/argo-workflows/blob/342abcd6d72b4cda64b01f30fa406b2f7b86ac6d/docs/events.md
Related
I want to implement a generic CI pattern using Argo Workflows. One Workflow is triggered by the git server using an Argo Events webhook. After validating the signature, the repo is cloned and a custom workflow from the git repo is started (with a fixed name, e.g. .argo-ci.yaml). The Workflow of Workflows pattern is recommended to be implemented using the template.resource field. When using the manifest key, I can use input variables as shown in the example. However, variables are not evaluated using manifestFrom, which I would need to start the Workflow from git.
For a CI use case, the child workflow either needs to be passed the git repo artifact or at least the git revision as a variable so that the Workflow can clone and checkout the correct sources for which the CI was triggered. Both does not seem to be possible as the manifest from template.resource.manifestFrom is applied as-is without templating options.
I just setup CI/CD for a GitHub repo.
The CI build which validates a pull request is setup up as GitHub Action.
The CD build (which should run after the pull request was merged) is setup using Azure Pipelines as i would like to use the artifacts generated as a trigger for a Release Pipeline using Azure Pipelines as well.
The only thing that's still bugging me is, that the CD Build is also triggering automatically for a pull request and i can't figure out where i can configure those checks.
The checks currently running when a pull request is created are the following:
I want to get rid of the Continous Delivery Build here.
I tried to configure the branch protection rules but this has no effect:
On the Azure Pipeline side i completely disabled the triggers:
But this also has no visible effect to me.
I tested Disable pull request validation in the Triggers of the azure devops pipeline. On my side, it works well, and the build pipeline validation check is not displayed in the github pull request.
You can first check whether the pipeline source repo that you set the "Disable pull request validation" option corresponds to the github repo that created the pull request. Then try a few more times, it is possible that the settings are not applied immediately.
In addition, as workaround you can opt out of pull request validation entirely by specifying pr: none in yaml. Please refer to this official document.
# no PR triggers
pr: none
We would like to use Nifi registry with git as storage engine. In that case, i modified providers.xml and i was able to save the flows there.
Challenges:
There is no 2 way sync. We can only save the flows modified by Nifi user but if we modify the flow directly in git location, it will not be reflected on nifi registry
There is no review or approval process for Nifi registry. A user has to login to nifi-registry server, create a branch and issue a pull request.
As a workaround, we can delete the database file ( H2) and restart the nifi resgistry.
Lastly, everything should be automated in CI/CD like what we do for regular maven project.
Any suggestions ?
The purpose of the git storage is mostly to let user visualize the differences through tools like git hub, or any other tools that can support diffs, plus by pushing to a remote you also get a remote backup of the flow content. It is not meant to be modified outside of the application, just like you wouldn't bypass an application and go right into it's database and start changing data.
So we are currently just deploying master but are running into issues where we want to deploy the commit/build in which all our testing was ran on. This is normally a snapshop of master at 4:30pm. We run our build configuation for all tests automatically at 4:30pm (lets call this build config ALLTESTS), so we can control how this commit/build is marked in the ALLTESTS config.
We separate testing and deploy, so when a deploy is executed (either manually or automatically) it should only pick a branch/tag/commit/build that has been marked. Adding the tests to our deploy build config is not a viable solution.
Originally I had planned on using Git tags. A tag called deploy would be deleted and added to certain commits and then when the deployment is triggered that commit would be deployed.
The issue I ran into here is that there isnt an easy way to manually add git tags in a build step. Should I just write command-line build step that uses git commands remove the tag deploy from whatever commit has it and to add it to commit that is running?
Is there a better teamcity way to do this? I have successfully got teamcity tags to work via REST API but I am not sure if those fit the need either.
I suppose I could write powershell to parse the rest API to get the build id that was last successful in ALLTESTS and then feed that into the deploy somehow. How would I go about getting a build number and using that as the basis of deploy?
Should I just write command-line build step that uses git commands remove the tag deploy from whatever commit has it and to add it to commit that is running?
Quick answer is no. You can actually use something like this:
git tag -f deploy <commit-sha>
And have your tag updated to the given commit.
Cheers.
What is the best practice for automatically-generating ruby documentation as part of a git commit workflow? We'd like to automatically generate new documentation whenever a commit is merged into master.
Use Git hooks.
If you want it to run on the client side, use a post-commit hook. Essentially, registering a hook (by storing it in .git/hooks) makes Git call an arbitrary script you provide after each successful git commit. Your script can then call RDoc or YARD to generate docs into some output directory (outside your source code repository, of course).
Alternatively, you can have it run on the server hosting your Git repo, using a post-receive hook. This will run after you push to the server's repo. For example, you could use this to automatically upload the new docs to /docs/dev/ on your project's web server.
For detailed instructions, see the chapter on hooks in the Git manual.