I have a few repositories that use the same submodule. Is there a way to set those repositories to run a pipeline automatically when the origin of their submodule changes?
For instance if I push something on the parent repo, I want it to automatically notify the change to all the repositories using it as a submodule that the change has happened and that their pipeline needs to be run.
There is a feature called multi-project pipeline in GitLab that can trigger builds between repositories, but only in the Premium and Silver version:
https://docs.gitlab.com/ee/ci/multi_project_pipelines.html
You could get something to work with this. You just need to remember that just because you push new changes to your submodule, the repositories that use this submodule are not updated. They are still on the same commit as before.
You can of course do anything you want if you really want to. If you have a server where you can set up a gitlab runner, you can do something like this:
Create a new GitLab user, RoboGit, with access to the submodule and repositories that use that submodule
Install and register a gitlab runner on the server
Create a job for the submodule
This job can then manually check out your dependent projects, find the submodule, update them to the newest version, and then commit these changes. The author of these changes would then be RoboGit, and these changes would trigger build jobs in these repositories. You would get something like what you described in your question.
Related
We have an app (let’s call it the main repo) on GitLab CE, that has a production build & deploy pipeline, which is only triggered when a tag is deployed. This is achieved in .gitlab-ci.yml via:
only:
- /^v.*$/
except:
- branches
We also have two other (let’s call them side) repositories (e.g. translations and utils). What I’d like to achieve is to rerun the latest (semver) tag’s pipeline of main, when either of those other side repositories’ master branches receives a push. A small detail is that one of the repositories is on GitHub, but I’d be happy to get them working on GitLab first and then work from there.
I presume I’d need to use the GitLab API to trigger the pipeline. What I’ve currently set up for the side repo on GitLab is a webhook integration for push events:
https://gitlab.com/api/v4/projects/{{ID}}/ref/master/trigger/pipeline?token={{TOKEN}}, where ID is the ID of the main project and TOKEN a deploy token for it.
However, this will only trigger a master pipeline for our main repo. How could I get this to (also) rerun the latest tag’s pipeline (or the latest tagged pipeline)?
Secondly, how would I go about triggering this on GitHub?
Either you can create new pipeline specifying ref which can be branches or tags, so in this case you need to know the exact tag value https://docs.gitlab.com/ee/api/pipelines.html#create-a-new-pipeline
Or you can retry already the executed pipeline by providing its id which you can get from https://docs.gitlab.com/ee/api/pipelines.html#list-project-pipelines by sorting by id and filtering by ref but it'll give you the last pipeline with a tag /^v.*$/ which may not match with the specific version you need.
So we are currently just deploying master but are running into issues where we want to deploy the commit/build in which all our testing was ran on. This is normally a snapshop of master at 4:30pm. We run our build configuation for all tests automatically at 4:30pm (lets call this build config ALLTESTS), so we can control how this commit/build is marked in the ALLTESTS config.
We separate testing and deploy, so when a deploy is executed (either manually or automatically) it should only pick a branch/tag/commit/build that has been marked. Adding the tests to our deploy build config is not a viable solution.
Originally I had planned on using Git tags. A tag called deploy would be deleted and added to certain commits and then when the deployment is triggered that commit would be deployed.
The issue I ran into here is that there isnt an easy way to manually add git tags in a build step. Should I just write command-line build step that uses git commands remove the tag deploy from whatever commit has it and to add it to commit that is running?
Is there a better teamcity way to do this? I have successfully got teamcity tags to work via REST API but I am not sure if those fit the need either.
I suppose I could write powershell to parse the rest API to get the build id that was last successful in ALLTESTS and then feed that into the deploy somehow. How would I go about getting a build number and using that as the basis of deploy?
Should I just write command-line build step that uses git commands remove the tag deploy from whatever commit has it and to add it to commit that is running?
Quick answer is no. You can actually use something like this:
git tag -f deploy <commit-sha>
And have your tag updated to the given commit.
Cheers.
We have a Subversion repository setup in this manor:
http://svn.vegicorp.net/svn/toast/api/trunk
http://svn.vegicorp.net/svn/toast/api/1.0
http://svn.vegicorp.net/svn/toast/data/trunk
http://svn.vegicorp.net/svn/toast/data/branches/1.2
http://svn.vegicorp.net/svn/toast/data/branches/1.3
I've setup a Jenkins Multi-Pipeline build for the entire toast project including all sub-projects -- each sub-project is a jarfile. What I want is for Jenkins to fire off a new build each time any file is changed in one of the toast projects. That project should rebuild. This way, if we create a new sub-project in toast or a new branch in one of the toast sub-projects, Jenkins will automatically create a new build for that.
Here's my Jenkins Multi-Branch setup:
Branch Sources
Subversion
Project Repository Base: http://svn.vegicorp.net/svn/toast
Credentials: builder/*****
Include Branches: */trunk, */branches/*
Exclude Branches: */private
Property Strategy: All branches get the same properties
Build Configuration
Mode: By Jenkinsfile
Build Triggers (None selected)
Trigger builds remotely (e.g., from scripts) Help for feature: Trigger * builds remotely (e.g., from scripts)
Build periodically Help for feature: Build periodically
Build when another project is promoted
Maven Dependency Update Trigger Help for feature: Maven Dependency Update Trigger
Periodically if not otherwise run
Note that the list of Build Triggers list does not include Poll SCM. Changes in the repository does not trigger any build. Jenkinsfiles are located at the root of each sub-project. If I force a reindex, all changed sub-projects get built and all new branches are found. I did originally checked Periodically and reindexed every minute to pick up a change, but that's klutzy and it seems to cause Jenkins to consume memory.
Triggering a build on an SCM change should be pretty basic, but I don't see a configuration parameter for this like I do with standard jobs. I also can't seem to go into sub-projects and set those to trigger builds either.
There must be something really, really simple that I am missing.
Configuration:
Jenkins 2.19
Pipeline 2.3
Pipeline API: 2.3
Pipeline Groovy: 2.17
Pipeline Job: 2.6
Pipeline REST API Plugin: 2.0
Pipeline Shared Groovy Libraries: 2.3
Pipeline: Stage View Plugin: 1.7
Pipeline: Supporting APIs 2.2
SCM API Plugin: 1.2
I finally found the answer. I found a entry in the Jenkins' Jira Database that mentioned this exact issue. The issue is called SCM polling is not being performed in multibranch pipeline with Mercurial SCM. Other users chimed in too.
The answer was that Jenkins Multi-branch projects don't need to poll the SCM because indexing the branches does that for you:
Branch projects (the children) do not poll in isolation. Rather, the multibranch project (the parent folder) subsumes that function as part of branch indexing. If there are new heads on existing branches, new branch project builds will be triggered. You need merely check the box Periodically if not otherwise run in the folder configuration.
So, I need to setup reindexing of the branches. I'm not happy with this solution because it seems rather clumsy. I can add post-commit and post-push hooks in SVN and Git to trigger builds when a change takes place, and then reindex on a periodic basis (say once per hour). The problem means configuring these hooks and then keeping them up to date. Each project needs its own POST action which means updating the repository server every time a project changes. With polling, I didn't have to worry about hook maintenance.
You never mentioned setting up a webhook for your repository, so this may be the problem (or part of it).
Jenkins by itself can't just know when changes to a repository have been made. The repository needs to be configured to broadcast when changes are made. A webhook defines a URL that the repository can POST various bits of information to. Point it to a URL that Jenkins can read, and that allows Jenkins to respond to specific types of information it receives.
For example, if you were using github, you could have Jenkins listen on a url such as https://my-jenkins.com/github-webhook/. Github could be configured to send a POST as soon as a PR is opened, or a merge is performed. This POST not only symbolizes that the action was performed, but will also contain information about the action, such as a SHA, branch name, user performing the action... etc.
Both Jenkins and SVN should be capable of defining the URL they each respectively POST and listen on.
My knowledge lies more specifically with git. But this may be a good place to start for SVN webhooks: http://help.projectlocker.com/knowledge_base/topics/how-do-i-use-subversion-webhooks
Maybe you need something under version control in the base directory. Try putting a test file here http://svn.vegicorp.net/svn/toast/test.txt. That may make the poll SCM option show up.
We have a maven project on github for which we use Semaphore-CI.
Each time we merge a PR, the person who merges it is responsible for deploying a copy to our public package repo, incrementing the version number and pushing it directly to the code repo.
I was wondering if there is a way to automate this. Any ideas/suggestions are highly appreciated.
You could use Semaphore's custom commands to add the steps for pushing to this other repository.
Ex:
git remote add public_package http://example.com/public.git
git push --mirror public_package
I am trying to use bamboo to manage my release procedure and just wondering if this is feasible:
The developer finished the integration test at local environment
The developer create a specific tag, e.g. "UAT_1.0.0" and pushed the tag to bitbucket
Bamboo sensed that there is an new tag "UAT_1.0.0" has been created on bitbucket and then start the building process; after that it will deploy the war file to the UAT server
Tester signed off the UAT and created a tag "REL_1.0.0" and pushed the tag to bitbucket
Bamboo sensed the new tag "REL_1.0.0" and start the building process. After build finished, it deploy the war file to the PROD server
It looks like the "Repository triggers the build when changes are committed" is to best way to implement the process. But I can't find out the way to move any further. Any idea?
Yes you can do this (Only if you can deploy custom plugins to it).
You need to build custom triggers as plugins for bamboo.
Get/Install the SDK
Create a plugin. See here.
If you have access to bamboo's source code I suggest you look into classes DependencyTriggerReason, InitialbuildTriggerReason, ScheduledTriggerReason. You need to create a class implementing TriggerReason. You should start with this tutorial if you are new to developing bamboo plugins.
Deploy it to bamboo.
A bit late but... I found a solution. You need to put a "negative" regular expression in the "Exclude changesets" section in the repository configuration, that include the word that you want.
The regular expression is like :
^(?!.test).$
like that: repository configuration
with this, bamboo will only build commits with this word in the message.
Now, if your commit is like "commit for test", bamboo will build it