As part of my CircleCI pipeline, I would like to start the pipeline by running a command on slack - in this case /yoink deploy (we are using Yoink to manage shared resources, and trying to come up with a low-effort way to make sure people aren't stepping on each others toes during deployments). If the response back from the yoink bot is affirmative, continue along the build pipeline. Otherwise, stop it.
The challenge that I'm having is that the Slack integrations for circle let me post a message under different circumstances, but I can't figure out a way to make it use a reply from that posted message to choose next pipeline steps. My question is this:
Has anyone previously managed to do something like this with a slack integration with Circle, or is there some other CircleCI method that folks have used to lock and release said lock on deployments?
Related
I'm trying to setup the CI/CD setup of a mono repository using Google Cloud Build. We have a single Cloud Build trigger that starts a build on a new commit, it does some general steps and then then starts a build for every (micro)service in the mono repository using gcloud build submit.
This however means that if 4 or 5 people are push code to the repository roughly at the same time we can have around 50-70 concurrent builds running in cloud build. Which in itself isn't an issue for us. The only issues is that when this happens the following errors will popup:
{
“code”: 429,
“message”: “Quota exceeded for quota metric ‘Build and Operation Get requests’ and limit ‘Build and Operation Get requests per minute’ of service ‘cloudbuild.googleapis.com’ for consumer ‘project_number:<PROJECT_NUMBER>’.“,
“status”: “RESOURCE_EXHAUSTED”,
“details”: [{
“#type”: “type.googleapis.com/google.rpc.ErrorInfo”,
“reason”: “RATE_LIMIT_EXCEEDED”,
“domain”: “googleapis.com”,
“metadata”: {
“service”: “cloudbuild.googleapis.com”,
“consumer”: “projects/<PROJECT_NUMBER>”,
“quota_limit”: “GetRequestsPerMinutePerProject”,
“quota_metric”: “cloudbuild.googleapis.com/get_requests”
}
}]
}
In other words: We are running into quota limits. The quota only allows us to only make 900 operational requests per minute.
We already tried switching to private pools in the hope that the above quota limit was only there for when you don't use private pools, but this unfortunately still makes us hit the quota.
Now, I am trying to find out if I can decrease the amount of these operational requests.
A possible solution might be related to how I am using gcloud build submit. When you run gcloud build submit, it starts a new build, waits for the build to finish, and shows the output of the build. To achieve this, I presume that gcloud is making requests every few seconds to find out what the status of the build is. I suspect that these 'status' requests are why my Cloud Build quota limit is reached. Which is why I'm trying to see how I can lower the amount of these requests per minute.
One option is to simple decrease the amount of builds running in parallel, which is unfortunately not an option in my situation. If I execute them sequentially it simply takes more time than acceptable in my situation.
Another option would be to increase the time in between such 'status' requests. However, on this page I did unfortunately not find a CLI flag to alter this.
Note: I did find the --async flag, however that does NOT help me, since I still want the process to wait until the build has succeeded. And I also did find the --supress-logs, which also does NOT help me, since these requests presumably don't interact with Cloud Build but with the GCS bucket where the logs are stored.
The only option left that I can think off, is that I can start my builds with the --async flag and then manually request whether the build has succeeded using a longer interval. However I do feel like that is a lot of manual work that, for which I need to write some bash scripts that need to be maintained. This preferably isn't a path I would like to take unless really necessary.
Does anyone know of another way of achieving this?
If 4 or 5 people are push code to the repository
This shouldn't happen. The reason it shouldn't happen is because you should use the "push" trigger on the main branch, not on a development branch.
What do I mean by this?
I mean that building should occur on the main branch, which would correspond to joined effort of those five users and a responsible party in charge of unifying their changes.
So, really, your users should be pushing to the development branch, and pushes to main should be reserved for things that need to be built.
How can we work around this if we're only allowed one branch or are required to have updates visible on one branch?
My recommendation would be to use the tag filter, specifically filter the pushes by tag, as mentioned in the documentation. That way only the pushes person in charge of merging the changes will be built (assuming that this person pushes to the tag you've set)
TL;DR
Don't create push triggers for Cloud Build on a branch multiple people are working on. Either create it with a tag filter or have seperate development and main branches (people work on dev, builds are only made from pushes to main)
I've got a pipeline that logs in and logs out off a web-application every 5 minutes to ensure that the apps backend works, the database is up...
There occurred a problem that was not even related to the app directly, and my boss was bombarded with email-notifications. Is it possible to limit the emails that notify of a series of broken pipelines to only one, and suppress all subsequent emails until the pipeline has been fixed?
It seems that the editor for "Pipelines emails" is rather limited and doesn't support this directly. However, this option exists in Jenkins, and I'm wondering if someone figured out a solution or a workaround to achieve this in Gitlab CI. (Is it possible to script something like this in the ".gitlab-ci.yml"-file?)
I need a better way to use my alerting code.Right now I have a code that check for space free on aws ecs and sends a simple notification to slack if space is less than 5gb using slack api.I used this code in jenkins and setup a periodic schedule to run every 15 min.But once the notification is triggered I wanted it to stop the check for 4 hours so, it won't fill the slack channel with messages .So, i used sleep 14400 after condition is triggered.But this leaves an executor of jenkins waiting.Is there a better way to do this?
If you really want a better way, you should use better tools. there are many tools (some free) out there, that can monitor something in a stateful manner (for example, using a daemon).
Writing to log (or slack channel) in this context of using Jenkins is sort of stateless, for example you cannot check whether an alarm is currently triggered or not.
Since you cannot check if an alarm is already triggered - using jenkins with the logic you requested in your question ('snooze feature') can be very ugly.
In general I would recommend using Conditional BuildStep to trigger a step if a condition is met (i.e. if alarm not already triggered), but since there is no way for you to poll this information, or achieve this with Jenkins without the solution being 'hackish' like creating a file to indicate alert is on, and deleting it from another job if it was created more than 4 hrs ago - I would suggest looking at tools more suitable for the job.
Currently we have a hipchat notification step as part of our process which lets us know if a release fails. Is there any way of detecting if a deploy is cancelled/aborted so we can display a relevant notification?
Thanks David.
Is it a scheduled deployment someone would cancel/deploy before it was run?
Or is it a deployment with a manual intervention where someone would stop it
Is your send notification step scheduled to run always?
After a bit of search found that one can alternatively subscribe to audit events, which were added into OD from version 3.5, could be something considerable for the situation.
I have a bot working on commit and it increases the build number and pushes to the same branch. I check the commit's user in Before integration script, and if it is the CI user (which is only and only used to push the increase number commits) i want to abort current integration. I saw this one:
https://stackoverflow.com/a/30062418/767329
/xcode/api/integrations/INTEGRATION_ID/cancel
This one makes a curl call to stop the integration but i want to stop the current integration before it starts. I know i may also check and push the increase commit if the bot is not run by ci user's increase commit lately. But i dont want even archive to work if it is a ci user commit (i want the integration to be aborted even before it starts).
Unfortunately there is no way to cancel an integration before it even starts. You could use a pre-integration trigger to stop the integration from going further given whatever conditions you are looking for.
If your only goal is to bump the build number, I would suggest you use the Xcode Server environment variable 'XCS_INTEGRATION_NUMBER' in your build number field.
Whenever Xcode Server integrates your project, it will automatically use the integration number as the build number. These will always be unique.