I'm trying to use a post-receive git-hook to automate the deploy of a simple maven project by triggering a Jenkins pipeline I set up. The source is hosted on a GitHub repo while Jenkins on a container running on my PC. So far, the hook is not triggered after I push to master branch.
Thing is if I try and run the script manually it just works! I also tried setting chmod +x with Git Bash (after all I'm on Windows) to the post-receive file, unfortunately without success: the hook still does not get triggered. What might be the issue?
I already tried looking for answers on similar topics here on stackoverflow, but nothing solved my issue. FYI, below the post-receive script (nothing fancy, as you can see):
#!/bin/bash
JENKINS_URL="http://localhost:8080"
JOB="deploy-to-slave-pipeline"
JENKINS_CREDENTIALS="theuser:11d422ee679503eeb328c5b1998327cc7f"
echo "Triggering Jenkins job..."
crumb=$(curl -u "$JENKINS_CREDENTIALS" -s '$JENKINS_URL/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)')
curl -u $JENKINS_CREDENTIALS -H "$crumb" -X POST "$JENKINS_URL/job/$JOB/build?delay=0sec"
EDIT
As pointed out by #bk2204 post-receive is a server-side hook. What I needed was a webhook, which can be set in the Settings/Webhook page of your GitHub repo. Just configure it as below where Payload URL is your Jenkins URL followed by /github-webhook/:
Then all you have to do is set your Jenkins job to get triggered by GitHub, by checking the related option on the Build Triggers section as below:
And then you're good to go! Also, if you're running your Jenkins instance locally, you could use ngrok to expose it and test your CI/CD pipeline!
[ref. https://dzone.com/articles/adding-a-github-webhook-in-your-jenkins-pipeline]
A post-receive hook is run on the server side, not on the client side. That means that it's run at GitHub, assuming you're pushing to GitHub, ant not on your local machine.
Normally, you'd want a GitHub webhook to notify you of the push event, but you cannot use one here because the machine is running on localhost and such an event has to be able to on a public IP address since GitHub has to send an HTTP request to it.
Related
I have a raspberry pi running Ubuntu that I am using to host a discord bot. I was trying to figure out if there is any way to detect when master of the bot's code repository was changed and then run a script on the pi to stop the bot, pull the changes and then restart it? I have written the script already I'm just not sure how to trigger it.
There are a two main solutions to this - cronjobs and webhooks. A cronjob is likely easiest, but webhooks give more external control.
Cronjobs
A cronjob will run a script at a given interval. If you set a cronjob on the pi to run git fetch it will update the refs of the remotes. You can then do git rev-parse <branch> to get the commmit ID of a given branch. So in bash you can do:
#!/bin/bash
REMOTE=origin
BRANCH=master
git fetch
if [[ "$(git rev-parse $BRANCH)" != "$(git rev-parse "$REMOTE/$BRANCH")" ]]; then
# Run your script
fi
Webhooks
If you are hosting your code on a major platform - like GitLab, GitHub, or BitBucket - it likely supports webhooks to do this. A webhook is where you give the repo host a URL which they will call when a certain event happens. If you are using somebody else's repository, you can make a fork that mirrors theirs and add the webhook to your fork.
This requires your pi to be accessible from the web and running an HTTP server, you will either need a static IP address or a DDNS service for this to work.
Background
I have a system where when I push changes to my Repository, A web hook sends a request to my site which runs a bash script to pull the changes and copy any updated files.
I added a second repository with its own deploy key but after doing so i was getting a permission denied error when trying to pull changes.
Question
Is there a way to use 2 deploy key's on the same server?
Environment Details
Site uses Laravel 5.6, Symfony used to run shell script
Git 1.7
Go Daddy web hosting (Basic Linux one)
Notes
Script just runs git pull command
Error given is " Permission denied (publickey) "
SHH is used as a deploy key so only read access, there is one other project also using a deploy key on the same server
Thank you in advance for you help! Any other suggestions are welcome!
Edit #1
Edited post to reflect true problem as it was different to what I though (Feel free to revert if this is bad practice), please see answer below for details and solution
What i though was an issue with authentication what actually an issue with the git service not knowing which ssh key to use as i had multiple on the server.
The solution was to use a config file in the .ssh folder and assign alias to specify which ssh key to use for git operations in separate repositories.
Solution is here: Gist with solution
This gist explains the general idea, it suggests using sub-domains however a comment further down uses alias which seems neater.
I have now resolved the issue and the system is working fine with a read-only, passphrase-less deploy key.
This can be done by customizing the GIT_SSH_COMMAND. As ssh .config only gets the host, you have to create aliases to handle different paths. Alternatively, as the git CLI sends the path of the repo to the GIT_SSH_COMMAND, you can intercept the request in a custom script, added in between git and ssh.
You can create a solution where you extract the path and add in the related identity file, if available on the server.
One approach to do this can be found here.
Usage:
cp deploy_key_file ~/.ssh/git-keys/github-practice
GIT_SSH_COMMAND=custom_keys_git_ssh git clone git#github.com:github/practice.git
I want to deploy django app to heroku via codeship and it generate an error and that is wget -O/dev/null http://something.herokuapp.com
How to fix the problem
Marko from the Codeship team here. Could you paste some more log output, or open a ticket on https://helpdesk.codeship.com?
Judging from the command you pasted above, I'd guess the check if the Heroku app, that you just deployed to, is running fails and this causes the build itself to fail.
Without any additional configuration we check the root URL for your application at http://HEROKU_APP_NAME.herokuapp.com via wget and see if it returns an HTTP/2xx or HTTP/3xx status code.
If it does, the check succeeds, else the build fails.
You can take a look at the script we use to perform the check at https://github.com/codeship/scripts/blob/master/utilities/check_url.sh
Does anyone know of any good articles on getting Jenkins to play nicely with Heroku?
What I want to do is:
1) Setup a Jenkins job to poll a private GitHub Repo when check-ins are made to developer branch.
2) Build this branch and make sure everything is good!
3) Push private GitHub Repo codebase to Heroku Repo. So it does the build and deploy on Heroku.
I've seen bits and pieces of articles but can't seem to get the complete flow to work. I've tried the GitHub plugin & Heroku plugin. I CAN get the GitHub plugin to pull down and build but I don't understand how to push to Heroku. Heroku plugin lets me deploy a WAR file but that doesn't bring up the app correctly. So I need to just push the codebase to the Heroku Repo so it does the compile and deploy.
I use Jenkins to push to Heroku for our apps. I dont use the Heroku plugin, I like the control that 'Execute Shell' gives me. This is a rather detailed answer, if I missed anything be sure to comment.
1) Polling a Private Repo :
Your job should be set up with the option 'Build when a change is
pushed to GitHub' under the 'Build Triggers' section.
on GitHub go to your project page, and click on the Settings menu in the
header (must have admin access). In the left sidebar of that page click 'Service Hooks'.
This will take you to a list of hooks you can choose. Select 'Jenkins (GitHub plugin)'.
Add the callback URL of your jenkins server (leave that page open for later). Something like :
http://jenkins.example.com/github-webhook/
You can test the callback by adding a 'Log Recorder' from Jenkins, by going to Jenkins|Manage Jenkins|System Log. Click 'Add New Log Recorder'
Enter 'test hook', Set Logger to 'com.cloudbees.jenkins.GitHubWebHook', Set Log Level to 'All'
Go back to GitHub, and click 'Test Hook', you can then see the callback log to confirm your hook is working.
2) Build the branch. Be sure you have all the GitHub configs set, as the callback will trigger the job only if these settings are done.
In the 'Source Code Management' section, Select 'Git' option and fill in the details of your repo, e.g. 'git#github.com:...'
In the 'Build Triggers' section, Select 'Build when a change is pushed to GitHub'
3) Push to Heroku. A few things to consider here.
You need to ensure your job had the Heroku remote repo added. When creating a new job this is a ONE time action, and does not need to be done for every build. For example :
heroku git:remote -a myApp
git remote add heroku git#heroku.com:myApp.git
create an Execute Shell script with just the above, for use only on your first build.
If you have Ping Targets (New Relic), disable them during deploy to avoid false notifications that your app is down.
curl https://heroku.newrelic.com/accounts/ACCTID/applications/APPID/ping_targets/disable -X POST -H "X-Api-Key: APIKEY"
Dont forget to turn it back on after:
curl https://heroku.newrelic.com/accounts/ACCTID/applications/APPID/ping_targets/enable -X POST -H "X-Api-Key: APIKEY"
Do the same for Maintenance Mode on the App
heroku maintenance:on --app myApp
heroku maintenance:off --app myApp
Putting this together, a typical deploy script on Jenkins may look like this :
#one off to ensure heroku remote is added to repo
heroku git:remote -a myApp
git remote add heroku git#heroku.com:myApp.git
#disbales
curl https://heroku.newrelic.com/accounts/ACCTID/applications/APPID/ping_targets/disable -X POST -H "X-Api-Key: APIKEY"
heroku maintenance:on --app myApp
#push to heroku
git push --force heroku master
heroku run rake db:migrate --app myApp
#enables
curl https://heroku.newrelic.com/accounts/ACCTID/applications/APPID/ping_targets/enable -X POST -H "X-Api-Key: APIKEY"
heroku maintenance:off --app myApp
As an alternative to using the Heroku API as outlined above to deploy, you can simply push your code to a remote Git repository (i.e. the one Heroku defined for your app) as a Post-Build Action. Your job would therefore define two Git repositories -- one being your Github repository and another being the Heroku one.
Give the Heroku repository a name, such as 'heroku' and in the Post-Build Actions section, use a Git publisher. Be sure to select the heroku name in the Target Remote name field.
Depending on how you've set up your Build Trigger on your Github project, when a build completes, Jenkins will push the resultant snapshot to the Heroku repository, resulting in a deployment.
First you should switch your project to use ssh-keys for authentication and add your public ssh-key to your heroku account:
heroku git:remote --ssh-git
heroku keys:add
With
git remote -v
heroku git#heroku.com:your-project.git (fetch)
heroku git#heroku.com:your-project.git (push)
you can figure out the URL to use as repository URL in your Jenkins configuration. Then follow the instructions given in the answer from #Andy Glover (see also http://thediscoblog.com/blog/2014/01/24/continuous-delivery-for-heroku-with-jenkins/) to configure your Jenkins.
Finally to allow Jenkins to push to your git repository at Heroku, you should ask your Jenkins administrator to give you the public ssh key of the root/jenkins user executing your Jenkins job. Add this public key via your Heroku dashboard to your Heroku account. If your build job is successful, Jenkins will then be able to push to your Heroku account, resulting in a deployment.
I have my rails application hosted on github. Whenever a commit happens on 'dev' branch I would like to do git pull on the dev box and also invoke '/etc/init.d/apache2 restart' command. Is this something I can do with Capistrano deployment or I can write some kind of custom shell script and somehow hook in my rails app to call that shell script?
You will need a post-receive hook on the git server that causes a git pull on the dev box. A quick DAFS for "git deploy with post-receive" found quite a few resources, like http://ryanflorence.com/deploying-websites-with-a-tiny-git-hook
I would personally prefer using a build system like Jenkins for this as it gives you more control and visibility. We have set up a continuous deployment system using Jenkins and github post-receive hooks with relatively little fuss. Another advantage is that you can run your tests before automatically deploying (if they pass) by chaining jobs.