I am trying to nest a shell script into a Jenkins job but the script will infinitely get stuck when using the bash git clone command.
It doesn't even send me an error. I suspect the script is asking for a password when coming to the git clone command.
Jenkins Job Configuration :
#!/bin/sh
set -x
whoami #returns 'jenkins'
sudo -u root ./my_script.sh
my_script.sh
whoami #returns ROOT
sudo git clone 'https://myusername:mypassword#my.forge.fr/project/my-project.git' -b "master" --verbose
What have I tried so far
I edited my sudoers and added jenkins ALL=NOPASSWD: ALL
I intentionnaly wrote wrong credential in the git clone command, and I get an error remote: HTTP Basic: Access denied
With a terminal, I went in my workspace project, switched to jenkins user and ran sudo ./my_script.sh and it worked fine, no password asked.
It also work fine when ran under root user ( not a surprise )
Does anyone know how I can get my job to make this command pass ?
Edit
I am on Mac OS
I finally solved it, it did come from a credential, but not the one I thought of. The problem came from Keychain Access. Also I appologize, I did not specified I was on Mac, I thought it wasn't relevant here since I was using bash.
When running the script under the Jenkins user, it worked fine as I was using sudo, but when I tried to run directly the sudo git clone 'https://myusername:mypassword#my.forge.fr/project/my-project.git' -b "master" --verbose I got an error :
I couldn't find the error message in english, but basically it says "unable to find a Keychain to store..." and the answers "cancel" and "reset my default keychain"
From there I :
Created a new Keychain called jenkins-dev in the Keychain manager
I copied it from /Users/myuser/Library/Keychains to /Users/Shared/Jenkins/Library/Keychains under myuser
Then I ran security unlock-keychain /Users/Shared/Jenkins/Library/Keychains/jenkins-dev.keychain-db under jenkins user
finally security default-keychain /Users/Shared/Jenkins/Library/Keychains/jenkins-dev.keychain-db under jenkins user
I ran once more the git clone ... command line under jenkins user, and then it found the keychain, stored the access and my Jenkins script wasn't stuck anymore
Hope this can help
Do you have a key in ~/.ssh/id_rsa.pub in that machine?
Related
I'm currently using Jenkins on Windows 10, and using git as version control system.
Although I provided correct repository URL and credential, I cannot use jenkins with error below.
Failed to connect to repository : Command "git.exe ls-remote -h REPOSITORY_URL HEAD" returned status code 128:
stdout:
stderr: git#URL: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Credential and URL doesn't seems wrong, since this build success previously with same credential. The only change after successful build was one line in build script.
git config --global core.sshCommand C:/Windows/System32/OpenSSH/ssh.exe
After I success with this script, git setting keeps failed. Is there any problem with that line? And how can I fix it?
EDIT
Problem solved! For anyone who has same problem, it was problem with the system user! Since ssh key stores in each user space, jenkins cannot detect where the ssh key located. Therefore, go to 'service' in windows, and change user of jenkins service to the user who has correct ssh key. It solved my problem!
Seems there is some problem in Jenkins.
It cannot locate OpenSSH folder in System32, and so that I cannot get log with it.
That would explain why the main Jenkins controller (aka "master" in old terminology) cannot contact the Git repository (assuming an SSH URL here, with technical remote user account "git")
Try and remove that git configuration to see if the error persists: Git should fall back to its own ssh.exe, packaged with Git For Windows.
As noted by the OP, this only works if said Jenkins is run as a user account, not as the system account.
Only then will it be able to access the %USERPROFILE%\.ssh folder.
I can't seem to su to jenkins user even though the login shell is already set to /bin/bash. When I attempt to run sudo su - jenkins -s /bin/bash, the user does not switch and I see the same prompt.
I've rebooted the machine and restarted the service using sudo systemctl restart jenkins.
I made sure that the jenkins login shell is not /bin/false like this question points out. My /etc/passwd shows this for jenkins:
jenkins:x:121:129:Jenkins,,,:/var/lib/jenkins:/bin/bash
Jenkins does run when I reboot the machine, but the service fails to start when I try to restart it.
Any idea why I can't su into jenkins?
Thanks
Edit: the below solution is not what fixed it for me. I'm still not sure what the root cause was, it may have been related to the Jenkins user not having read permission to some files, but I resolved it by reinstalling Jenkins.
It turns out that my /etc/sudoers file had:
jenkins ALL=(ALL:ALL) ALL
I changed it to:
jenkins ALL= NOPASSWD: ALL
and now I can switch to the jenkins user.
I have a bash script that run commands on a windows machine (lets call it machine A) to pull a repository on bitbucket (using git) and then run specific programs to work on the pulled content.
I have also a ubuntu machine running Jenkins. On a build step I use the "Publish Over SSH" plugin to run the bash script on the machine A.
Everything goes fine all the times unless git has to pull a commit containing a large file and its fails with this error :
Downloading test x_map.fbx (31 MB) Error downloading object: x_map.fbx (63b3f85): Smudge error: Error downloading x_map.fb x
(hash): batch request: git#bitbucket.org: Permission denied (publickey).:
exit status 255
The problem is that the LFS part of git doesn't use the SSH key even though I did set the core.sshCommand in git config to use a key for all the ssh commands.
So when I do this :
ssh git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod
its fails with this message : git#bitbucket.org: Permission denied (publickey)
But it works fine when I run it with the ssh key option :
ssh -i "ssh-key-path" git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod
So I was wondering if there was any solution to kind of overload all the LFS ssh commands to use a specific ssh key like the core.sshCommand in git config.
I m hardly stuck on this so I hope you guys have a miraculous solution! Thanks in advance for your help.
EDIT :
adding prior to the git pull command :
eval $(ssh-agent -s)
ssh-add "key-path" makes this command work : [ ssh git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod ] but the git pull still fails with the same error
I need to run a bash script that periodically deletes old git branches. I am having trouble finding a way to connect to the git repo via the execute shell option.
Currently I am using cygwin in order to run git commands. Here is what I have in execute shell:
#!c:\cygwin64\bin\bash --login
git ls-remote git#10.1.1.126:/external-web/collette-com.git
This command is throwing the following error.
[Delete Branches] $ c:\cygwin64\bin\bash --login
C:\Users\tbraga\AppData\Local\Temp\hudson5750784484659728632.sh
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Build step 'Execute shell' marked build as failure
I have tried running this command in the command line and am prompted for a password. Could this be the issue?
I have the git plugin configured within Jenkins and the connection works perfectly when using Source Code Management Git.
Any suggestions on how to make this connections work in the execute shell field would be greatly appreciated.
I solved this problem by passing my credentials to my execute shell script through the Credentials Binding Plugin in Jenkins
It's simple enough to create an SSH key associated with your user.
Try here : https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html
Put keys under %userprofile%/.ssh and try running it again.
You can also use the same credentials used in your Jenkins configuration
I use SSH keys for auth to Github and had this same issue. My Jenkins configuration has EC2 slaves, so the default SSH key on the machine wasn't correct for Github.
I fixed it with the SSH Agent Plugin. In the job, enable the "SSH Agent" setting and choose the stored SSH key for Github authentication. It should be the same one selected for the Git-SCM configuration used to clone the repo.
I have been trying to follow tutorials and this one: Deploy as Jenkins User or Allow Jenkins To Run As Different User?
but I still can't for the love of the computing gods, run as a different user. Here are the steps of what I did:
download the macosx pkg for jenkins(LTS)
setup plugins etc and git
try to build it
I keep getting a can't clone error because jenkins keeps starting as anonymous:
Started by user anonymous
How do I set it up so that jenkins runs as me? I was using the jenkins web UI so it was in localhost:8080
I tried logging in also using /login but I can't even login using my name or as root.
The people tab doesn't even have a create user link, so yeah I've been stuck. Help please?
The "Issue 2" answer given by #Sagar works for the majority of git servers such as gitorious.
However, there will be a name clash in a system like gitolite where the public ssh keys are checked in as files named with the username, ie keydir/jenkins.pub. What if there are multiple jenkins servers that need to access the same gitolite server?
(Note: this is about running the Jenkins daemon not running a build job as a user (addressed by #Sagar's "Issue 1").)
So in this case you do need to run the Jenkins daemon as a different user.
There are two steps:
Step 1
The main thing is to update the JENKINS_USER environment variable. Here's a patch showing how to change the user to ptran.
BEGIN PATCH
--- etc/default/jenkins.old 2011-10-28 17:46:54.410305099 -0700
+++ etc/default/jenkins 2011-10-28 17:47:01.670369300 -0700
## -13,7 +13,7 ##
PIDFILE=/var/run/jenkins/jenkins.pid
# user id to be invoked as (otherwise will run as root; not wise!)
-JENKINS_USER=jenkins
+JENKINS_USER=ptran
# location of the jenkins war file
JENKINS_WAR=/usr/share/jenkins/jenkins.war
--- etc/init.d/jenkins.old 2011-10-28 17:47:20.878539172 -0700
+++ etc/init.d/jenkins 2011-10-28 17:47:47.510774714 -0700
## -23,7 +23,7 ##
#DAEMON=$JENKINS_SH
DAEMON=/usr/bin/daemon
-DAEMON_ARGS="--name=$NAME --inherit --env=JENKINS_HOME=$JENKINS_HOME --output=$JENKINS_LOG - -pidfile=$PIDFILE"
+DAEMON_ARGS="--name=$JENKINS_USER --inherit --env=JENKINS_HOME=$JENKINS_HOME --output=$JENKINS_LOG --pidfile=$PIDFILE"
SU=/bin/su
END PATCH
Step 2
Update ownership of jenkins directories:
chown -R ptran /var/log/jenkins
chown -R ptran /var/lib/jenkins
chown -R ptran /var/run/jenkins
chown -R ptran /var/cache/jenkins
Step 3
Restart jenkins
sudo service jenkins restart
ISSUE 1:
Started by user anonymous
That does not mean that Jenkins started as an anonymous user.
It just means that the person who started the build was not logged in. If you enable Jenkins security, you can create usernames for people and when they log in, the
"Started by anonymous"
will change to
"Started by < username >".
Note: You do not have to enable security in order to run jenkins or to clone correctly.
If you want to enable security and create users, you should see the options at Manage Jenkins > Configure System.
ISSUE 2:
The "can't clone" error is a different issue altogether. It has nothing to do with you logging in to jenkins or enabling security. It just means that Jenkins does not have the credentials to clone from your git SCM.
Check out the Jenkins Git Plugin to see how to set up Jenkins to work with your git repository.
Hope that helps.
On Mac OS X, the way I enabled Jenkins to pull from my (private) Github repo is:
First, ensure that your user owns the Jenkins directory
sudo chown -R me:me /Users/Shared/Jenkins
Then edit the LaunchDaemon plist for Jenkins (at /Library/LaunchDaemons/org.jenkins-ci.plist) so that your user is the GroupName and the UserName:
<key>GroupName</key>
<string>me</string>
...
<key>UserName</key>
<string>me</string>
Then reload Jenkins:
sudo launchctl unload -w /Library/LaunchDaemons/org.jenkins-ci.plist
sudo launchctl load -w /Library/LaunchDaemons/org.jenkins-ci.plist
Then Jenkins, since it's running as you, has access to your ~/.ssh directory which has your keys.
If you really want to run Jenkins as you, I suggest you check out my Jenkins.app. An alternative, easy way to run Jenkins on Mac.
See https://github.com/stisti/jenkins-app/
Download it from https://github.com/stisti/jenkins-app/downloads
To run jenkins as different user on ubuntu os you need to change below things.
Update below two lines in /etc/default/jenkins file
JENKINS_USER=$USERNAME
JENKINS_GROUP=$NAME
In our case we set user as ubuntu.
#JENKINS_USER=$NAME
#JENKINS_GROUP=$NAME
JENKINS_USER="ubuntu"
JENKINS_GROUP="ubuntu"
Update below two lines in /lib/systemd/system/jenkins.service file
User=jenkins
Group=jenkins
In our case we set user as ubuntu.
#User=jenkins
#Group=jenkins
User=ubuntu
Group=ubuntu
Change file ownership of jenkins owned folders.
sudo chown -R ubuntu:ubuntu /var/lib/jenkins
sudo chown -R ubuntu:ubuntu /var/cache/jenkins
sudo chown -R ubuntu:ubuntu /var/log/jenkins
After above changes run below command to reload systemctl
sudo systemctl daemon-reload
Now you can restart jenkins
sudo systemctl restart jenkins.service
you can integrate to LDAP or AD as well. It works well.