I have a bash script that run commands on a windows machine (lets call it machine A) to pull a repository on bitbucket (using git) and then run specific programs to work on the pulled content.
I have also a ubuntu machine running Jenkins. On a build step I use the "Publish Over SSH" plugin to run the bash script on the machine A.
Everything goes fine all the times unless git has to pull a commit containing a large file and its fails with this error :
Downloading test x_map.fbx (31 MB) Error downloading object: x_map.fbx (63b3f85): Smudge error: Error downloading x_map.fb x
(hash): batch request: git#bitbucket.org: Permission denied (publickey).:
exit status 255
The problem is that the LFS part of git doesn't use the SSH key even though I did set the core.sshCommand in git config to use a key for all the ssh commands.
So when I do this :
ssh git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod
its fails with this message : git#bitbucket.org: Permission denied (publickey)
But it works fine when I run it with the ssh key option :
ssh -i "ssh-key-path" git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod
So I was wondering if there was any solution to kind of overload all the LFS ssh commands to use a specific ssh key like the core.sshCommand in git config.
I m hardly stuck on this so I hope you guys have a miraculous solution! Thanks in advance for your help.
EDIT :
adding prior to the git pull command :
eval $(ssh-agent -s)
ssh-add "key-path" makes this command work : [ ssh git#bitbucket.org git-lfs-authenticate "reopsitory" downlaod ] but the git pull still fails with the same error
Related
I cannot clone or push to a repository on my server.
I have a bare repo that is located is a directory user#host in directory home/user/test.git that I am trying to access via git clone. I used ssh-add <pathtokey> to add my ssh key. It asked me for the passphrase. I can then ssh user#host successfully.
However if I then try to git clone ssh://user#host/~/test.git I get:
Cloning into 'test'...
user#host: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
also tried
git clone ssh://user#host/home/user/test.git
git clone user#host:home/user/test.git
git clone user#host:/home/user/test.git
with the same result
I am guessing the git credential manager isn't picking up the keys?
On the server /var/auth/log says
Feb 20 02:25:36 xxxxx sshd[24674]: Connection closed by authenticating user XXXX x.x.x.x port 56433 [preauth]
Git version: git version 2.30.1.windows.1
Git Credential Manager: Git Credential Manager version 2.0.318-beta+44acfafa98 (Windows, .NET Framework 4.0.30319.42000)
git config -l reports credential.helper=manager-core
Tried both PowerShell and git bash shells, same result
user has read, execute permissions to the repo
To add to #VonC's response
In git-bash things work as normal.
The normal flow of starting the ssh-agent (via eval 'ssh-agent'
), adding the key via ssh-add <path_to_key> enables git clone to work.
In PowerShell Core, or Cmd, via Windows-Terminal more work is required
the ssh-agent starts automatically (assuming you previously started the OpenSSH Authentication Agent service), adding the key works, and you can ssh after this, but git commands do NOT work, initially, but if you do
git config --global core.sshCommand C:/Windows/System32/OpenSSH/ssh.exe
This will replace the (default) ssh that comes with git-for-windows with the Windows10 implementation.
After this it should work fine in things other than git-bash. Confirmed in Powershell-Core, Command Prompt
See also:
https://gist.github.com/danieldogeanu/16c61e9b80345c5837b9e5045a701c99
The git credential manager is only involved for caching credentials (username/password) for HTTPS URL, not SSH.
Only the ssh-agent could be involved, for caching a possible passphrase, if the private key was defined with it.
I would try first using the full path, since ~ might not be interpreted by the remote shell, but the local (which has a different path for ~):
git clone ssh://user#host/home/user/test.git
# or
git clone user#host:/home/user/test.git
If not, in a git bash session, type:
export GIT_SSH_COMMAND='ssh -v'
git clone ...
The OP confirms in the discussion it works in a bash session:
In git bash, I started the ssh-agent,
added the key there, then it worked.
I want to perform CI/CD on remote server running on windows for hosting my VueJs project. For doing that I have performed the following steps
I have added my ssh of my local machine to remote server as authorised keys and I am able to access the admin(cmd) remote server via ssh by using the following command.
ssh remote_server#<ip_address>
Using the above command the git bash opens cmd with following location c:/users/remote_user1 of remote server where I had added ssh of my local machine as authorized key.
I created bare git repo in C:/users/remote_user1 by logging in remote computer by rdp using the following command
git init test.git --bare
I gave Full access rights to test.git from remote server
I tried to clone test.git on my local machine using the command from git bash
git clone ssh://remoteserver#<ip_address>:/test.git
when I use the command from my local machine I get the error message from git bash
fatal: ''/test.git'' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I am not able to understand what am I am doing wrong here, Please guide me on this.
The format for ssh urls must be one of :
# 'ssh://' url :
ssh://remoteserver#<ip_address>/test.git
# scp-like syntax :
remoterserver#<ip_address>:test.git
(link to docs)
The url you posted contains a :/ which makes it a mixture of both, you should modify this url to match one of the two accepted formats.
When i run git pull origin master from the terminal on my ubuntu server it works perfectly.
I have 3 repositories and it works in each repository.
Now when i try to run the same command through the "push" from the webhooks i'm getting :
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
The webhook seem to execute correctly :
[webhook][ff2a14] executing /home/ubuntu/projects/webhooks/admin-hooks/deploy.sh (/home/ubuntu/projects/webhooks/admin-hooks/deploy.sh) with arguments ["/home/ubuntu/projects/webhooks/admin-hooks/deploy.sh"] and environment [] using /home/ubuntu/projects/myproject/foofolder/foofolder2 as cwd
and calling the bash file from the right directory : /home/ubuntu/projects/myproject/foofolder/foofolder2
the bash file is simply :
#!/bin/bash
git pull origin master
if i change directory manually and call the bash file from the right directory : /home/ubuntu/projects/myproject/foofolder/foofolder2 and run git pull origin master it will work.
The rsa key seem to be used properly since i can pull manually ( no ssh-add used , it's an ssh config file with files that don't have a passphrase... )
Last thing to note is that altought all 3 repositories in my hooks are configured there's one which is working fine..
Edit :
i've just figured out that :
If i run the webhooks handler starting with "sudo" the hooks works for repo 1 and not for repo 2 and 3.
BUT if i run the handler without sudo it doesn't work for repo 1 or 2 but will work for the 3rd one.
Each of my repository has it's own rsa key set registered in the ssh config file and configured in each repository.
It is better for your hook to be more explicit, and:
display its user (id -a), to check who is executing the webhook.
displey the content of ~/.ssh (ls -alrth ~/.ssh)
cd to the right path of the local repository
display the remote origin URL used (git remote -v), to check it is an SSH one (git#github.com:...)
If the script works with sudo for repo1, that means /root/.ssh has the right id_rsa private key.
But first: make sure those repositories actually use an SSH URL.
I am trying to nest a shell script into a Jenkins job but the script will infinitely get stuck when using the bash git clone command.
It doesn't even send me an error. I suspect the script is asking for a password when coming to the git clone command.
Jenkins Job Configuration :
#!/bin/sh
set -x
whoami #returns 'jenkins'
sudo -u root ./my_script.sh
my_script.sh
whoami #returns ROOT
sudo git clone 'https://myusername:mypassword#my.forge.fr/project/my-project.git' -b "master" --verbose
What have I tried so far
I edited my sudoers and added jenkins ALL=NOPASSWD: ALL
I intentionnaly wrote wrong credential in the git clone command, and I get an error remote: HTTP Basic: Access denied
With a terminal, I went in my workspace project, switched to jenkins user and ran sudo ./my_script.sh and it worked fine, no password asked.
It also work fine when ran under root user ( not a surprise )
Does anyone know how I can get my job to make this command pass ?
Edit
I am on Mac OS
I finally solved it, it did come from a credential, but not the one I thought of. The problem came from Keychain Access. Also I appologize, I did not specified I was on Mac, I thought it wasn't relevant here since I was using bash.
When running the script under the Jenkins user, it worked fine as I was using sudo, but when I tried to run directly the sudo git clone 'https://myusername:mypassword#my.forge.fr/project/my-project.git' -b "master" --verbose I got an error :
I couldn't find the error message in english, but basically it says "unable to find a Keychain to store..." and the answers "cancel" and "reset my default keychain"
From there I :
Created a new Keychain called jenkins-dev in the Keychain manager
I copied it from /Users/myuser/Library/Keychains to /Users/Shared/Jenkins/Library/Keychains under myuser
Then I ran security unlock-keychain /Users/Shared/Jenkins/Library/Keychains/jenkins-dev.keychain-db under jenkins user
finally security default-keychain /Users/Shared/Jenkins/Library/Keychains/jenkins-dev.keychain-db under jenkins user
I ran once more the git clone ... command line under jenkins user, and then it found the keychain, stored the access and my Jenkins script wasn't stuck anymore
Hope this can help
Do you have a key in ~/.ssh/id_rsa.pub in that machine?
I installed Jenkins CI from homebrew (brew install jenkins). So, it runs under my own user. Everything works fine except for SSH access to my Git repository. I do not want to create a separate user (e.g. jenkins) to run Jenkins and create separate SSH keys for it. I just want to use my own SSH keys. Accessing my remote git repo on the command-line works fine with my ssh keys, but in Jenkins it does not work. It gives a 'Permission denied (publickey)' error (username and project are obfuscated (<user> and <project>)):
Started by user anonymous
Checkout:workspace / /Users/<user>/.jenkins/jobs/<project>_run_tests/workspace - hudson.remoting.LocalChannel#456d3d51
Using strategy: Default
Last Built Revision: Revision 093f1641f26632afd1a74c14276ec544356c1d08 (origin/dev)
Checkout:workspace / /Users/<user>/.jenkins/jobs/<project>_run_tests/workspace - hudson.remoting.LocalChannel#456d3d51
Fetching changes from 1 remote Git repository
Fetching upstream changes from ssh://git#bitbucket.org/<user>/<project>.git
ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway
ERROR: (Underlying report) : Error performing command: /usr/local/bin/git fetch -t ssh://git#bitbucket.org/<user>/<project>.git +refs/heads/*:refs/remotes/origin/*
Command "/usr/local/bin/git fetch -t ssh://git#bitbucket.org/<user>/<project>.git +refs/heads/*:refs/remotes/origin/*" returned status code 128: Permission denied (publickey).
fatal: The remote end hung up unexpectedly
ERROR: Could not fetch from any repository
FATAL: Could not fetch from any repository
hudson.plugins.git.GitException: Could not fetch from any repository
at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:1012)
at hudson.plugins.git.GitSCM$2.invoke(GitSCM.java:972)
at hudson.FilePath.act(FilePath.java:783)
at hudson.FilePath.act(FilePath.java:765)
at hudson.plugins.git.GitSCM.checkout(GitSCM.java:972)
at hudson.model.AbstractProject.checkout(AbstractProject.java:1195)
at hudson.model.AbstractBuild$AbstractRunner.checkout(AbstractBuild.java:568)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:457)
at hudson.model.Run.run(Run.java:1404)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:230)
It seems it is not using my ssh keys to connect to the git server. I tried adding a UserName key to the jenkins launchagent plist, did a restart, but no effect.
How can I make jenkins use my SSH keys to connect to my remote git repo?
I do not know how homebrew sets up Jenkins, but apparently it is running in a context where it has no access to your ssh agent and your ssh key is passphrase-protected.
When you run it in a Terminal window, you have SSH_AUTH_SOCK variable in your environment which ssh client uses to authenticate. If you cannot use the ssh agent, you need to remove the passphrase from the ssh key.