I'm try to update my private repository via a cron job but nothing seems to be happening. I'm using Satis to create the repository for my private packages.
I can do this manually by logging into my account via SSH and running:
php bin/satis build satis.json ./ -n
which updates everything fine apart from I have to enter my passphrase a million times. I can get round this by using SSH Agent and think this may be my problem...
php /home/accountname/public_html/bin/satis build /home/accountname/public_html/satis.json /home/accountname/public_html/ -n
Is there anything I'm missing?
UPDATE
It is the SSH auth as I've received this error via email (shortened version)
Reading composer.json of vendor/package
(master)
Failed to update
git#bitbucket.org:vendor/package.git, package information from this
repository may be outdated (Permission denied (publickey). fatal: The
remote end hung up unexpectedly error: Could not fetch origin )
In the docs it says about using -n to use the SSH Key but I am using it.
Any ideas?
UPDATE
Thought I would add my satis file structure:
{
"name": "Name Of My Repo",
"homepage": "http://repodomain.co.uk",
"repositories": [
{ "type": "vcs", "url": "git#bitbucket.org:vendor/package.git" }
],
"require-all": true
}
really stuck on this one, the documentation is crap!
You are right that the SSH authentication is the problem here.
When the cronjob is running, it must access your Bitbucket repository somehow. You chose to use the "git" protocol, which uses SSH. When using SSH, using key-based authentication is way better than using passwords - and when it comes to git repos, it is the only way most of the time depending on the hosting.
Either you put your private key onto that cron server to get access to Bitbucket (might be a bad idea), or you create a new key pair and use that to allow access for your repo (is better, unless you experience some limitations of your bitbucket account, like too many users if that key counts as a user - on the other hand you could limit this key to be only allowed to read, not write).
Make sure that the user that is used to run your cronjob is using these keys, e.g. you should be able to manually start the script without any agent, and it should finish without asking for passwords. The correct place for the private key is ~/.ssh, the public key goes to Bitbucket. After that everything should run perfectly in the cronjob.
The other way could be to use a different protocol (like HTTPS) for the repo access and see what happens.
Related
I'm trying to configure git so I don't have to keep authenticating (and also learn a bit more about git).
I had previously been using password-based authentication, which is repetitive to keep typing in my passwords. I followed the steps (using windows) to set up SSH keys in github. I gave my key a specific name, anticipating that I will need more than one SSH key at some point. It doesn't make sense to always have id_rsa for github! I received the email that the key was successfully created, and I used git bash (because windows) to start the SSH client silently and added my private key. But running a git clone gave me this error:
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
I then had the idea to try it with the default name (id_rsa), rather than my custom name, and of course it works. So my question is how do I avoid this? I don't want the default name to be reserved for github.
You can set up an ssh configuration file to tell the ssh command to use that specific key when connecting to github.
Create the file .ssh/config with the content:
Host github.com
IdentityFile ~/.ssh/github_rsa_key
Assuming that you've named your private key ~/.ssh/github_rsa_key.
Now try connecting to github:
$ ssh git#github.com
You should see:
Hi <your github username>! You've successfully authenticated, but GitHub does not provide shell access.
I'm trying out WSL 2 on windows 10 and it's gone well so far, but I've been struggling for 2 weeks to make this work, because for some reason go get doesn't use or is not able to make the Git Credentials Manager to prompt for my credentials.
I followed this blog to set up WSL2 with GCM https://www.edwardthomson.com/blog/git_credential_manager_with_windows_subsystem_for_linux.html
And it works very good for most of the daily tasks like cloning, read and write. But when using go get I get this error.
go get <remote github repo>#<latest commit id>
go: <remote github repo> 681dceefc81203e094872401c184d038090d6049 => v0.0.17-0.20200501212733-681dceefc812
go get: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: verifying module: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: reading https://sum.golang.org/lookup/<remote github repo>#v0.0.17-0.20200501212733-681dceefc812: 410 Gone
server response:
not found: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/gopath/pkg/mod/cache/vcs/232ff028cb2fdebd254e30bfc612843483f0fe3fbeb18d5fc8fb4b20f21c9021: exit status 128:
fatal: could not read Username for 'https://github.com': terminal prompts disabled
Already tried ssh-keys and the solutions proposed here go get results in 'terminal prompts disabled' error for github private repo
But the error remains the same, when enabling env GIT_TERMINAL_PROMPT=1 nothing happens, I guess it's because WSL 2 doesn't have the permissions to do that. Anyway I also tried this tool https://github.com/microsoft/Git-Credential-Manager-for-Mac-and-Linux and by setting a variable for plain credentials store, it prompts in the terminal for credentials. But I'm using 2FA because it's required by the organization and the prompt only asks for username and password, so the authentication fails.
So I have to reach out to a mate who is using Mac. He is able to go get the dependency to affect go.mod, make a commit and push the change so I can pull it and continue from there. But of course this is not ideal, and he doesn't have any problem, he uses osxkeychain to manage his git credentials.
Anyone has faced this issue? or know how to solve it? Thank you so much in advance.
Go is not able to understand that certain modules are private and their checksum should not be validated against go's checksum library. The following error comes from that
verifying module: <remote github repo>#v0.0.17-0.20200501212733-681dceefc812/go.mod: reading https://sum.golang.org/lookup/<remote github repo>#v0.0.17-0.20200501212733-681dceefc812: 410 Gone
If possible use at least go 1.13 or, higher. Go had introduced an env variables by name GOPRIVATE, GONOPROXY and GONOSUMDB for managing private modules better. Simplest way to signal to Go that you are importing a private repo is to use GOPRIVATE. Set the pattern of private repos to GOPRIVATE env variable to suppress checksum validation and usage of GOPROXY. Example below avoids checksum for all repos in that hierarchy:
GOPRIVATE=github.com/<your org>/*
Check out answers here and here. You can also do go help module-private for help.
Background
I have a system where when I push changes to my Repository, A web hook sends a request to my site which runs a bash script to pull the changes and copy any updated files.
I added a second repository with its own deploy key but after doing so i was getting a permission denied error when trying to pull changes.
Question
Is there a way to use 2 deploy key's on the same server?
Environment Details
Site uses Laravel 5.6, Symfony used to run shell script
Git 1.7
Go Daddy web hosting (Basic Linux one)
Notes
Script just runs git pull command
Error given is " Permission denied (publickey) "
SHH is used as a deploy key so only read access, there is one other project also using a deploy key on the same server
Thank you in advance for you help! Any other suggestions are welcome!
Edit #1
Edited post to reflect true problem as it was different to what I though (Feel free to revert if this is bad practice), please see answer below for details and solution
What i though was an issue with authentication what actually an issue with the git service not knowing which ssh key to use as i had multiple on the server.
The solution was to use a config file in the .ssh folder and assign alias to specify which ssh key to use for git operations in separate repositories.
Solution is here: Gist with solution
This gist explains the general idea, it suggests using sub-domains however a comment further down uses alias which seems neater.
I have now resolved the issue and the system is working fine with a read-only, passphrase-less deploy key.
This can be done by customizing the GIT_SSH_COMMAND. As ssh .config only gets the host, you have to create aliases to handle different paths. Alternatively, as the git CLI sends the path of the repo to the GIT_SSH_COMMAND, you can intercept the request in a custom script, added in between git and ssh.
You can create a solution where you extract the path and add in the related identity file, if available on the server.
One approach to do this can be found here.
Usage:
cp deploy_key_file ~/.ssh/git-keys/github-practice
GIT_SSH_COMMAND=custom_keys_git_ssh git clone git#github.com:github/practice.git
I know that thousands of topics touch my problem but I am fighting this from several days with almost zero progress ;(.
I installed Jenkins on Windows machine.
Then acquired Git plugin and the Git itself.
Everything is latest version for the day of my writing.
I was just happy to start to play with Jenkins but I failed at configuring my Bitbucket repository to be used by Job.
I set the exact path to git.exe (as it seemed the reason for initial problems) [C:\Program Files\Git\cmd\git.exe].
Then I tried accessing the repository by HTTPS with account credentials, many strange errors here so I thought let's try with SSH.
I have generated rsa keys, set the public key in the Bitbucket repository. Then I created credential configuration in Jenkins for SSH authentication with passphrase.
I have many problems here so for temporary workaround I explicitly set the value of private key.
Finaly I got to the place where I am now:
+refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout:
stderr: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
When I try to manualy copy repository, it politely asks me about passphrase and the copy is done. When Jenkins job does it the above happens. With help of %USERNAME% and %USERDOMAIN% in the Job's script I get the same values like when I do it manually.
After plenty of hours I can say I can't use Git in Jenkins ;(
UPDATE
I don't believe that it could be so simple. As mainframer mentioned, I left passphrase empty for the whole SSH authentication and it really worked (the repository is cloned even that Jenkins still shows the same error in job's configuration page). I only wonder about any consequences of such setup, but working thing is more valuable for me now.
You mean leaving empty passphrase while generating keys and then also no passphrase config in Jenkins ssh credentials?
Yes, as Jenkins wouldn't be able to feed the passphrase to the ssh session.
But make sure the Jenkins slave service runs with your account (and not with the system account). It needs your account to find the ssh keys in %USERPROFILE%\.ssh\id_rsa(.pub)
I have tried following how-set-up-your-own-private-git-server-linux and private-remote-git-repositories-ubuntu-linode but I am still having problems.
My local environment is windows and my remote linux. I have a couple of questions:
In the first article it describes setting up a public ssh key for the server - I've never done this before and I'm not sure where to type the commands (not even sure whether local or remote!!): ssh myuser#server.com mkdir .ssh and scp ~/.ssh/id_rsa.pub myuser#server.com:.ssh/authorized_keys
On my local dev machine should I be running msysgit? Is there an alternative because the version I installed is taking up 1.4 GB! I installed msysGit-netinstall-1.7.4-preview20110204.exe from http://code.google.com/p/msysgit/downloads/list
I tried skipping making the git user and public key, created the repositories on the remote machine but then when I try git remote add origin ssh://[username]#[domain/ip/hostname]/srv/git/[project-name] as root user it says: fatal: Not a git repository (or any of the parent directories): .git
I'm not sure if this should be here or if it would be best migrated over to a different site, but since I might be able to help I'll go ahead and answer.
I just skimmed the articles you linked. It looks like they both deal with accessing a git server over ssh, which you mention, so that's what I'll focus on.
First, on your server:
You need to set up an account on the server so that you can log in. This can be either a generic git account, or your own personal account. For the moment we'll assume that you are setting it up to work with a personal account. What you want to do is create your account, then somewhere accessible to that account (say, in your home directory), create the git repository.
mkdir myrepo.git
cd myrepo.git
git --bare init --shared=all
So now you have a git repository up on the server. Depending on the git client you are using, you might not need to mess with keys right now. If SSH is configured on your server to allow password login then you can probably just connect and enter your password when you need to interact with the server. If you want to set up keys, what you need to do is to generate an ssh public key. I don't know off hand how to do this in windows, but in linux you'd do something like:
ssh-keygen -t rsa -b 1024
That command will generate two files, "id_rsa" and "id_rsa.pub"; whatever tool you use should also generate two files, a public key and a private key. They might have different names, but assume for now that "id_rsa.pub" is the name of your public key file.
You want to copy the public key to the server, you can use scp, ftp, or just move it over using a thumbdrive. Either way, once you get it onto the server, and it's accessible by your user, log in as your user on the server. You want to add the public key to your "authorized_hosts" file, so after logging in to your account on the server, do this:
cd
mkdir .ssh
cat id_rsa.pub >> .ssh/authorized_hosts
rm id_rsa.pub
Now, from your workstation you need to configure your ssh client to use the private key your generated. Again, I don't know how to do this on Windows and it will probably vary depending on what ssh client you are using, so you'll need to get that information somewhere else.
Next, you need to create your local repository, add some files, and make a commit. Note that you can't clone the remote repository you made yet because there's nothing in there. Once you have some commits made locally, you need to set the remote server in your repository.
If you are using the git command line tools, you can run:
git remote add origin user#yourserver:myrepo.git
If you put the repository somewhere other than your home directory, use the full path:
git remote add origin user#yourserver:/path/to/repo.git
Note that you need the ".git" in there, since your directory name has ".git" as part of the name.
If you are using a GUI tool, then you instead just edit the configuration file for the repository. This will be in the top level of your repository under ".git/config". You'll want to add something like
[remote "origin"]
url = user#yourserver:/path/to/repo.git
fetch = +refs/heads/*:refs/remotes/origin/*
Now that your remote is configured, and you have some commits locally, you can push your master branch up to the server. If you're using the command line use:
git push origin master
Or, if you're working on a different branch:
git push origin mybranch
If you are using a GUI frontend for get then you'll need to look up the documentation for that tool on how to push.
I would like to recommend to use Gitolite that allows to setup git hosting on central srever with great features like access control and creating/managin repositories etc.
And for windows, the tool called git-scm can be used for setting up git client and also have GUI panel for git repositories. After installation of this git tool, you will get a git bash tool and a git GUI.
Gitolite makes settting up of a git hosting server and client very simple.
I hope this may help new people who are looking for an efficient solution for settting up git hosting server and clients in some easy to understand simple steps.
I followed the most voted answer and I didnt get it to work. I found this tutorial:
https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2
and I got it working. I think the problem was that, in the server, the public key is to be copied to a file "authorized_keys" and not "authorized_hosts".
I had been trying to generate the key pairs using putty but couldn't get it to work. In the end I realised that I could generate the keys via the msysgit command line using ssh-keygen -C "git#example.com" -t rsa. Copy the public key to the server using nano /home/git/.ssh/authorized_keys and hey presto!
I had downloaded the full source code hence the size! The non-source app, Git-1.7.4-preview20110204.exe was just 13MB via http://code.google.com/p/msysgit/downloads/list?can=3