I have a launch configuration and auto-scaling group set up. The launch config uses an AMI that I've already created, based on Ubuntu 14.04, that installs Nginx, Git, and has my static files stored as a Git repo in Nginx's /usr/share/nginx/html/ directory.
The problem: the static files in my nginx/html directory are only as new as the files that were loaded in the AMI when I created it.
To remedy this, I have tried to add a "User Data" field into the launch config. The field is defined as:
#! /bin/bash
cd /usr/share/nginx/html/
git pull origin master
<my git repo's password>
But when I check to see if the instance has the latest version of the repo, I see that it doesn't. Something is wrong with my script, and I'm not sure what.
I have tested entering these commands one-by-one exactly as is into the EC2 instance via SSH, and it works exactly as expected.
Why doesn't this work in the user data field?
Note: I have verified that the 'bash' file is indeed present in /bin/bash.
You need to pass username and password of your repository with the repo url
Sample example :
#! /bin/bash
cd /usr/share/nginx/html/
git clone https://username:password#yourRepoURL.git
Problem is definitely in the bash script. Everything it contains is executed by bash, so it actually tries to execute your password as a command.
There are multiple ways to provide a password to Git in a script. See for example this question: How to provide username and password when run "git clone git#remote.git"?
It basically depends on how secure do you need it. Maybe it's enough to have a plain text password in Git's config (it doesn't have to be so bad if you set a restricted mode for that file, it would be similar to using a private key without passphrase).
It's been a while since I asked this- I've learned a lot since then.
You can pass your username and password as part of the URL, but that is bad form as if you share the code with anyone or give anyone access to your script then they will know your account credentials.
The best way to do this would be to set up your server to connect to your Git repo over SSH - I believe this is industry best practice as it is more secure and password-less.
Related
I am git cloning ~3000 repositories in a bash script like this:
[loop-sy]
git clone https://foobar.com
[ys-pool]
Most of these are public repositories and it works fine, but some of them are private and they return a username and password challenge. Is there a way to just move to the next repo (i.e. fail and get on with it) rather than having the bash script stop for me to enter the username and password.
I have searched high and low, but I have not found an answer yet.
I am running an Ubuntu 18.04 server and have root privileges.
You can set GIT_TERMINAL_PROMPT to 0, which will prevent Git from prompting for credentials. It will still use any that it can find in any credential helper you may be using, but if those don't work, it will just fail instead of prompting.
I am trying to set up a git repository on our Windows server.
I installed OpenSSH for Windows and Git for Windows.
The OpenSSH server runs on port 2345 to filter out generic bots.
I created a bare and shared repository on C:\git\protocolrepo.git
I couldn't reach it with the URL:
ssh://User#domain#server.com:2345:C:/git/protocol.git
It seems to me to be a malformed URL but everywhere I look on the web I get this, but it just doesn't work.
As I found out in this thread: https://github.com/PowerShell/Win32-OpenSSH/issues/895, the shell might be the cause, so I set the powershell as the default shell of OpenSSH.
This didn't work either and I still needed to use this oddly formed URL.
So I installed Cygwin and set the Cygwin bash as the default shell, and used a more appropriately formed url:
ssh://User#domain#server.com:2345/cygdrive/c/git/protocol.git
I had a little bit more success with this, but it now states the following error message:
fatal: '/cygdrive/c/git/protocol.git' does not appear to be a git repository
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
The access rights should be fine as I make use of the same user alltogether, but I'm not 100% sure how this works on Windows.
I'm not sure either how Git/ssh handles this path on Windows.
On a Linux based server this worked just fine with /git/protocol.git path.
Pushing a new branch to a bare repository is something I haven't done in a while either so that also might be done wrong here, but I can't recall it to be an issue.
What would you recommend to debug this issue?
I would start by checking if an interactive SSH session works, before considering using a Git repo SSH URL.
ssh -p 2345 User#domain#server.com
If it does, try and access /C/git/protocol.git in that interactive shell.
If that works, then try for your git commands the URL
ssh://User#domain#server.com:2345/C/git/protocol.git
# or
ssh://User#domain#server.com:2345/c/git/protocol.git
Background
I have a system where when I push changes to my Repository, A web hook sends a request to my site which runs a bash script to pull the changes and copy any updated files.
I added a second repository with its own deploy key but after doing so i was getting a permission denied error when trying to pull changes.
Question
Is there a way to use 2 deploy key's on the same server?
Environment Details
Site uses Laravel 5.6, Symfony used to run shell script
Git 1.7
Go Daddy web hosting (Basic Linux one)
Notes
Script just runs git pull command
Error given is " Permission denied (publickey) "
SHH is used as a deploy key so only read access, there is one other project also using a deploy key on the same server
Thank you in advance for you help! Any other suggestions are welcome!
Edit #1
Edited post to reflect true problem as it was different to what I though (Feel free to revert if this is bad practice), please see answer below for details and solution
What i though was an issue with authentication what actually an issue with the git service not knowing which ssh key to use as i had multiple on the server.
The solution was to use a config file in the .ssh folder and assign alias to specify which ssh key to use for git operations in separate repositories.
Solution is here: Gist with solution
This gist explains the general idea, it suggests using sub-domains however a comment further down uses alias which seems neater.
I have now resolved the issue and the system is working fine with a read-only, passphrase-less deploy key.
This can be done by customizing the GIT_SSH_COMMAND. As ssh .config only gets the host, you have to create aliases to handle different paths. Alternatively, as the git CLI sends the path of the repo to the GIT_SSH_COMMAND, you can intercept the request in a custom script, added in between git and ssh.
You can create a solution where you extract the path and add in the related identity file, if available on the server.
One approach to do this can be found here.
Usage:
cp deploy_key_file ~/.ssh/git-keys/github-practice
GIT_SSH_COMMAND=custom_keys_git_ssh git clone git#github.com:github/practice.git
I'm trying to set up MS WebMatrix to use a Git repository from my siteground hosting account. I created the repository using their cpanel plugin and it tells me that I can clone it using this command
git clone ssh://username#sm3.siteground.biz:18765/home/username/public_html/
I replaced username of course and I created an rsa key using ssh-keygen. In the Webmatrix GUI it just opens a window saying "Clone is in progress" but it doesn't to anything.
And when I run that command in PowerShell, this is the output:
Cloning into 'public_html'...
Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Any help is highly appreciated!
EDIT:
I haven't used github before but I'm pretty sure I'm not connecting to it. The repository is on siteground's server I think. Anyway, I couldn't figure it out in PowerShell so now I'm using putty to load the appropriate key and connect using an external git tool (source tree) that doesn't use the same ssh client as PowerShell. That is the solution that's working for me now.
I'll leave this question open as maybe someone comes around and can help with how to set this up using PowerShell.
The missing piece to the Siteground guide is:
Create a blank file in ~/.ssh/ or C:\Users\username\.ssh on your computer. It does not matter what you name it. I named it siteground_dsa. You could also name it id_dsa_siteground.
Copy the private ssh key that you get from siteground.com and paste the whole of it in the this newly created file.
Open Git Bash locally on your computer and run the following command
$ eval ssh-agent -s
Then run the following. Remember to use the filename that you gave it.
$ ssh-add ~/.ssh/siteground_dsa
Now you need to enter the passphrase for the ssh key. You will have defined it when creating the ssh key.
Now you should be logged in and you can run git clone the directory of your wish.
git clone ssh://username#ams14.siteground.eu:18765/home/username/public_html/
To permanently add the SSH key extend ~/.ssh/config with the following and updating server_name and username.
Host server_name
User username
Port 18765
IdentityFile ~/.ssh/siteground_dsa
keep in mind that for Windows operators, you should write eval $(ssh-agent)
eval $(ssh-agent)
chmod 600 file_name
ssh-add C:\Users\username\.ssh\siteground
Then you can easily clone your file into your local server following inserting your passphrase of the SSH key.
GitHub isn't able to authenticate you. Probably your key isn't associated with your GitHub account.
Take a look to GitHub's recommended method
I have tried following how-set-up-your-own-private-git-server-linux and private-remote-git-repositories-ubuntu-linode but I am still having problems.
My local environment is windows and my remote linux. I have a couple of questions:
In the first article it describes setting up a public ssh key for the server - I've never done this before and I'm not sure where to type the commands (not even sure whether local or remote!!): ssh myuser#server.com mkdir .ssh and scp ~/.ssh/id_rsa.pub myuser#server.com:.ssh/authorized_keys
On my local dev machine should I be running msysgit? Is there an alternative because the version I installed is taking up 1.4 GB! I installed msysGit-netinstall-1.7.4-preview20110204.exe from http://code.google.com/p/msysgit/downloads/list
I tried skipping making the git user and public key, created the repositories on the remote machine but then when I try git remote add origin ssh://[username]#[domain/ip/hostname]/srv/git/[project-name] as root user it says: fatal: Not a git repository (or any of the parent directories): .git
I'm not sure if this should be here or if it would be best migrated over to a different site, but since I might be able to help I'll go ahead and answer.
I just skimmed the articles you linked. It looks like they both deal with accessing a git server over ssh, which you mention, so that's what I'll focus on.
First, on your server:
You need to set up an account on the server so that you can log in. This can be either a generic git account, or your own personal account. For the moment we'll assume that you are setting it up to work with a personal account. What you want to do is create your account, then somewhere accessible to that account (say, in your home directory), create the git repository.
mkdir myrepo.git
cd myrepo.git
git --bare init --shared=all
So now you have a git repository up on the server. Depending on the git client you are using, you might not need to mess with keys right now. If SSH is configured on your server to allow password login then you can probably just connect and enter your password when you need to interact with the server. If you want to set up keys, what you need to do is to generate an ssh public key. I don't know off hand how to do this in windows, but in linux you'd do something like:
ssh-keygen -t rsa -b 1024
That command will generate two files, "id_rsa" and "id_rsa.pub"; whatever tool you use should also generate two files, a public key and a private key. They might have different names, but assume for now that "id_rsa.pub" is the name of your public key file.
You want to copy the public key to the server, you can use scp, ftp, or just move it over using a thumbdrive. Either way, once you get it onto the server, and it's accessible by your user, log in as your user on the server. You want to add the public key to your "authorized_hosts" file, so after logging in to your account on the server, do this:
cd
mkdir .ssh
cat id_rsa.pub >> .ssh/authorized_hosts
rm id_rsa.pub
Now, from your workstation you need to configure your ssh client to use the private key your generated. Again, I don't know how to do this on Windows and it will probably vary depending on what ssh client you are using, so you'll need to get that information somewhere else.
Next, you need to create your local repository, add some files, and make a commit. Note that you can't clone the remote repository you made yet because there's nothing in there. Once you have some commits made locally, you need to set the remote server in your repository.
If you are using the git command line tools, you can run:
git remote add origin user#yourserver:myrepo.git
If you put the repository somewhere other than your home directory, use the full path:
git remote add origin user#yourserver:/path/to/repo.git
Note that you need the ".git" in there, since your directory name has ".git" as part of the name.
If you are using a GUI tool, then you instead just edit the configuration file for the repository. This will be in the top level of your repository under ".git/config". You'll want to add something like
[remote "origin"]
url = user#yourserver:/path/to/repo.git
fetch = +refs/heads/*:refs/remotes/origin/*
Now that your remote is configured, and you have some commits locally, you can push your master branch up to the server. If you're using the command line use:
git push origin master
Or, if you're working on a different branch:
git push origin mybranch
If you are using a GUI frontend for get then you'll need to look up the documentation for that tool on how to push.
I would like to recommend to use Gitolite that allows to setup git hosting on central srever with great features like access control and creating/managin repositories etc.
And for windows, the tool called git-scm can be used for setting up git client and also have GUI panel for git repositories. After installation of this git tool, you will get a git bash tool and a git GUI.
Gitolite makes settting up of a git hosting server and client very simple.
I hope this may help new people who are looking for an efficient solution for settting up git hosting server and clients in some easy to understand simple steps.
I followed the most voted answer and I didnt get it to work. I found this tutorial:
https://www.digitalocean.com/community/tutorials/how-to-set-up-ssh-keys--2
and I got it working. I think the problem was that, in the server, the public key is to be copied to a file "authorized_keys" and not "authorized_hosts".
I had been trying to generate the key pairs using putty but couldn't get it to work. In the end I realised that I could generate the keys via the msysgit command line using ssh-keygen -C "git#example.com" -t rsa. Copy the public key to the server using nano /home/git/.ssh/authorized_keys and hey presto!
I had downloaded the full source code hence the size! The non-source app, Git-1.7.4-preview20110204.exe was just 13MB via http://code.google.com/p/msysgit/downloads/list?can=3