puphpet how to configura to automatically clone a repository? - puphpet

I'm starting to use puphpet, I want to use it to automatically clone a repository on provision but I'm not able to find a way, is this possible?

One posibility that I found is create a .sh script on puphpet/files/exec-once-unprivileged/
The content of the script could be something like:
cd /vagrant
# Add github.com into known hosts to avoid interactive question
ssh -T git#github.com -o StrictHostKeyChecking=no
git clone ssh://git#github.com/repository
If you want to clone a private repository you can use ssh-agent to use the ssh keys of the host machine in the VM. To do this do on the host machine:
ssh-add ~/.ssh/id_rsa

Related

How to clone repository using SSH in EC2 userdata? [duplicate]

I am trying to connect to a remote Git repository that resides on my web server and clone it to my machine.
I am using the following format for my command:
git clone ssh://username#domain.example/repository.git
This has worked fine for most of my team members. Usually after running this command Git will prompt for the user's password, and then run the cloning. However, when running on one of my machines I get the following error:
Host key verification failed.
fatal: Could not read from remote
repository.
We are not using SSH keys to connect to this repository, so I'm not sure why Git is checking for one on this particular machine.
As I answered previously in Cloning git repo causes error - Host key verification failed. fatal: The remote end hung up unexpectedly, add GitHub to the list of known hosts:
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
You are connecting via the SSH protocol, as indicated by the ssh:// prefix on your clone URL. Using SSH, every host has a key. Clients remember the host key associated with a particular address and refuse to connect if a host key appears to change. This prevents man in the middle attacks.
The host key for domain.example has changed. If this does not seem fishy to you, remove the old key from your local cache by editing ${HOME}/.ssh/known_hosts to remove the line for domain.example or letting an SSH utility do it for you with
ssh-keygen -R domain.example
From here, record the updated key either by doing it yourself with
ssh-keyscan -t rsa domain.example >> ~/.ssh/known_hosts
or, equivalently, let ssh do it for you next time you connect with git fetch, git pull, or git push (or even a plain ol’ ssh domain.example) by answering yes when prompted
The authenticity of host 'domain.example (a.b.c.d)' can't be established.
RSA key fingerprint is XX:XX:...:XX.
Are you sure you want to continue connecting (yes/no)?
The reason for this prompt is domain.example is no longer in your known_hosts after deleting it and presumably not in the system’s /etc/ssh/ssh_known_hosts, so ssh has no way to know whether the host on the other end of the connection is really domain.example. (If the wrong key is in /etc, someone with administrative privileges will have to update the system-wide file.)
I strongly encourage you to consider having users authenticate with keys as well. That way, ssh-agent can store key material for convenience (rather than everyone having to enter her password for each connection to the server), and passwords do not go over the network.
I had the similar issue, but, using SSH keys. From Tupy's answer, above, I figured out that the issue is with known_hosts file not being present or github.com not being present in the list of known hosts. Here are the steps I followed to resolve it -
mkdir -p ~/.ssh
ssh-keyscan -t rsa github.com >> ~/.ssh/known_hosts
ssh-keygen -t rsa -C "user.email"
open the public key with this command $ cat ~/.ssh/id_rsa.pub and copy it.
Add the id_rsa.pub key to SSH keys list on your GitHub profile.
This is happening because github is not currently in your known hosts.
You should be prompted to add github to your known hosts. If this hasn't happened, you can run ssh -T git#github.com to receive the prompt again.
For me, I just had to type "yes" at the prompt which asks "Are you sure you want to continue connecting (yes/no)?" rather than just pressing Enter.
If you are in office intranet (otherwise dangerous) which is always protected by firewalls simply have the following lines in your ~/.ssh/config.
Host *
StrictHostKeyChecking no
UserKnownHostsFile=/dev/null
When asked:
Are you sure you want to continue connecting (yes/no)?
Type yes as the response
That is how I solved my issue. But if you try to just hit the enter button, it won't work!
I got the same problem on a newly installed system, but this was a udev problem. There was no /dev/tty node, so I had to do:
mknod -m 666 /dev/tty c 5 0
What worked for me was to first add my SSH key of the new computer, I followed these instructions from GitLab - add SSH key. Note that since I'm on Win10, I had to do all these commands in Git Bash on Windows (it didn't work in regular DOS cmd Shell).
Then again in Git Bash, I had to do a git clone of the repo that I had problems with, and in my case I had to clone it to a different name since I already had it locally and didn't want to lose my commits. For example
git clone ssh://git#gitServerUrl/myRepo.git myRepo2
Then I got the prompt to add it to known hosts list, the question might be this one:
Are you sure you want to continue connecting (yes/no)?
I typed "yes" and it finally worked, you should typically get a message similar to this:
Warning: Permanently added '[your repo link]' (ECDSA) to the list of known hosts.
Note: if you are on Windows, make sure that you use Git Bash for all the commands, this did not work in regular cmd shell or powershell, I really had to do this in Git Bash.
Lastly I deleted the second clone repo (myRepo2 in the example) and went back to my first repo and I could finally do all the Git stuff like normal in my favorite editor VSCode.
When the remote server wants to connect to the private repo, it would authenticate via ssh.
Create the private-public key pair with ssh-keygen or if you already have the public-private key. copy&paste the public key in the Settings of the private repo.
YourPrivateRepo -> Settings -> Deploy Keys -> Add deploy key -> Paste the public key.
Now the remote server would be able to connect to the private repo.
NOTE: The deploy keys has access only for reading the repo. Need to explicitly allow write access.
If you are using git for Windows.
Open the git GUI.
Open the local git repository in git GUI.
Add the remote or push if the remote already exists.
Answer "yes" to the question about whether you want to continue.
The GUI client adds the key for you to ~/.ssh/known_hosts. This is easier to remember if you don't do it often and also avoids the need to use the git command line (the standard Windows command lines don't have the ssh-keyscan executable.
The solutions mentioned here are great, the only missing point is, what if your public and private key file names are different than the default ones?
Create a file called "config" under ~/.ssh and add the following contents
Host github.com
IdentityFile ~/.ssh/github_id_rsa
Replace github_id_rsa with your private key file.
I was facing the same error inside DockerFile during build time while the image was public. I did little modification in Dockerfile.
RUN git clone https://github.com/kacole2/express-node-mongo-skeleton.git /www/nodejs
This would be because using the git#github.com:... syntax ends up > using SSH to clone, and inside the container, your private key is not > available. You'll want to use RUN git clone > https://github.com/edenhill/librdkafka.git instead.
Check permissions on the known_hosts file as well - both the user's (~/.ssh/known_hosts) and the global one (/etc/ssh/ssh_known_hosts).
In my case the old host was in /etc/ssh/ssh_known_hosts. When I removed it as root with sudo ssh-keygen -f /etc/ssh/ssh_known_hosts -R THE_HOST it changed permissions on that file to 0600, so SSHing to THE_HOST as root worked, but for any other user it failed with "Host key verification failed". The fix was:
sudo chmod 644 /etc/ssh/ssh_known_hosts
One small addition to Tupy's answer, you may need to add the port number for your repository host:
ssh-keyscan -p 8888 -t rsa domain.example >> ~/.ssh/known_hosts
If you have another machine that does have remote access you can find the port number by viewing ~/.ssh/known_hosts:
[user]$ less ~/.ssh/known_hosts
[domain.example]:8888,[000.00.000.000]:8888 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCi...
Its means your remote host key was changed (May be host password change),
Your terminal suggested to execute this command as root user
$ ssh-keygen -f "/root/.ssh/known_hosts" -R [www.website.net]
You have to remove that host name from hosts list on your pc/server. Copy that suggested command and execute as a root user.
$ sudo su // Login as a root user
$ ssh-keygen -f "/root/.ssh/known_hosts" -R [www.website.net] // Terminal suggested command execute here
Host [www.website.net]:4231 found: line 16 type ECDSA
/root/.ssh/known_hosts updated.
Original contents retained as /root/.ssh/known_hosts.old
$ exit // Exist from root user
Try Again, Hope this works.
You kan use https instead of ssh for git clone or git pull or git push
ex:
git clone https://github.com/user/repo.git
Reason seems to be that the public key of the remote host is not stored or different from the stored one. (Be aware of security issues, see Greg Bacon's answer for details.)
I was used to git clone prompting me in this case:
The authenticity of host 'host.net (10.0.0.42)' can't be established.
ECDSA key fingerprint is 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00.
Are you sure you want to continue connecting (yes/no)?
Not sure, why this error is thrown instead. Could be the configuration of your shell or the git SSH command.
Anyhow, you can get the same prompt by running ssh user#host.net.
A other alternative worked for me, instead of cloning the SSH link
git#gitlab.company.net:upendra/mycode.git
there is a option to select http link
http://gitlab.company.net:8888/upendra/mycode.git
So I used http link to clone for Visual studio and it worked for me
If you are not using a Windows Session to update the code, and you use PortableGit, you need to set the HOMEPATH environment variable before running the git command.
This example fits better for other use case, but I think it is a good of proof-of-concept for this post.
$env:HOMEPATH="\Users\Administrator";C:\path\to\PortableGit\bin\git.exe -C C:\path\to\repository.git pull'
Pushing to Git returning Error Code 403 fatal: HTTP request failed
Check if there is Billing issue.
Google Cloud stops uploading files to https://source.cloud.google.com/
I got this problem went away after Payment issue was fixed.
But did not change the Keys.
Thanks
Dashboard > Manage Jenkins > Configure Global Security > Git Host Key Verification Configuration.
Then in Host Key Verification Strategy select Accept first connection.
You can use your "git url" in 'https" URL format in the Jenkinsfile or wherever you want.
git url: 'https://github.com/jglick/simple-maven-project-with-tests.git'
Alternatively, if you're using MSYS2 terminals (on Windows*) and a passphrase, it might be that the terminal does not prompt the 'Enter passphrase' properly, thus denying access to SSH.
If you're on Windows, you can instead use the Git Bash or Powershell to get the prompt and properly connect. (I'm currently looking for a solution for MSYS.)
*Not sure if relevant.
Problem:
Host key verification failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Solution: I've checked all the settings and also checked the key settings in GitHub. Finally, I changed the Git URL from "git#github.com:palvsv/travelo-moon.git" to "https://github.com/palvsv/travelo-moon.git" in .config file "yourprojectdirectory/.git/config" and it works.
for me, I just rename the "known_hosts" file to "known_hosts.del" for backup. and then rerun git clone xxx and type "yes". I will create new "known_hosts"
Just type 'yes' and press enter this should work
When the terminal shows:
Are you sure you want to continue connecting (yes/no)?
DO NOT I repeat DO NOT directly pressed Enter.
You MUST TYPE yes first in the terminal, then press Enter.
I had the similar issue, unfortunately I used the GitExtensions HMI and forgot that I wrote a passphrase.
With HMI.... forget it ! Do not enter passphrase when you generate your key !
I got this message when I tried to git clone a repo that was not mine. The fix was to fork and then clone.

How to use the GitBash to clone data with ssh (windows 10 environment)

What can I solve this problem?
when I wanted to connect my personal GitLab account, I got an error message like the following picture
### shell script
git clone git#gitlab.com:<username>/test1.git
GitLab server-side
Clone ssh address
PS: I have checked and found some reference from StackOverflow but unfortunately it isn't okay for me
1.git clone through ssh
2.SSH and Git Clone
3.git clone with ssh issue
I have found a great reference but sorry for Mandarin website, however, I can use my way to share how to deal with this issue.
Step 1:
ls -al ~/.ssh
Step 2:
ssh-keygen
(using enter key for default value)
Step 3: To setup config file
vim /c/Users/Willie/.ssh/config
Host gitlab.com
HostName gitlab.com
User git
IdentityFile ~/.ssh/id_rsa
Step 4:
git clone git#gitlab.com:<username>/test2.git
Step 5:
When you finished Step 4
1.the test2.git file will be download done
2.you will get the new file(known_hosts) in the ~/.ssh
PS: I create the id_rsa and id_rsa.ub by myself and I deliver it to the Gitlab server. using both keys to any client-sides(windows and Linux).
Check first if you do have a ~/.ssh/id_rsa private key/~/.ssh/id_rsa.pub public key.
If so, check your private key: if it has 70 chars per line, try and regenerate with the old PEM format:
ssh-keygen -m PEM -t rsa -P "" -f ~/.ssh/id_rsa
(That will override your current key, but if said current key is not working anyway, that should be OK).
Update the public key on GitLab side, and try again, with a simple:
ssh -T git#gitlab.com
Using a config file means not using the user and using a shorter name:
Host gitlab
HostName gitlab.com
User git
IdentityFile ~/.ssh/id_rsa
means: ssh -T gitlab or git clone gitlab:<username>/test2.git will work.

git clone with SSH only working in Git Bash not on Windows CMD

So, I've followed this tutorial on how to Setup SSH for github with Windows CMD and all was working fine until I went to clone a repo with
git clone git#github.com:{myusername}/{myrepo}.git
Where I get
git#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Even when I run ssh -T git#github.com I get the expected message telling me I'm authenticated.
After scratching my brain for a while, I decided to try it on git bash.
First thing I noticed was that running
ssh-add -l
in git bash, I was getting The agent has no identities. but when I run the same command on Windows CMD I get all my SSH keys?
So, after adding my ssh key in git bash I was able to clone my repository.
So, why is it only on git bash I can do this and not on the cmd or powershell? Is it something to do with what seems like they are using different ssh agents? How can I sync them together if that is the case?
Furthermore, when I run the following command
ssh -Tv git#github.com
with the cmd I get
debug1: identity file C:\\Users\\{myuserdirectory}/.ssh/id_rsa type 0
debug1: key_load_public: No such file or directory
but with git bash I get
debug1: identity file /c/Users/{myuserdirectory}/.ssh/id_rsa type 0
Another difference is that in windows cmd I don't get any instances of
debug1: Will attempt key: ....
When I exit git bash and open up another git bash terminal, running ssh-add -l again, it returns The agent has no identities. even after I added it before, it's like it only persists for each session, which also isn't desirable.
Any help with this would be greatly appreciated!
Probably you were right and they were using different ssh-agents. I had exactly the same problem and this answer helped me a lot:
https://stackoverflow.com/a/40720527/6486458
By default git refers to its own ssh in C:\Program Files\Git\usr\bin. I added GIT_SSH environment variable and set it to C:\Windows\System32\OpenSSH\ssh.exe. This prevents inconsistency between the versions of ssh. After that git started to work as expected from both Git Bash and Windows cmd.
From git documentation:
GIT_SSH, if specified, is a program that is invoked instead of ssh
when Git tries to connect to an SSH host. It is invoked like $GIT_SSH [username#]host [-p <port>] <command>.
See also this answer: https://stackoverflow.com/a/8713121/6486458
Looks like your ssh-agent is not running or not recognize your ssh key
try this:
# add the default ~/.ssh keys to the ssh-agent
ssh-add
# restart the ssh-agent
eval $(ssh-agent)
# On windows:
start-ssh-agent
ssh-add
ssh-add adds RSA or DSA identities to the authentication agent, ssh-agent.
When run without arguments, it adds the files ~/.ssh/id_rsa, ~/.ssh/id_dsa and ~/.ssh/identity.
Alternative file names can be given on the command line
There is a weird bug on Windows if you install Git bash. Open Command prompt, and do
ls ~/.ssh
if you find this folder already created, then copy the public and private key from your user folder to this path:
cp C:\Users\username\.ssh\id_* ~/.ssh/
For some reason, windows command prompt creates this path the first time you do a git clone, and after that it just requests for git#gitlab / git#github password.

Use Git (Windows) with a private key

I have a Git repository on a Linux server and I want to clone it on Windows.
I have Git for Windows installed and I prefer using it with the command line.
I use SSH with a public key to connect to my Linux server and I have my keys generated and ready to use, but I don't know how to tell git to use the key.
When I just SSH into the box, I use this command:
ssh -i c:\path\to\private\key\id_rsa user#192.168.2.104
So, I guess I'm looking for Git's equivalent of the -i option.
Obviously, this doesn't work:
git -i c:\path\to\private\key\id_rsa clone user#192.168.2.104:/home/user/dev/myproject.git myproject
This is all on my local network.
On Windows, provided %HOME% is set to %USERPROFILE%, use a %HOME%\.ssh\config file, which can indicate, for a given entry name, the user and the path of the private key:
Host yourServer
HostName 192.168.2.104
User user
IdentityFile C:/path/to/yourPrivateKey
Test it with ssh -Tv yourServer
Then a clone would be:
git clone yourServer:/home/user/dev/myproject.git myproject
(no need to indicate user#192.168.2.104 or the path to the private key anymore: everything is in the %HOME%/.ssh/config)
That way, you can manage multiple ssh sets of public/private keys.

Install gitolite v3 on the Centos 6.4. The second user is asking a password

Please help me to understand what i am doing wrong. I have server on CentOS 6.4 and I need to install gitolite v3. What am i doing? by the first i am install one using next command:
yum install git* perl-Time-HiRes
useradd -m --system --shell /bin/bash --user-group gitolite
su - gitolite
mkdir bin
echo "PATH=$HOME/bin:$PATH" > .bash_profile
source .bash_profile
ssh-keygen -t rsa
cp .ssh/id_rsa.pub ./gitolite.pub
git clone git://github.com/sitaramc/gitolite
gitolite/install -ln
gitolite setup -pk gitolite.pub
Install was successful. Than I clone gitolite-admin repositories:
git clone gitadmin:gitolite-admin
cd gitolite-admin/keydir
And generate public key for the new user(lodar.pub)
ssh-keygen -t rsa -f lodar
mv ./lodar ../../.ssh/lodar
Than i edit conf/gitolite.conf:
repo gitolite-admin
RW+ = gitolite
repo testing
RW+ = #all
repo empty
RW+ = lodar
Commit all changes:
git add keydir/lodar.pub
git commit -m 'add user lodar and new repo empty'
git push
And that is all. Push was successful too. After all manipulation i created .ssh/config
Host gitadmin
User gitolite
Hostname 192.168.0.1
Port 22
IdentityFile ~/.ssh/gitolite
Host gitlodar
User gitolite
Hostname 192.168.0.1
Port 22
IdentityFile ~/.ssh/lodar
But if i try to connect using
ssh gitlodar info
lodar#192.168.0.1's password:
i must to enter the password.
ssh gitadmin info
hello, gitolite, this is gitolite#lodar-14452 running gitolite3 v3.5.1-4-g2f48a3e on git 1.7.1
R W gitolite-amdin
R W testing
What am i doing wrong?
You did:
mv ./lodar ../../.ssh/lodar
I supposed you meant
cp ./lodar.pub ../../.ssh/
mv ./lodar ../../.ssh/
Because if you don't copy the public key in ~/.ssh, you wouldn't be able to use it in an ssh session.
But the other mistake is in your ~/.ssh/config file:
Host gitlodar
User gitolite # not loadar!
Hostname 192.168.0.1
Port 22
IdentityFile ~/.ssh/lodar
See "How do programs like gitolite work?":
The idea is to always use the same user to connect to the gitolite server (in your case, connect as 'gitolite'), and open an *non-interactive shell.
The force-command ssh line will call the gitolite script with a parameter representing the name of the user (here 'loadar'), because that named is registered with the public key in the ~gitolite/.ssh/authorized_keys.
$pwd
/home/lodar/gitolite-admin/keydir
$ssh-keygen -t rsa -f lodar
Last command generated two keys: public (lodar.pub) and private (lodar). I moved private key in the .ssh on my client computer. The public one stayed in keydir.
$mv ./lodar ../../.ssh/lodar
That is why after 'git push' user lodar added to giotlite repo. I checked it on the server by the command
$gitolite sshkeys-lint
sshkeys-lint: === checking authkeys file:
sshkeys-lint: === checking pubkeys:
sshkeys-lint: === gitolite.pub maps to user gitolite
sshkeys-lint: === lodar.pub maps to user lodar
YOU ARE RIGHT. MY MISTAKE was IN MY .ssh/config. Thanks for the HELP. it is working great now.

Resources