Capistrano error : could not connect to ssh-agent - ruby

I'm using Bedrock with Capistrano deploys.
When I use command bundle exec cap staging deploy:check I get an authentication error :
...
D, [2015-05-09T15:39:53.878464 #15636] DEBUG -- net.ssh.authentication.session[1e34a58]: trying publickey
D, [2015-05-09T15:39:53.878464 #15636] DEBUG -- net.ssh.authentication.agent[1e30d2c]: connecting to ssh-agent
E, [2015-05-09T15:39:53.879447 #15636] ERROR -- net.ssh.authentication.agent[1e30d2c]: could not connect to ssh-agent
E, [2015-05-09T15:39:53.879447 #15636] ERROR -- net.ssh.authentication.session[1e34a58]: all authorization methods failed (tried publickey)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as deploy#SERVER_IP: Authentication failed for user deploy#SERVER_IP
Tasks: TOP => git:check => git:wrapper
Capistrano could not connect to ssh-agent on my server.
But I can log in on my server via SSH like this ssh deploy#SERVER_IP without password. I dit all the instructions in Capistrano Authentication & Authorisation Docs page, so I can use command like me#localhost $ ssh deploy#one-of-my-servers.com 'hostname; uptime'.
If I enter command ssh -A deploy#SERVER_IP 'env | grep SSH_AUTH_SOCK' I get result
SSH_AUTH_SOCK=/tmp/ssh-UweQkw7578/agent.7578
Here is my deploy.rb file :
set :application, 'APP'
set :repo_url, 'URL'
set :branch, :master
set :tmp_dir, '~/tmp'
set :log_level, :info
set :linked_files, fetch(:linked_files, []).push('.env')
set :linked_dirs, fetch(:linked_dirs, []).push('web/app/uploads')
Here is my staging.rb file :
set :stage, :staging
set :deploy_to, -> { "/var/www/vhosts/project/dev" }
server 'SERVER_IP', user: 'deploy', roles: %w{web app}
set :ssh_options, {
user: 'deploy',
keys: %w('/c/Users/alexander/.ssh/id_rsa'),
forward_agent: true,
auth_methods: %w(publickey),
verbose: :debug
}
fetch(:default_env).merge!(wp_env: :staging)
Apache's agent forwarding agent instruction is enabled in sshd_config file : AllowAgentForwarding yes
What should do with my config files to make my deploy work?
Windows 8.1
Ruby 2.2.0
Capistrano 3.2.1
Git Bash

OK so I had the same issue, and I spent way too long working out exactly what is happening here, and the upshot is -
for ruby on windows, you must run pagent, not ssh-agent, for Capistrano and agent forwarding to work - in fact pretty much any tool that uses the Ruby net-ssh library on Windows.
And I dont think that will change, at least not for a while.
Agent Forwarding
See An Illustrated Guide to SSH Agent Forwarding for more about agent forwarding, and how the key challenge ends back up on our workstation.
Terminology
workstation - the machine (Windowa server/desktop/laptop) our SSH
client software is running from, and, most importantly, our PKI
private key is stored on (with or without a passphrase)
deployment node - the target of our Capistrano deployment task, most
like defined in the 'server' key in our config/deploy.rb, or
config/deploy/.rb file
git repo - where we will pull the code from, first queried via "git
ls-remote" - we will access this git repo via SSH, and the deployment
node will use agent forwarding to pass the key challenge back to the
workstation
SSH client software - how we reach out to sshd on remote servers, and
which has access to our private key. Might be putty, an OpenSSH ssh
client or the net-ssh library in Ruby.
Setup
I have a Windows 7 workstation box, with Git-Bash, and its OpenSSH ssh client, plus the script from Joe Reagle that sets up some environmental variables that say which port and pid the ssh-agent is operating on.
I also have Putty and Pageant, but I focussed, initially, on just the OpenSSH/Git-Bash tools.
I have set up passwordless ssh from the workstation to the deployment node, I have the ssh-agent running, I have my key added through ssh-add, and I have my public key registered as a read-only access key to the git repo.
Basics
So we are trying to use SSH agent forwarding to have Capistrano pull from our Git repo onto our deployment node.
Now we can test this all ourselves by setting up our public SSH key on the deployment node and using, say, the OpenSSH ssh client, to confirm we have passwordless ssh working. Then we can setup ssh-agent by
starting ssh-agent and setting the SSH_AUTH_SOCK and SSH_AGENT_PID as required.
adding our private key to the ssh-agent via ssh-add
add our public key as an authorised key to the git repo
ssh to the deployment node, and from there do a "git ls-remote git#" (or a ssh -T git#)
If everything is setup correctly, this will all work, and so we will think "ok I can do a 'cap deploy:check'" - and it will fail.
What Went Wrong
We will get an error
"Error reading response length from authentication socket"
Who is telling us this ? It isnt immediately clear, but it
isn't the git repo
it isnt the git client on the deployment node
it isnt the sshd daemon on the deployment node, that wants to pass the key challenge back to the workstation.
Its the Ruby ssh client library on the workstation.
How do we know this
In the ssh_options hash in the deploy.rb file, we add the following :
verbose: :debug
When we do this we see this message
Pageant not running.
Why is Capistrano trying to use Pageant instead of ssh-agent
When running via Capistrano, the ssh client is different to the one you used when verifying things by hand.
When verifying by hand, it was an OpenSSH ssh client. Now it is the net-ssh library in Ruby.
And on Windows, net-ssh has these lines
if Net::SSH::Authentication::PLATFORM == :win32
require 'net/ssh/authentication/pageant'
end
or
case Net::SSH::Authentication::PLATFORM
when :java_win32
require 'net/ssh/authentication/agent/java_pageant'
else
require 'net/ssh/authentication/agent/socket'
So loading pageant is hard-coded into net-ssh. It doesnt even try to see if you are running under a unix-like shell (like git-bash or cygwin), and to then use the unix-domain ssh-agent SSH_AUTH_SOCK
At present net-ssh doesnt try to open a unix-domain named socket. In theory I think it could, through the UNIXSocket class in the stdlib. But I haven't experimented with that on a Windows machine yet.

Related

VS Code - Openfolders and edit files through SSH tunnel

I typically use VS Code's Remote-SSH extension to connect to an AWS instance where I do my work. There's a nice section of their documentation about how to do that here. I hate just using the terminal alone because I like the debugger and the ability to open up folders and files for viewing. Until recently I had been SSH-ing from my MacBook Pro (A) to an AWS instance running Ubuntu 18.04.4 LTS (B) without any issues. Now, however, I need to work on a new instance, also Ubuntu 18.04.4 LTS, (C) that I can only SSH into from B, and not from A.
This part of the documentation indicates that I can SSH from A to B and then forward a port to create an SSH tunnel. However, this only allows me to open a terminal into C. However, this doesn't allow me to open folders and files in the Explorer like I want to.
I've followed Qinsheng's instructions here to set up an SSH tunnel and use the Remote-SSH extension's "ProxyJump" functionality My config file looks like this:
Host B
HostName B.mydomain.com
IdentityFile /Users/MyID/.ssh/B_private_key
User me
Host C
HostName C
IdentityFile /Users/MyID/.ssh/C_private_key
ProxyJump B
User me
This works perfectly when I use the terminal on VS Code to run ssh C. Without having to input any password(s) I am taken to C and can perform my work. However, when I try to use the "Remote-SSH: Connect to Host" functionality to SSH into C, using ProxyJump through B and subsequently open a folder/view files there, I get the message "Could not establish connection to 'C'". The stack trace looks like this:
[13:59:48.537] Log Level: 2
[13:59:48.539] remote-ssh#0.51.0
[13:59:48.539] darwin x64
[13:59:48.541] SSH Resolver called for "ssh-remote+C", attempt 1
[13:59:48.541] SSH Resolver called for host: C
[13:59:48.541] Setting up SSH remote "C"
[13:59:48.546] Acquiring local install lock: /var/folders/w2/0w6jys8d34x940rj7f5r20vnh83ht7/T/vscode-remote-ssh-C-install.lock
[13:59:48.590] Looking for existing server data file at /Users/MyID/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-v1-91899dcef7b8110878ea59626991a18c8a6a1b3e-0.51.0/data.json
[13:59:48.593] Using commit id "91899dcef7b8110878ea59626991a18c8a6a1b3e" and quality "stable" for server
[13:59:48.595] Install and start server if needed
[13:59:48.599] Checking ssh with "ssh -V"
[13:59:48.629] > OpenSSH_8.1p1, LibreSSL 2.7.3
[13:59:48.632] askpass server listening on /var/folders/w2/0w6jys8d34x940rj7f5r20vnh83ht7/T/vscode-ssh-askpass-c4eabfddec4ae91c1e423a3167dd184d8553da26.sock
[13:59:48.633] Spawning local server with {"ipcHandlePath":"/var/folders/w2/0w6jys8d34x940rj7f5r20vnh83ht7/T/vscode-ssh-askpass-b00842fb08dbbff49678439b1335f105364fd9d5.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","52911","-o","ConnectTimeout=15","C"],"dataFilePath":"/Users/MyID/Library/Application Support/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-v1-91899dcef7b8110878ea59626991a18c8a6a1b3e-0.51.0/data.json"}
[13:59:48.633] Local server env: {"DISPLAY":"1","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/Users/MyID/.vscode/extensions/ms-vscode-remote.remote-ssh-0.51.0/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/Users/MyID/Desktop/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer)","VSCODE_SSH_ASKPASS_MAIN":"/Users/MyID/.vscode/extensions/ms-vscode-remote.remote-ssh-0.51.0/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/var/folders/w2/0w6jys8d34x940rj7f5r20vnh83ht7/T/vscode-ssh-askpass-c4eabfddec4ae91c1e423a3167dd184d8553da26.sock"}
[13:59:48.635] Spawned 29414
[13:59:48.734] > local-server> Spawned ssh: 29415
[13:59:48.738] stderr> OpenSSH_8.1p1, LibreSSL 2.7.3
[13:59:48.740] stderr> /bin/false: No such file or directory
[13:59:48.740] stderr> kex_exchange_identification: Connection closed by remote host
[13:59:48.741] > local-server> ssh child died, shutting down
[13:59:48.745] Local server exit: 0
[13:59:48.745] Received install output: OpenSSH_8.1p1, LibreSSL 2.7.3
/bin/false: No such file or directory
kex_exchange_identification: Connection closed by remote host
[13:59:48.746] Stopped parsing output early. Remaining text: OpenSSH_8.1p1, LibreSSL 2.7.3/bin/false: No such file or directorykex_exchange_identification: Connection closed by remote host
[13:59:48.746] Failed to parse remote port from server output
[13:59:48.747] Resolver error:
[13:59:48.750] ------
It seems clear to me that the main problem is something trying to access /bin/false as the interpreter. I assume that this is an issue with my MacBook Pro (A). I did notice that when running echo $SHELL on A that it was returning /bin/false, but I ran export SHELL=/bin/bash to change that and followed the suggestions in this post to set VS Code's integrated shell for MacOS to /bin/bash as well. I've checked echo $SHELL on B as well and it is similarly showing /bin/bash too.
My questions are these:
Is what I am trying to accomplish possible?
If it is, what am I doing wrong?/what can I check to help make sure that I can get to the VS Code functionalities that I want?
I figured it out. I had a bunch of /bin/false references in my /etc/passwd file, so I changed those out to /bin/bash and now it works like a charm. :)

Creating a executable for mac terminal commands

I'm using a series of commands for deploying my files to production,This is done manually currently.Is it possible to make these commands to an executable file,So that i don't have to copy paste these commands each and every time.
When the first line is executed ie, connecting to the root server it will ask for the password and have to paste the password: xxxxxxxxxxxxxx
I'm using mac system.
if you have any idea please let me know?
ssh root#server
pwd: xxxxxxxxxxxxxx
ssh-agent bash
ssh-add bi-master
cd /home/trans/bimaster
git status
git pull git#bitbucket.org:xxxxxxxxx/bimaster.git master
Yes it is possible, normally people use services like Jenkins (free) or bamboo (not free) to run deployment tasks, then you can save credentials on the bamboo/Jenkins server and write a deployment script or pipeline to do the deployment steps.
You should also consider setting up ssh keys on your local machine (public private keypair with ssh-keygen) and the remote (just the public key - ssh-copy-id can do this IIRC) so you don't have to rely on password logins on the remote login. generally, ssh keys are considered more secure and practical than using passwords.
Also, don't do things as the root user unless you have no other choice or you're just testing. Running things as root in production is asking for trouble.

Capifony forward_agent not working. Asks for GitHub password and fails

I am using capifony to deploy a symfony2 web app onto Ubuntu. I have the following in my deploy.rb:
set :repository, "git#github.com:Username/Repo.git"
set :branch, "develop"
set :scm_verbose, true
set :deploy_via, :remote_cache
default_run_options[:pty] = true
set :ssh_options, {:forward_agent => true}
ssh_options[:keys] = ["/Users/myuser/Sites/file.pem"]
ssh_options[:auth_methods] = ["publickey"]
Capifony is able to ssh in but then asks for a GitHub password during the deploy. I enter the password correctly but it then give me the following authentication fail error:
remote: Invalid username or password
fatal: Authentication failed for 'https://UserName#github.com/UserName/Repo.git/'
I don't know why it is asking for my Github password, and why it doesn't use agent forwarding to ssh to github?
I have added my public key from the server to GitHub but I am a bit confused about these keys, and my local keys? does it forward my local key or does it use the server one? What permissions do I need to set etc?
I am using capifony v2.8.6
I recently had to reinstall ruby and capifony die to upgrading to El Capitan which stopped capifony from working altogether. This is when my problems started.
I am deploying to Ubuntu 14.04
I have tried editing /etc/ssh/ssh_config on the server and adding
ForwardAgent yes
Then restarting ssh, but this has no effect.
Thanks
I do not use Capifony anymore and cannot comment yet so I try to answer :)
First, if you deploy from your local machine (your Mac), check :
your ssh-agent is started,
you can git clone one of your github repo
you can ssh to your Ubuntu server
Then you could just set this two line below.
default_run_options[:pty] = true
ssh_options[:forward_agent] = true
No need to set other ssh_options nor to tweak you ssh server config , it might be very confusing.
Second, you can test if ssh forwarding is working for the selected environment by creating a simple rake tasks inside your project :
namespace :ssh do
desc "Check if agent forwarding is working"
task :forwarding do
on roles(:all) do |h|
if test("env | grep SSH_AUTH_SOCK")
info "Agent forwarding is up to #{h}"
else
error "Agent forwarding is NOT up to #{h}"
end
end
end
end
And test with a cap [env] ssh:forwarding
Finally, if you reinstalled ruby/capifony recently, try to make the switch to Capistrano v3 and its Symfony gem instead, it really worth a try.

How can I get a Vagrant provision shell script to successfully clone a mercurial repository?

I am trying to configure a virtual machine using Vagrant as a local development environment.
I would like the provision script to clone a Mercurial repository from Bitbucket within the guest machine.
In order to accomplish this I have added hg clone ssh://hg#bitbucket.org/myuser/myrepo in my provision shell script
I also added the flag config.ssh.forward_agent = true in my Vagrantfile.
However I keep getting the following error:
==> default: remote: Host key verification failed.
==> default: abort: no suitable response from remote hg!
The SSH command responded with a non-zero exit status. Vagrant assumes that
this means the command failed. The output for this command should be
in the log above. Please read the output to determine what went wrong.
I've successfully set up SSH keys for my Bitbucket account on my host machine but for some reason they don't seem to be forwarded in the guest machine even though I've set the flag true.
My host machine is running on Windows 7 and the guest is running on Linux Trusty Tarh
Sincerely appreciate any tips on how I can accomplish this.
You need to add the host key of bitbucket to your known_hosts file. You could either do that manually, or try something like https://serverfault.com/questions/132970/can-i-automatically-add-a-new-host-to-known-hosts.
Hope that helps.

Capistrano, Firewalls and Tunnel

We're using Capistrano to automate pushing new versions of a PHP application to a production server. The production server (we'll call it production) is public, while our repository server (we'll call it repo) sits behind our corporate firewall, along with our own machines.
Capistrano, as configured by default, won't work, as production can't talk to repo.
I was wondering if there was someway I could setup capistrano to SSH to repo first, then SSH to production opening a tunnel on a port that I can then use to SSH from production back to repo to pull the changes from SCM.
I just can't figure out how to set this up or figure out a better solution. Ideas?
Edit:
I've tried this:
role :web, "deploy.com"
namespace :deploy do
task :remote_tunnel do
run 'Creating SSH tunnel...' do |channel, stream, data|
ssh = channel.connection
ssh.forward.remote(22, 'server.com', 10000, '127.0.0.1')
ssh.loop {!ssh.forward.active_remotes.include?([10000, '127.0.0.1'])}
end
end
end
before "deploy:update_code", "deploy:remote_tunnel"
But I keep getting this error:
failed: "sh -c 'Creating SSH tunnel...'" on deploy.com
Here's are 2 ways to accomplish it.
1st way
not sure if you've seen this thread?
https://groups.google.com/forum/?fromgroups=#!topic/capistrano/RVwMim-qnMg
It makes use of the net-ssh-gateway library, but creates copies of the local forwarding methods but they're geared for remote access.
class Net::SSH::Gateway
# Opens a SSH tunnel from a port on a remote host to a given host and port
# on the local side
# (equivalent to openssh -R parameter)
def open_remote(port, host, remote_port, remote_host = "127.0.0.1")
ensure_open!
#session_mutex.synchronize do
#session.forward.remote(port, host, remote_port, remote_host)
end
if block_given?
begin
yield [remote_port, remote_host]
ensure
close_remote(remote_port, remote_host)
end
else
return [remote_port, remote_host]
end
rescue Errno::EADDRINUSE
retry
end
# Cancels port-forwarding over an open port that was previously opened via
# open_remote.
def close_remote(port, host = "127.0.0.1")
ensure_open!
#session_mutex.synchronize do
#session.forward.cancel_remote(port, host)
end
end
end
2nd way
Outlined in an answer to this SO question:
Is it possible to do have Capistrano do a checkout over a reverse SSH tunnel?
This technique is very similar to the 1st way. First you need to create 2 paths to the repository:
# deploy.rb
set :local_repository, "ssh://git#serverbehindfirewall/path/to/project.git"
set :repository, "ssh://git#localhost:9000/path/to/project.git"
Then before you deploy you'll need to setup the remote forward:
% ssh -R 9000:serverbehindfirewall:22 deploybot#deployserver.com
# CTRL + C + A (Screen) or ⌘ + T (Terminal.app) to open new tab
Followed by your deploy:
% cap HOSTFILTER=deployserver.com deploy # HOSTFILTER reduces set to specified host. Only useful if you have multiple servers.
See this answer to that SO question for more details:
https://stackoverflow.com/a/3953351/33204
Using Capistrano 3.x, the following works for me:
namespace :deploy do
desc "Open SSH Tunnel to GitLab"
task :open_tunnel do
on roles(:app) do
info "Opening SSH Remote Tunnel..."
self.send(:with_ssh) do |ssh|
# ssh -R 9000:192.168.1.123:22
ssh.forward.remote(22, "192.168.1.123", 9000)
end
end
end
before "deploy:check", "deploy:open_tunnel"
end
Please note that ssh.forward.remote expects parameters in a different order than ssh -R, the above is equivalent to ssh -R 9000:192.168.1.123:22
This task calls a private method, if anyone knows an official way to get the access Capistrano's ssh connection, please comment or edit.
Edit: Also see the section Tunneling and other related SSH themes of SSHKit's README

Resources