I use em-ssh gem (which is a net-ssh adapter for EventMachine) to execute some shell commands on multi remote hosts. Commands are executed successfully on the first host, but failed on other hosts, it complains “EventMachine::Ssh::ConnectionTerminated”
and “EventMachine::Ssh::NegotiationTimeout”.
It seems that these hosts are competing for something maybe the port？ Can someone explains this?
def em_ssh_connection(ssh_host,ssh_user,ssh_password,&cb)
EM.run do
EM::Ssh.start("#{ssh_host}","#{ssh_user}",:password => "#{ssh_password}") do|connection|
connection.errback do |err|
#logger.info "#{err} (#{err.class})"
EM.stop
end
connection.callback do |ssh|
#logger.info "succcessfully connected with #{ssh_host},#{ssh_user},#{ssh_password}"
yield ssh
ssh.close
EM.stop
end
end
end
end
def search_servers(ssh_host,ssh_user,ssh_password)
em_ssh_connection(ssh_host,ssh_user,ssh_password) {|ssh|
command = 'sed -n \'/.*server s1 [0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/=\' '+"#{#haproxy_conf}"
myarray = em_ssh_exec(ssh,command)
if myarray[2] == 0
if myarray[0] != ""
#logger.info "all tcp servers(lines): #{myarray[0]}"
lines = myarray[0].split("\n")
if lines.length >0
start_line = lines[0].to_i
stop_line = lines[lines.length-1].to_i
#logger.info "start_line:#{start_line},stop_line:#{stop_line}"
delete_tcp_upstream(ssh,start_line,stop_line)
else
end
else
#logger.info "the stream field in proxy file is empty"
end
else
#logger.info "search all servers command failed:#{myarray[1]}"
end
}
end
def clear_haproxy(&cb)
search_servers(#ssh_host1,#ssh_user1,#ssh_password1)
search_servers(#ssh_host2,#ssh_user2,#ssh_password2)
cb.call
end
Related
I'm new to ruby and I'm trying to make a client to connect to a TCPServer, and it seems that in order to do so I have to call the method close_write every time I finish sending data one way, to let the client/server know that the other end is finished sending data. Whenever I do that then Im not able to write info to the server or client again because the socket is not opened for writing anymore.
This is my code:
client.rb
require "socket"
socket = TCPSocket.open("localhost", 6666)
loop do
input = gets.chomp
socket.puts input # Send data to server
socket.close_write
while(line = socket.gets)
puts line
end # Print sever response
break if input=="EXIT"
end
socket.close
server.rb
require "socket"
server = TCPServer.new 6666
data = Hash.new { |hash, key| hash[key] = {} }
WAITING_SET_VALUE = "1"
WAITING_NEW_COMMAND = "0"
loop do
Thread.start(server.accept) do |session|
thread_status ||= WAITING_NEW_COMMAND
....
puts "Entering If..."
if(thread_status == WAITING_NEW_COMMAND) #Check thread status
puts "thread_status == WAITING_NEW_COMMAND"
puts "CHECKING COMMAND.."
case line.strip
when /^set \w* \w* \d{1,7} \d{1,7}$/
puts "COMMAND SET"
thread_status = WAITING_SET_VALUE
lineArr = line.strip.split(" ")
varName = lineArr[1]
flag = lineArr[2]
ttl = lineArr[3]
size = lineArr[4]
puts "END SET EXECUTION"
session.write "Executed"
session.close_write
...
Is there a way to open the socket for writing again, or a better way to do this back and forth connection between server and client without losing context? Thanks!
When designing a client-server protocol, you have to decide:
How a client knows when a response has more lines/data.
How a client knows when a response is complete.
How a client knows when a response is invalid/valid.
How a client knows when there was some type of server error.
A simple approach is for the server to return a response with the number of lines (as in the code below). However, instead, you could use END or something so that the client knows when there is no more data to read. I would strongly suggest looking into tutorials about Protocols.
Save the below into a file called client_server.rb. First, run the server with ruby ./client_server.rb s and then in a separate terminal run the client with ruby ./client_server.rb c. Next, type in list over and over to see the different responses. I added list so that you don't have to type in set w w 1 1 over and over for testing purposes. Check it out and let me know if you have any questions.
# frozen_string_literal: true
require 'socket'
# Use:
# First, run the server: ruby ./client_server.rb s
# Then, run the client: ruby ./client_server.rb c
# Use "netcat localhost 6666" on the command line to test
# without implementing a client.
# Returns false to close this client socket.
# Returns true to continue reading from this client socket.
def handle_client(client_id, client_socket, command)
# TODO: Define some type of client-server Protocol here.
case command
when /^set \w* \w* \d{1,7} \d{1,7}$/
puts "Running command for client #{client_id}: #{command}"
# This is just for testing purposes.
case rand(3)
when 0
client_socket.puts 'lines 0'
when 1
client_socket.puts 'lines 3'
client_socket.puts 'This is line 1.'
client_socket.puts 'This is line 2.'
client_socket.puts 'This is line 3.'
when 2
client_socket.puts 'The set command returned an error.'
end
when 'list'
puts "Responding to client #{client_id} with list of messages."
# This is just for testing purposes.
case rand(3)
when 0
client_socket.puts 'messages 0'
when 1
client_socket.puts 'messages 2'
client_socket.puts 'This is message 1.'
client_socket.puts 'This is message 2.'
when 2
client_socket.puts 'Unable to retrieve messages due to error.'
end
when 'bye'
puts "Killing client #{client_id}."
return false # Disconnect and kill the thread.
else
client_socket.puts "[ERROR] Invalid command: #{command}"
end
client_socket.flush # Flush all output just in case.
true # Continue connection.
end
case ARGV[0].to_s.downcase
when 's' # server
TCPServer.open(6666) do |server_socket|
global_client_id = 1
loop do
Thread.start(global_client_id, server_socket.accept) do |client_id, client_socket|
puts "Accepting new client #{client_id}."
loop do
command = client_socket.gets
if command.nil?
puts "Client #{client_id} disconnected manually."
break
end
command = command.strip
keep_alive = handle_client(client_id, client_socket, command)
break unless keep_alive
end
client_socket.close
end
global_client_id += 1
end
end
when 'c' # client
TCPSocket.open('localhost', 6666) do |socket|
puts '[Commands]'
puts 'set <word> <word> <num> <num> Run set command.'
puts 'list Get list of messages.'
puts 'exit, bye Exit the client.'
puts
loop do
print '> '
input = $stdin.gets.strip
# TODO: Define some type of client-server Protocol here.
case input
when /EXIT|BYE/i
socket.puts 'bye'
socket.flush
break
when /\Aset .+\z/
socket.puts input
socket.flush
response = socket.gets
if response.nil?
puts 'Server is not running anymore! Disconnecting.'
break
end
response = response.strip
match_data = response.match(/\Alines (?<lines>\d+)\z/)
if match_data
line_count = match_data[:lines].to_i
puts "Received #{line_count} lines from server."
1.upto(line_count) do |i|
line = socket.gets
puts ">> Resp[#{i}] #{line}"
end
else
# Can check for "response == 'ERROR'" or something.
puts "Server error or invalid response from server: #{response}"
end
when 'list'
socket.puts input
socket.flush
response = socket.gets
if response.nil?
puts 'Server is not running anymore! Disconnecting.'
break
end
response = response.strip
match_data = response.match(/\Amessages (?<messages>\d+)\z/)
if match_data
message_count = match_data[:messages].to_i
puts "Received #{message_count} messages from server."
1.upto(message_count) do |i|
line = socket.gets
puts ">> Resp[#{i}] #{line}"
end
else
# Can check for "response == 'ERROR'" or something.
puts "Server error or invalid response from server: #{response}"
end
else
puts "Invalid command: #{input}"
end
end
end
else
puts "Pass in 'c' for client or 's' for server."
end
Capistrano 3 does not use command cap env shell anymore.
Now we should use cap env console
But it is not interactive and we can not use for example arrow keys for history or autocompletion on tab button
And what should I do?
I suggest to write your own little rake task to do it. Use readline gem
First of all thanks to follow materials:
https://thoughtbot.com/blog/tab-completion-in-gnu-readline-ruby-edition
How to write a Ruby command line app that supports tab completion?
desc "Remote console"
task :console do
require 'readline'
# https://thoughtbot.com/blog/tab-completion-in-gnu-readline-ruby-edition
host_args = (ENV['HOSTS'] || '').split(',').map { |r| r.to_sym }
role_args = (ENV['ROLES'] || '').split(',').map { |r| r.to_sym }
LIST = `ls /usr/bin`.split("\n").sort + `ls /bin`.split("\n").sort
comp = proc { |s| LIST.grep(/^#{Regexp.escape(s)}/) }
Readline.completion_append_character = " "
Readline.completion_proc = comp
while line = Readline.readline('cap> ', true)
begin
next if line.strip.empty?
exec_cmd(line, host_args, role_args)
rescue StandardError => e
puts e
puts e.backtrace
end
end
end
def exec_cmd(line, host_args, role_args)
line = "RAILS_ENV=#{fetch(:stage)} #{line}" if fetch(:stage)
cmd = "bash -lc '#{line}'"
puts "Final command: #{cmd}"
if host_args.any?
on hosts host_args do
execute cmd
end
elsif role_args.any?
on roles role_args do
execute cmd
end
else
on roles :all do
execute cmd
end
end
end
And do what you want with it, cheers! =))
I created a script which checks healthcheck and ports status from a .json file populated with microservices.
So for every microservice from the .json file the script will output the HTTP status and healthcheck body and other small details, and I want to add multithreading here in order to return all the output at once.Please see the script below:
#!/usr/bin/env ruby
... get the environment argument part...
file = File.read('./services.json')
data_hash = JSON.parse(file)
threads = []
service = data_hash.keys
service.each do |microservice|
threads << Thread.new do
begin
puts "Microservice: #{microservice}"
port = data_hash["#{microservice}"]['port']
puts "Port: #{port}"
nodes = "knife search 'chef_environment:#{env} AND recipe:#{microservice}' -i"
node = %x[ #{nodes} ].split
node.each do |n|
puts "Node: #{n}"
uri = URI("http://#{n}:#{port}/healthcheck?count=10")
res = Net::HTTP.get_response(uri)
status = Net::HTTP.get(uri)
puts res.code
puts status
puts res.message
end
rescue Net::ReadTimeout
puts "ReadTimeout Error"
next
end
end
end
threads.each do |thread|
thread.join
end
Anyway in this way the script return first the puts "Microservice: #{microservice}" and puts "Port: #{port}" and after this it will return the nodes and only after the STATUS.
How can I return all the data for each loop together?
Instead of puts write output to a variable (hash).
If you wand to wait for all threads to finish their job before showing the output, use ThreadsWait class.
require 'thwait'
file = File.read('./services.json')
data_hash = JSON.parse(file)
h = {}
threads = []
service = data_hash.keys
service.each do |microservice|
threads << Thread.new do
thread_id = Thread.current.object_id.to_s(36)
begin
h[thread_id] = "Microservice: #{microservice}"
port = data_hash["#{microservice}"]['port']
h[thread_id] << "Port: #{port}"
nodes = "knife search 'chef_environment:#{env} AND recipe:#{microservice}' -i"
node = %x[ #{nodes} ].split
node.each do |n|
h[thread_id]<< "Node: #{n}"
uri = URI("http://#{n}:#{port}/healthcheck?count=10")
res = Net::HTTP.get_response(uri)
status = Net::HTTP.get(uri)
h[thread_id] << res.code
h[thread_id] << status
h[thread_id] << res.message
end
rescue Net::ReadTimeout
h[thread_id] << "ReadTimeout Error"
next
end
end
end
threads.each do |thread|
thread.join
end
# wait untill all threads finish their job
ThreadsWait.all_waits(*threads)
p h
[edit]
ThreadsWait.all_waits(*threads) is redundant in above code and can be omitted, since line treads.each do |thread| thread.join end does exactely the same thing.
Instead of outputting the data as you get it using puts, you can collect it all in a string and then puts it once at the end. Strings can take the << operator (implemented as a method in Ruby), so you can just initialize the string, add to it, and then output it at the end, like this:
report = ''
report << 'first thing'
report << 'second thing'
puts report
You could even save them all up together and print them all after all were finished if you want.
I don't understand why this won't do what the title states.
#!/usr/bin/env ruby
require 'socket'
require 'timeout'
class Scanner
def initialize(host, port)
#host = host
#port = port
end
def popen
begin
array = []
sock = Socket.new(:INET, :STREAM)
sockaddr = Socket.sockaddr_in(#port, #host)
Timeout::timeout(5) do
array.push("Port #{#port}: Open") if sock.connect(sockaddr)
end
puts array
rescue Timeout::Error
puts "Port #{#port}: Filtered"
rescue Errno::ECONNREFUSED
end
end
end # end Scanner
def main
begin
p = 1
case ARGV[0]
when '-p'
eport = ARGV[1]
host = ARGV[2]
else
eport = 65535
host = ARGV[0]
end
t1 = Time.now
puts "\n"
puts "-" * 70
puts "Scanning #{host}..."
puts "-" * 70
while p <= eport.to_i do
scan = Scanner.new(host, p)
scan.popen
p += 1
end
t2 = Time.now
time = t2 - t1
puts "\nScan completed: #{host} scanned in #{time} seconds."
rescue Errno::EHOSTUNREACH
puts "This host appears to be unreachable"
rescue Interrupt
puts "onnection terminated."
end
end
main
What I'm trying to achieve is an output similar to nmap, in the way that it scans everything, and then shows all open or closed ports at the end. Instead what happens is that it prints them out as it discovers them. I figured pushing the output into an array then printing the array would achieve such an output, yet it still prints out the ports one at a time. Why is this happening?
Also, I apologize for the formatting, the code tags are a little weird.
Your loop calls popen once per iteration. Your popen method sets array = [] each time it is called, then populates it with one item, then you print it with puts. On the next loop iteration, you reset array to [] and do it all again.
You only asked "why," but – you could solve this by setting array just once in the body of main and then passing it to popen (or any number of ways).
I've got a script thats supposed to mimic ffmpeg on my local machine, by sending the command of to a remote machine, running it there and then returning the results.
(see previous stackoverflow question.)
#!/usr/bin/env ruby
require 'rubygems'
require 'net/ssh'
require 'net/sftp'
require 'highline/import'
file = ARGV[ ARGV.index( '-i' ) + 1] if ARGV.include?( '-i' )
puts 'No input file specified' unless file;
host = "10.0.0.10"
user = "user"
prod = "new-#{file}" # product filename (call it <file>-new)
rpath = "/home/#{user}/.rffmpeg" # remote computer operating directory
rfile = "#{rpath}/#{file}" # remote filename
rprod = "#{rpath}/#{prod}" # remote product
cmd = "ffmpeg -i #{rfile} #{rprod}"# remote command, constructed
pass = ask("Password: ") { |q| q.echo = false } # password from stdin
Net::SSH.start(host, user ) do |ssh|
ssh.sftp.connect do |sftp|
# upload local 'file' to remote 'rfile'
sftp.upload!(file, rfile)
# run remote command 'cmd' to produce 'rprod'
ssh.exec!(cmd)
# download remote 'rprod' to local 'prod'
sftp.download!(rprod, prod)
end
end
now my problem is at
ssh.exec!(cmd)
I want to display the cmd's output to the local user in real-time. But making it
puts ssh.exec!(cmd)
I only get the resulting output after the command has finished running. How would I have to change the code to make this work?
On the display side of your question, you can generate an updating progress bar in Ruby using the "\r" string char. This backs you up to the beginning of the current line allowing you to re-write it. For example:
1.upto(100) { |i| sleep 0.05; print "\rPercent Complete #{i}%"}
Or if you just want a progress bar across the screen you can simply do something similar to this:
1.upto(50) { sleep 0.05; print "|"}
Also, relating to stdout, in addition to flushing output per previous example (STDOUT.flush), you can ask Ruby to automatically sync writes to an IO buffer (in this case STDOUT) with associated device writes (basically turns off internal buffering):
STDOUT.sync = true
Also, I find that sometimes flush doesn't work for me, and I use "IO.fsync" instead. For me that's mostly been related to file system work, but it's worth knowing.
From ri Net::SSH::start:
-------------------------------------------------------- Net::SSH::start
Net::SSH::start(host, user, options={}, &block) {|connection| ...}
------------------------------------------------------------------------
The standard means of starting a new SSH connection. When used with
a block, the connection will be closed when the block terminates,
otherwise the connection will just be returned. The yielded (or
returned) value will be an instance of
Net::SSH::Connection::Session (q.v.). (See also
Net::SSH::Connection::Channel and Net::SSH::Service::Forward.)
Net::SSH.start("host", "user") do |ssh|
ssh.exec! "cp /some/file /another/location"
hostname = ssh.exec!("hostname")
ssh.open_channel do |ch|
ch.exec "sudo -p 'sudo password: ' ls" do |ch, success|
abort "could not execute sudo ls" unless success
ch.on_data do |ch, data|
print data
if data =~ /sudo password: /
ch.send_data("password\n")
end
end
end
end
ssh.loop
end
So it looks like you can get more interactive by using #open_channel
Here's some example code:
user#server% cat echo.rb
#! /usr/local/bin/ruby
def putsf s
puts s
STDOUT.flush
end
putsf "hello"
5.times do
putsf gets.chomp
end
putsf "goodbye"
And on your local machine:
user#local% cat client.rb
#! /usr/local/bin/ruby
require 'rubygems'
require 'net/ssh'
words = %w{ earn more sessions by sleaving }
index = 0;
Net::SSH.start('server', 'user') do |ssh|
ssh.open_channel do |ch|
ch.exec './echo.rb' do |ch, success|
abort "could not execute ./echo.rb" unless success
ch.on_data do |ch, data|
p [:data, data]
index %= words.size
ch.send_data( words[index] + "\n" )
index += 1
end
end
end
end
user#local% ./client.rb
[:data, "hello\n"]
[:data, "earn\n"]
[:data, "more\n"]
[:data, "sessions\n"]
[:data, "by\n"]
[:data, "sleaving\n"]
[:data, "goodbye\n"]
So you can interact with a running process this way.
It's important that the running process flush its output before requesting input - otherwise, the program might hang as the channel may not have received the unflushed output.