I decided to rewrite a ruby script I did as a gem/module with an executable component. The original works fine, but as a gem it is easier to maintain/install with all dependencies.
The command takes an array of hosts and a command, and runs it on the hosts via threads. The problem I am now having though is that the output of the command is not displayed on the terminal. Executing the module code from within IRB however produces the output.
The module code is as follows:
require "rcmd/version"
require 'net/ssh'
require 'thread'
module Rcmd
#queue = Queue.new
class << self
attr_accessor :nthreads
attr_accessor :user
attr_accessor :quiet
attr_accessor :command
attr_accessor :host_list
attr_accessor :threads
end
# Built in function called by each thread for executing the command on individual hosts
def Rcmd.run_command_on_host(conn_options)
begin
# Create ssh session to host
Net::SSH.start(conn_options[:host], conn_options[:user], :password => conn_options[:passwd]) do |session|
# Open channel for input/output control
session.open_channel do |channel|
channel.on_data do |ch, data|
# Print recieved data if quiet is not true
puts "#{conn_options[:host]} :: #{data}" unless conn_options[:quiet]
end
channel.on_extended_data do |ch,type,data|
# Always print stderr data
puts "#{conn_options[:host]} :: ERROR :: #{data}"
end
# Execute command
channel.exec #command
end
# Loop until command completes
session.loop
end
rescue
puts "#{conn_options[:host]} :: CONNECT ERROR :: Unable to connect to host!\n"
end
end
# Main method of module for starting the execution of the specified command on provided hosts
def Rcmd.run_command()
if not #command
raise ArgumentError.new("No command set for execution")
end
if not #host_list.count >= 1
raise ArgumentError.new("host_list must contain at least one system")
end
#host_list.each do |host|
#queue << host
end
until #queue.empty?
# Don't start more threads then hosts.
num_threads = #nthreads <= #host_list.count ? #nthreads : #host_list.count
# Prepare threads
#threads = { }
num_threads.times do |i|
#threads[i] = Thread.new {
conn_options = { :user => #user, :host => #queue.pop, :password => nil, :quiet => #quiet}
unless conn_options[:host].nil?
self.run_command_on_host(conn_options)
end
}
end
# Execute threads
#threads.each(&:join)
end
end
end
Test snippet:
require 'rcmd'
Rcmd.host_list= ["localhost", "dummy-host"]
Rcmd.nthreads= 2
Rcmd.user= 'root'
Rcmd.command= "hostname -f"
Rcmd.run_command
Running the above in IRB produces:
dummy-host :: CONNECT ERROR :: Unable to connect to host!
localhost :: darkstar.lan
As expected. However, Running the same from a script file (the gem command) or in ruby directly results in no output:
ruby-newb#darkstar:~/repos/rcmd$ rcmd -n localhost,dummy-host -c 'hostname -f'
ruby-newb#darkstar:~/repos/rcmd$
Previously I used $stdout.puts and $stderr.puts as well as the constants variants but in desperation moved it to just puts. I have also tried with print and various other ways to include handing the streams to the threads and printing all output once the thread finished , but to no avail.
After adding tons of 'p' statements through out the code, when running from the command the output stops just before the Net::SSH.start call in the run_command_on_host method.
I have also tried having that entire method within the thread creations but then it failed to execute completely. Hence having two methods. One for creating the threads, and one used by the threads for executing the ssh session and command.
Failing in both ruby-2.3.3 and ruby-2.0.0-p648 so I think I am just doing something stupid.
If someone could find it in their hearts to tell this Ruby newbie where he has gone wrong it would be very appreciated.
I am not sure exactly why but it seems the method call was exiting (not waiting on the threads despite the thread joins) and thus killing the threads. As such, the Net::SSH module did not have time to get/return data.
I added the following unless statement near the end of the code to force the method to wait for the threads to complete and now output is displayed as desired. Also reduced the amount of code a bit.
unless #threads.each.map {|t| t.alive?}.none?
sleep 1
end
Related
I have a problem with the testing the Sensu Plugin.
Everytime when I start rspec to test plugin it test it, but anyway at the end of test, the original plugin is started automatically. So I have in my console:
Finished in 0 seconds (files took 0.1513 seconds to load)
1 example, 0 failures
CheckDisk OK: # This comes from the plugin
Short explanation how my system works:
Plugin call system 'wmic' command, processes it, checks the conditions about the disk parameters and returns the exit statuses (ok, critical, etc)
Rspec mocks the response from system and sets into the input of plugin. At the end rspec checks the plugin exit status when the mocked input is given.
My plugin looks like that:
require 'rubygems' if RUBY_VERSION < '1.9.0'
require 'sensu-plugin/check/cli'
class CheckDisk < Sensu::Plugin::Check::CLI
def initialize
super
#crit_fs = []
end
def get_wmic
`wmic volume where DriveType=3 list brief`
end
def read_wmic
get_wmic
# do something, fill the class variables with system response
end
def run
severity = "ok"
msg = ""
read_wmic
unless #crit_fs.empty?
severity = "critical"
end
case severity
when /ok/
ok msg
when /warning/
warning msg
when /critical/
critical msg
end
end
end
Here is my test in Rspec:
require_relative '../check-disk.rb'
require 'rspec'
def loadFile
#Load template of system output when ask 'wmic volume(...)
end
def fillParametersInTemplate (template, parameters)
#set mocked disk parameters in template
end
def initializeMocks (options)
mockedSysOutput = fillParametersInTemplate #loadedTemplate, options
po = String.new(mockedSysOutput)
allow(checker).to receive(:get_wmic).and_return(po) #mock system call here
end
describe CheckDisk do
let(:checker) { described_class.new }
before(:each) do
#loadedTemplate = loadFile
def checker.critical(*_args)
exit 2
end
end
context "When % of free disk space = 10 >" do
options = {:diskName => 'C:\\', :diskSize => 1000, :diskFreeSpace => 100}
it 'Returns ok exit status ' do
begin
initializeMocks options
checker.run
rescue SystemExit => e
exit_code = e.status
end
expect(exit_code).to eq 0
end
end
end
I know that I can just put "exit 0" after the last example, but this is not a solution because when I will try to start many spec files it will exit after the first one. How to start only test, without running the plugin? Maybe someone can help me and show how to handle with such problem?
Thank you.
You can stub the original plugin call and optionally return a dummy object:
allow(SomeObject).to receive(:method) # .and_return(double)
you can put it in the before block to make sure that all assertions will share the code.
Another thing is that you are using rescue blocks to catch the situation when your code aborts with an error. You should use raise_error matcher instead:
expect { run }.to raise_error(SystemExit)
I'm building a task runner where each task is built from a number of commands:
def run
begin
#Validating task params
set_progress "Validating params", "Validating params: #{#params}"
validate_params
#task_info["steps"].each do |step|
#log.info "Running command: #{step["description"]}"
set_progress step["description"]
command = Command.factory #params, step, #signature, #log
timeout = General.in_seconds step["timeout"]
command_res = Timeout.timeout(timeout) do
command.execute
end
end
set_progress "Completed"
rescue Exception => exception
#log.error exception.message + "\nBACK TRACE:\n" + exception.backtrace.join("\n")
set_progress #progress, "Failed, check logs. exception: #{exception.message}"
end
end
Now the command is ran by "command.execute", and there is a field inside the command class which is called "current_status" which i would like to monitor each X seconds and check the command status in order to update the user about the command status, how can i do it ? i probably need to run the command in a separate thread and then monitor it, but how can i monitor it ?
a quick and dirty methodology which might contain syntax errors :P
class JobManager
def initialize
#threads =[]
end
def registered_jobs
#registered_jobs ||= [Job1.new, Job2.new]
end
def start_jobs
registered_jobs.each {|j| #threads << Thread.new { j.run } }
end
def statuses?
registered_jobs.collect {|j| j.status? }
end
end
Usage:
manager = JobManager.new
manager.start_jobs
# elsewhere
manager.statuses? # returns [Job1.status?, Job2.status?]
This is the sort of idiom I'd use in my code. It's important to be aware that the status variables are subject to race conditions if they are not properly guarded against concurrent modification and access.
I'm a DBA, and I stumbled upon such case: developers run irb sessions (from Ruby on Rails app). This irb keeps database connection open. Sometimes - they forget about it, and it keeps on "running" - not doign anything, but still using one db connection.
I'd like to add some kind of "idle timeout" to their irb config. Is it possible? How to do it?
Here's a quick hack how you might implement this.
Note that this does not take into account that the user might be executing some long-running task inside the irb session. It simply looks at the time stamp of the last input; if it has not changed then it just flat out kills the process:
Update: it now checks if irb is currently running a command and ignores any timeouts if that is the case.
# Add some methods to IRB::Context and IRB::Irb
# for easier timeout implementation.
class IRB::Irb
def status
#signal_status
end
end
class IRB::Context
attr_reader :irb
attr_reader :line_no
def is_evaluating?
self.irb.status == :IN_EVAL
end
end
# Implement an auto exit timer thread. Timeout is given in seconds.
module IRB
def self.auto_exit_after(timeout = 60)
Thread.new {
context = IRB.conf[:MAIN_CONTEXT]
last_input = Time.now
last_line = context.line_no
loop {
sleep 10
# Check if irb is running a command
if context.is_evaluating?
# Reset the input time and ignore any timeouts
last_input = Time.now
next
end
# Check for new input
if last_line != context.line_no
# Got new input
last_line = context.line_no
last_input = Time.now
next
end
# No new input, check if idle time exceeded
if Time.now - last_input > timeout
$stderr.puts "\n** IRB exiting due to idle timeout. Goodbye..."
Process.kill("KILL", Process.pid)
end
}
}
end
end
To use it add the code to .irbrc, or some other place that auto-loads when irb is started, and then just start the timer:
IRB.auto_exit_after(60)
I have an eventmachine app where one script is reading from a file, sending data to another script line by line, and the "server" script is acting upon that data. Unfortunately, the "server" script fails to execute receive_data as it should. I know that a connection is being made because it eecutes post_init, and I know the sender script is sending data. Here is some of my code along with how I start the server.
module BT_Server
def post_init
puts "-- someone connected to the echo server!"
end
def receive_data(data)
puts "hi"
int, time, *int_macs = data.split("-")
# more stuff that isn't needed here
end
def bt_left(dev)
dev.save
if t = Device.macs.index(dev.mac)
Device.all[t].add_int(dev.int, dev.t_0, dev.t_l)
else
Device.new(dev.mac, dev.int, dev.t_0, dev.t_l)
end
return false
end
def unbind
puts "disconnection"
end
end
EventMachine::run {
EventMachine::start_server 'localhost', 8081, BT_Server
puts t_0 = Time.new
puts 'listening...'
}
Note: I have the Module definition in a separate file, along with my classes, which I require into the server script, if that makes any difference.
i tested your code and it outputs 'hi' every time i send something via telnet.
from my point of view, the code is correct.
are you sure the sending script is working? try with a manual telnet on port 8081.
regards.
This is related to a question I asked here:
Thread Locking in Ruby (use of soap4r and QT)
However it is particular to one part of that question and is supported by a simpler example. The test code is:
require 'rubygems'
require 'thread'
require 'soap/rpc/standaloneserver'
class SOAPServer < SOAP::RPC::StandaloneServer
def initialize(* args)
super
# Exposed methods
add_method(self, 'test', 'x', 'y')
end
def test(x, y)
return x + y
end
end
myServer = SOAPServer.new('monitorservice', 'urn:ruby:MonitorService', 'localhost', 4004)
Thread.new do
puts 'Starting web services'
myServer.start
puts 'Ending web services'
end
sleep(4)
#Thread.new do
testnum = 0
while testnum < 4000 do
testnum += 1
puts myServer.test(0,testnum)
sleep(2)
end
#end
puts myServer.test(0,4001)
puts myServer.test(0,4002)
puts myServer.test(0,4003)
puts myServer.test(0,4004)
gets
When I run this with the thread commented out everything runs along fine. However, once the thread is put in the process hangs. I poked into Webrick and found that the stop occurs here (the puts are, of course, mine):
while #status == :Running
begin
puts "1.1"
if svrs = IO.select(#listeners, nil, nil, 2.0)
svrs[0].each{|svr|
puts "-+-"
#tokens.pop # blocks while no token is there.
if sock = accept_client(svr)
th = start_thread(sock, &block)
th[:WEBrickThread] = true
thgroup.add(th)
else
#tokens.push(nil)
end
}
end
puts ".+."
When run with the thread NOT commented out I get something like this:
Starting web services
1.1
.+.
1.1
4001
4002
4003
4004
1
.+.
1.1
If the problem is caused by the gets() call and the purpose of the gets() call in your code is to prevent the Ruby interpreter from exiting, you can replace it with Thread.join() calls for each thread that you create. Join() will block until that thread has finished executing and therefore it'll prevent the Ruby interpreter from exiting.
E.g.:
t1 = Thread.new do
puts 'Starting web services'
myServer.start
puts 'Ending web services'
end
t2 = ...
...
t1.join
t2.join
Alternatively, if you can join() only one of the threads if there is a single thread that controls the execution of the application, and the other threads will be killed on exit.
The trailing gets blocks Ruby's IO. I'm not sure why. If it is replaced with pretty much anything the program works. I used a sleeping loop:
loop do
sleep 1
end
ADDED:
I should note that I also get strange behavior with sleep based on the sleep increment. In the end I abandoned Ruby since the threading behavior was too wonky.