Okay, so I am new to Ruby and I have a strong background in bash/ksh/sh.
What I am trying to do is use a simple for loop to run a command across several servers. In bash I would do it like:
for SERVER in `cat etc/SERVER_LIST`
do
ssh -q ${SERVER} "ls -l /etc"
done
etc/SERVER_LIST is just a file that looks like:
server1
server2
server3
etc
I can't seem to get this right in Ruby. This is what I have so far:
#!/usr/bin/ruby
### SSH testing
#
#
require 'net/ssh'
File.open("etc/SERVER_LIST") do |f|
f.each_line do |line|
Net::SSH.start(line, 'andex') do |ssh|
result = ssh.exec!("ls -l")
puts result
end
end
end
I'm getting these errors now:
andex#master:~/sysauto> ./ssh2.rb
/usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize': newline at the end of hostname (SocketError)
from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `open'
from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize'
from /usr/lib64/ruby/1.8/timeout.rb:53:in `timeout'
from /usr/lib64/ruby/1.8/timeout.rb:93:in `timeout'
from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh/transport/session.rb:65:in `initialize'
from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh.rb:179:in `new'
from /usr/lib64/ruby/gems/1.8/gems/net-ssh-2.0.23/lib/net/ssh.rb:179:in `start'
from ./ssh2.rb:10
from ./ssh2.rb:9:in `each_line'
from ./ssh2.rb:9
from ./ssh2.rb:8:in `open'
from ./ssh2.rb:8
The file is sourced correctly, I am using the relative path, as I am sitting in the directory under etc/ (not /etc, I'm running this out of a scripting directory where I keep the file in a subdirectory called etc.)
File.open("/etc/SERVER_LIST", "r") do |file_handle|
file_handle.each_line do |server|
# do stuff to server here
end
end
The first line opens the file for reading and immediately goes into a block. (The block is the code between do and end. You can also surround blocks with just { and }. The rule of thumb is do..end for multi-line blocks and {...} for single-line blocks.) Blocks are very common in Ruby. Far more idiomatic than a while or for loop.) The call to open receives the filehandle automatically, and you give it a name in the pipes.
Once you have a hold of that, so to speak, you can call each_line on it, and iterate over it as if it were an array. Again, each iteration automatically passes you a line, which you call what you like in the pipes.
The nice thing about this method is that it saves you the trouble of closing the file when you're finished with it. A file opened this way will automatically get closed as you leave the outer block.
One other thing: The file is almost certainly named /etc/SERVER_LIST. You need the initial / to indicate the root of the file system (unless you are intentionally using a relative value for the path to the file, which I doubt). That alone may have kept you from getting the file open.
Update for new error: Net::SSH is barfing up over the newline. Where you have this:
Net::SSH.start(line, 'andex') do |ssh|
make it this:
Net::SSH.start(line.chomp, 'andex') do |ssh|
The chomp method removes any final newline character from a string.
Use File.foreach:
require 'net/ssh'
File.foreach('etc/SERVER_LIST', "\n") do |line|
Net::SSH.start(line, 'andex') do |ssh|
result = ssh.exec!("ls -l")
puts result
end
end
The most common construct I see when doing by-line iteration of a file is:
File.open("etc/SERVER_LIST") do |f|
f.each_line do |line|
# do something here
end
end
To expand on the above with some more general Ruby info... this syntax is equivalent to:
File.open("etc/SERVER_LIST") { |f|
f.each_line { |line|
# do something here
}
}
When I was first introduced to Ruby, I had no idea what the |f| and |line| syntax meant. I knew when to use it, and how it worked, but not why they choose that syntax. It is, in my opinion, one of the magical things about Ruby. That simple syntax above is actually hiding a very advanced programming concept right under your nose. The code nested inside of the "do"/"end" or { } is called a block. And you can consider it an anonymous function or lambda. The |f| and |line| syntax is in fact just the handle to the parameter passed to the block of code by the executing parent.
In the case of File.open(), the anonymous function takes a single argument, which is the handle to the underyling File IO object.
In the case of each_line, this is an interator function which gets called once for every line. The |line| is simply a variable handle to the data that gets passed with each iteration of the function.
Oh, and one nice thing about do/end with File.open is it automatically closes the file at the end.
Edit:
The error you're getting now suggests the SSH call doesn't appreciate the extra whitespace (newline) at the end of the string. To fix this, simply do a
Net::SSH.start(line.strip, 'andex') do |ssh|
end
Reading in lines from a file is a common operation, and Ruby has an easy way to do this:
servers = File.readlines('/etc/SERVER_LIST')
The readlines method will open the file, read the file into an array, and close the file for you (so you don't have to worry about any of that). The variable servers will be an array of strings; each string will be a line from the file. You can use the Array::each method to iterate through this array and use the code you already have. Try this:
servers = File.readlines('/etc/SERVER_LIST')
servers.each {|s|
Net::SSH.start(s, 'andex') {|ssh| puts ssh.exec!("ls -l") }
}
I think this is what you want for the in 'initialize': newline at the end of hostname (SocketError) error:
Net::SSH.start(line.chomp, 'andex')
The each_line method includes the "\n", and the chomp will remove it.
Related
I have something similar.
# MAIN.RB
require 'sockets'
require_relative 'replies.rb'
hostname = 'localhost'
port = 6500
s = TCPSocket.open(hostname, port)
$connected = 0
while line = s.gets # Read lines from the socket
#DO A BUNCH OF STUFF
if line == "Hi"
reply line
end
end
s.close
Then I have the reply function in a secondary file.
# REPLIES.RB
def reply(input)
if input == "Hi"
s.write("Hello my friend.\n"
end
end
However calling on the object s from the second file does not seem to work. How would I go about making this work. I'm new to Ruby. I've searched google for the answer, but the only results I have found is with sharing variables across files. I could always do a return "Hello my friend.\n", but I rather be able to write to the socket object directly from the function in REPLIES.rb
Remember that variables are strictly local unless you expressly pass them in. This means s only exists in the main context. You can fix this by passing it in:
reply(s, line)
And on the receiving side:
def reply(s, input)
# ...
end
I'd strongly encourage you to try and indent things consistently here, this code is really out of sorts, and avoid using global variables like $connected. Using a simple self-contained class you could clean up this code considerably.
Also, don't add .rb extensions when calling require. It's implied.
Facing below issue eventhough the file is present in the folder.
H:\Ruby_test_works>ruby hurrah.rb
hurrah.rb:7:in `read': No such file or directory # rb_sysopen - H:/Ruby_
test_works/SVNFolders.txt (Errno::ENOENT)
from hurrah.rb:7:in `block in <main>'
from hurrah.rb:4:in `each_line'
from hurrah.rb:4:in `<main>'
Input file (input.txt) Columns are tab separated.
10.3.2.021.asd 10.3.2.041.def SVNFolders.txt
SubversionNotify Subversionweelta post-commit.bat
Commit message still rake customemail.txt
mckechney.com yahoo.in ReadMe.txt
Code :
dir = 'H:/Ruby_test_works'
file = File.open("#{dir}/input.txt", "r")
file.each_line do |line|
initial, final, file_name = line.split("\t")
#puts file_name
old_value = File.read("#{dir}/#{file_name}")
replace = old_value.gsub( /#{Regexp.escape(initial)}, #{Regexp.escape(final)}/)
File.open("#{dir}/#{file_name}", "w") { |fi| fi.puts replace }
end
I have tried using both forward and backward slashes but no luck. What I'm missing, not sure. Thanks.
puts file_name gives the below values
SVNFolders.txt
post-commit.bat
customemail.txt
ReadMe.txt
The file_name contains the newline character \n at the end, which won't get printed but messes up the path. You can fix the issue by stripping the line first:
initial, final, file_name = line.strip.split("\t")
When debugging code, be careful with puts. Quoting its documentation reveals an ugly truth:
Writes the given object(s) to ios. Writes a newline after any that do not already end with a newline sequence.
Another way to put this is to say it ignores (potential) newlines at the end of the object(s). Which is why you never saw that the file name actually was SVNFolders.txt\n.
Instead of using puts, you can use p when troubleshooting issues. The very short comparison between the two is that puts calls to_s and adds a newline, while p calls inspect on the object. Here is a bit more details about the differences: http://www.garethrees.co.uk/2013/05/04/p-vs-puts-vs-print-in-ruby/
Sometimes the issue is not the file, but the path to the file. Consider compare the file path with what you think the file path is with something like:
File.expand_path('my_file.rb')
Due to some OS restrictions I am not able to install ruby and mysql on my prod node. I have to process a Log file written on that node. So I planning to write the contents of the log file to a different file as its written to the log and work on that new file.
def watch_for(file)
f = File.open(file,"r")
f.seek(0,IO::SEEK_END)
while true do
select([f])
line = f.gets
open('myfile.out', 'a+') { |s|
s.puts "#{line}"
}
end
end
But this seems to failing; Any help to this is much appreciated.
It's not a great idea to reopen the output file each time through the loop. If you are just planning on appending and not reading the file, use 'a' rather than 'a+' for the IO mode. Also, there's no good reason to use string interpolation like "#{line}". I rewrote the method to open each file just once:
def watch_for(file)
f = File.open(file,"r")
f.seek(0,IO::SEEK_END)
s = File.open('myfile.out', 'a')
while true do
s.write(f.readline)
end
end
Probably better if this isn't a method at all since there's no way to exit except to kill the process. And, of course, you can just use tail -f $file >> myfile.out as mentioned in this comment.
When trying to delete a directory (+ contents) and after reading the files inside, FileUtils.rm_rf(path) will not delete all the folders, although it does delete all the files and some of the folders.
After some experimentation it seems to be related to a File.open block. (I actually do a regex match inside the block, but I'm just using a puts here to keep things clear)
File.open(file).each do |line|
puts line
end
From what I've read, the above should automatically close the file but when using this, FileUtils fails to complete its task.
However, if I use the following code, FileUtils works as desired.
open_file = File.open(file)
open_file.each do |line|
puts line
end
open_file.close
It's no big deal to use the code in the second example, but I do prefer the cleanliness of the first.
Is there any reason why that first example breaks FileUtils?
P.S. I'm new to both Ruby and Stack Overflow....Hi. My system is Ubuntu 11.04 (64bit), running RVM with Ruby 1.9.2-p180
You should use something like this:
File.open(file) do |f|
f.each{|line| puts line}
end
In your example the block is supplied to the each method and the version of open without a block is executed returning an IO object on which the each method is called.
All of the following API do the same thing: open a file and call a block for each line. Is there any preference we should use one than another?
File.open("file").each_line {|line| puts line}
open("file").each_line {|line| puts line}
IO.foreach("file") {|line | puts line}
There are important differences beetween those 3 choices.
File.open("file").each_line { |line| puts line }
File.open opens a local file and returns a file object
the file stays open until you call IO#close on it
open("file").each_line { |line| puts line }
Kernel.open looks at the string to decide what to do with it.
open(".irbrc").class # => File
open("http://google.com/").class # => StringIO
File.open("http://google.com/") # => Errno::ENOENT: No such file or directory - http://google.com/
In the second case the StringIO object returned by Kernel#open actually holds the content of http://google.com/. If Kernel#open returns a File object, it stays open untill you call IO#close on it.
IO.foreach("file") { |line| puts line }
IO.foreach opens a file, calls the given block for each line it reads, and closes the file afterwards.
You don't have to worry about closing the file.
File.read("file").each { |line| puts line }
You didn't mention this choice, but this is the one I would use in most cases.
File.read reads a file completely and returns it as a string.
You don't have to worry about closing the file.
In comparison to IO.foreach this makes it clear, that you are dealing with a file.
The memory complexity for this is O(n). If you know you are dealing with a small file, this is no drawback. But if it can be a big file and you know your memory complexity can be smaller than O(n), don't use this choice.
It fails in this situation:
s= File.read("/dev/zero") # => never terminates
s.each …
ri
ri is a tool which shows you the ruby documentation. You use it like this on your shell.
ri File.open
ri open
ri IO.foreach
ri File#each_line
With this you can find almost everything I wrote here and much more.