I need to append lines to a file on a remote server via SSH in Ruby. At the moment I'm doing it with the decidedly inelegant:
Net::SSH.start(host, user) do |ssh|
ssh.exec!("echo #{string} >> #{path}")
end
I'm guessing (hoping?) this isn't the best way to go about such things. Any suggestions?
Related
This is not a duplicate of How to SSH interactive Session or Net::SSH interactive terminal scripting. How? or Ruby net-ssh interactive response/reply
I'm trying to write an interactive SSH client using Net:SSH, because I'm already using it for run non-interactive commands on the target hosts.
I could just shell out to system "ssh" but it would require converting the connection settings, proxying, etc to ssh params.
The problem is streaming the data from STDIN to the shell channel. The Net::SSH documentation for listen_to shows how to do it when the input is from a socket instead of STDIN. $stdin or IO.console are not sockets and thus not compatible with Net::SSH::BufferedIo.
Is there a way to create a socket from STDIN that can be used for this? Or is there a better way to send everything from the STDIN to the Net::SSH::Channel until the channel closes?
Here's code that works, but is way too slow to be usable:
require 'net/ssh'
require 'io/console'
require 'io/wait'
Net::SSH.start('localhost', 'root') do |session|
session.open_channel do |channel|
channel.on_data do |_, data|
$stdout.write(data)
end
channel.on_extended_data do |_, data|
$stdout.write(data)
end
channel.request_pty do
channel.send_channel_request "shell"
end
channel.connection.loop do
$stdin.raw do |io|
input = io.readpartial(1024)
channel.send_data(input) unless input.empty?
end
channel.active?
end
end.wait
end
Sockets are really nothing more than file descriptors, and since STDIN is a file descriptor too, it doesn't hurt to try.
What you want however, is to put the TTY into raw mode first to get interactivity.
This code seems to work fine:
begin
system('stty cbreak -echo')
Net::SSH.start(...) do |session|
session.open_channel do |...|
...
session.listen_to(STDIN) { |stdin|
input = stdin.readpartial(1024)
channel.send_data(input) unless input.empty?
}
end.wait
end
ensure
system('stty sane')
end
I want a ruby script that will dump all the existing cron jobs to a text file using "crontab -l" or anything else that will achieve the same objective. Also the text file should be possible to use with crontab txtfile to create the cron jobs again.
Below is the code I already wrote:
def dump_pre_cron_jobs(file_path)
begin
cron_list = %x[crontab -l]
if(cron_list.size > 0)
cron_list.each do |crl|
mymethod_that_writes_tofile(file_path, crl) unless crl.chomp.include?("myfilter")
end
end
rescue Exception => e
raise(e.message)
end
end
Why does this need to be a Ruby script?
As you say, you can dump the crontab to a file with crontab -l > crontab.txt.
To read them back in again, simply use crontab crontab.txt, or cat crontab.txt | crontab -
I agree with #Vortura that you do not need to create a Ruby script to do this.
If you really want to, here is a probable way:
File.open('crontab.txt', 'w') do |crontab|
crontab << `crontab -l`
end
NOTE: Running this as root, or using sudo should capture all the cron jobs on a system, not just a single users' jobs. Run it as yourself or as that user and it might capture just those jobs. I haven't test that aspect of it.
Trying to run crontab -l to capture crontab files for all the users and packages seems the indirect way to do the task and could have the hassle of dealing with password requests hanging your code. I'd write code to comb through the directories that store them, rather than mess with prompts. Run the code using sudo and you shouldn't have any problems accessing the files.
Take a look at the discussion at: http://www.linuxquestions.org/questions/linux-newbie-8/etc-crontab-vs-etc-cron-d-vs-var-spool-cron-crontabs-853881/ for information on where the actual cron tab files are stored on disk.
Also https://superuser.com/questions/389116/how-to-recover-crontab-jobs-from-filesystem/389137 has similar information.
Mac OS varies a little from Linux in where Apple puts the cron files. Run man cron at the command-line for the definitive details on either OS.
Here's slightly-tested code for how I'd back up the files. How you restore them is for you to figure out, but it shouldn't be hard to figure out:
require 'fileutils'
BACKUP_PATH = '/path/to/some/safe/storage/directory'
CRONTAB_DIRS = %w[
/usr/lib/cron/tabs
/var/spool/cron
/etc/anacrontab
/etc/cron.d
]
CRONTAB_FILES = %w[
/etc/cron_list
]
def dump_pre_cron_jobs(file_path)
full_backup_path = File.join(
BACKUP_PATH,
File.dirname(file_path)
)
FileUtils.mkdir_p(full_backup_path) unless Dir.exist?(full_backup_path)
File.write(
File.join(
full_backup_path,
file_path
),
File.read(file_path)
)
rescue Exception => e
STDERR.puts e.message
end
CRONTAB_DIRS.each do |ct|
next unless Dir.exist?(ct)
begin
Dir.entries(File.join(ct, '*')).each { |fn| dump_pre_cron_jobs(fn) }
rescue Errno::EACCES => e
STDERR.puts e.message
end
end
CRONTAB_FILES.each do |fn|
dump_pre_cron_jobs(fn)
end
You'll need to run this as root via sudo to access the directories and files as they're usually locked down from unauthorized prying eyes.
The code creates a repository of crontabs, in BACKUP_PATH, based on their original file paths. No changes are made to the file contents so they can be restored as-is by copying them back via cp or writing code to reverse this process.
I have this code to download a file from a remote machine, but I want to limit it to files that are less than 5MB.
So far the code works. but is there a better way to check the filesize before downloading?
Net::SCP.start(hname, uname, :password => pw ) do|scp|
fsize=scp.download!("#{remdir}/#{filname}").size;puts fsize
scp.download!("#{remdir}/#{filname}", "#{filname}") do |ch, name, sent, total|
File.open(lfile, 'a'){ |f| f.puts "#{name}: #{(sent.to_f * 100 / total.to_f).to_i}% complete"}
#puts "Sending : #{(sent.to_f * 100 / total.to_f).to_i}% complete"
#puts "#{name}: #{sent}/#{total}"
#print "\r#{name}: #{(sent.to_f * 100 / total.to_f).to_i}%"
end
end
Does this cause any problem if I am use it for large files?
fsize=scp.download!("#{remdir}/#{filname}").size;puts fsize
This page says the file will be returned as a string:
http://ruby.about.com/od/ssh/ss/netscp_6.htm
Update:
I tried SFTP aswell. first it did not work for full path to file. and secondly it did not do what i wanted. so was using scp.download!().size. i know i am doing the download twice :(
require 'net/sftp'
# did not take full path "/user/myname/filename"
remote_path="somefile.txt"
Net::SFTP.start(hname, uname, :password => pw) do |sftp|
attrs = sftp.stat!("rome_desc.txt") ; puts attrs # gave # ☼ ↨?% (? '→ ??Q{ ?Qt;?
sftp.stat!(remote_path) do |response| puts response #returned no such file (2)
# but did not do below upload operation.
unless response.ok?
sftp.upload!("#{filname}", remdir)
end
end
end
Update:2 Solution
found the solution using the the comments provided from below users and after searching the net.
Net::SFTP.start(hname, uname, :password => pw) do |sftp| #, :verbose => Logger::DEBUG
sftp.dir.glob("./#{remdir}", "#{filname}") do |entry| p entry.name
file_size=entry.attributes.size; file_size = '%.2f' % (("#{file_size}").to_f / 2**20) ; File.open(lfile, 'a'){ |f| f.puts "File size is #{file_size} mb"}
if file_size < file_size_param then
sftp.download!("#{remdir}/#{filname}", filname)
else
File.open(lfile, 'a'){ |f| f.puts "File size is greater than #{file_size_param} mb. so can not Download File"}
end
end
end
used .attributes.size to obtain the file size and perform the download operation by checking the filesize.
sftp.dir.glob("./#{remdir}", "#{filname}") do |entry| p entry.name
file_size=entry.attributes.size
Does this cause any problem if I am use it for large files?
We don't know because we don't know how fast your internet connection is, how much RAM you have, how fast the pipe is from the host you're downloading the file from?
Basically though, you are reading the file twice, once into memory to see how big it is, then again if it meets your requirement, which seems really... silly.
You're doubling the traffic to the host you're reading from and on your network connection, and, if the file is larger than RAM on your local machine, it is going to go nuts.
As Darshan says, look at using Net::SFTP. It will give you the ability to query the file's size before you try to load it, without pulling the entire thing down. It's a bit more complicated to use, but that complexity translates into flexibility.
"/user/myname/filename"
(S)FTP might not necessarily have its base path where it can see that file. To probe the system and figure out, ask the system, via the SFTP connection, what its current directory is when you first login, then ask it for the files it can see using something like this example from the Net::STFP docs:
sftp.dir.glob("/base/path", "*/**/*.rb") do |entry|
p entry.name
end
That will recursively look through the "/base/path" hierarchy, searching for all "*.rb" files.
Your current code downloads the file, checks the size of the downloaded file (presumably to check if it is less than 5MB, but you don't actually do that), and then downloads it again. Even if you did something with fsize, it's too late to have not downloaded it.
I'd look into the sftp gem rather than scp; it should be pretty straightforward to do what you want with sftp, but not with scp.
Since Heroku does not allow saving dynamic files to disk, I've run into a dilemma that I am hoping you can help me overcome. I have a text file that I can create in RAM. The problem is that I cannot find a gem or function that would allow me to stream the file to another FTP server. The Net/FTP gem I am using requires that I save the file to disk first. Any suggestions?
ftp = Net::FTP.new(domain)
ftp.passive = true
ftp.login(username, password)
ftp.chdir(path_on_server)
ftp.puttextfile(path_to_web_file)
ftp.close
The ftp.puttextfile function is what is requiring a physical file to exist.
StringIO.new provides an object that acts like an opened file. It's easy to create a method like puttextfile, by using StringIO object instead of file.
require 'net/ftp'
require 'stringio'
class Net::FTP
def puttextcontent(content, remotefile, &block)
f = StringIO.new(content)
begin
storlines("STOR " + remotefile, f, &block)
ensure
f.close
end
end
end
file_content = <<filecontent
<html>
<head><title>Hello!</title></head>
<body>Hello.</body>
</html>
filecontent
ftp = Net::FTP.new(domain)
ftp.passive = true
ftp.login(username, password)
ftp.chdir(path_on_server)
ftp.puttextcontent(file_content, path_to_web_file)
ftp.close
David at Heroku gave a prompt response to a support ticket I entered there.
You can use APP_ROOT/tmp for temporary file output. The existence of files created in this dir is not guaranteed outside the life of a single request, but it should work for your purposes.
Hope this helps,
David
I'm trying to connect, using Net::SSH, to a server that immediately after
login executes a script that requires input from user. The user has to enter "1" or "2" and will receive some data via in the terminal afterwards.
My problem is that, although I am able to connect, I can not figure out a way to send "1\n" to the server and to receive the output.
The following code stops at "INFO -- net.ssh.connection.session[80906b74]: channel_open_confirmation: 0 0 0 32768".
Using channel.exec( "1\n" ) instead of channel.send_data unsurprisingly does not work either.
Net::SSH.start('host', 'user', :password => "pass", :auth_methods => ["password"], :verbose => :debug) do |session|
session.open_channel do |channel|
channel.on_data do |ch, data|
STDOUT.print data
end
channel.send_data( "1\n")
end
session.loop
end
Any ideas, anyone?
Thanks in advance
Can you verify that your send_data call is happening after you get the prompt from the remote server? Try constructing a channel.on_data block around your send_data call so that you can verify that you get the expected prompt from the server before you send a response.
You might not want to be using exec here. From the docs for Net::SSH::Connection::Channel:
Sends a channel request asking that
the given command be invoked.
You are wanting to send a text string to reply to a prompt, not invoke a command. The docs show exec being used to send full CLI commands like "ls -l /home".
Instead, send_data is probably what you want. The docs show it used to send arbitrary text such as channel.send_data("the password\n"). Note, however, this sentence in the docs:
Note that it does not immediately send
the data across the channel, but
instead merely appends the given data
to the channel‘s output buffer,
preparatory to being packaged up and
sent out the next time the connection
is accepting data.
You might want to take a look at channel.request_pty. It appears to be designed for interaction with a console-based application.
If you are trying to (in essence) script an SSH session that you would normally do manually, you may find it easier to use an expect-like interface (for example, a gem like sshExpect might be worth a try).
Thank you all for the pointers. I have been able to put my finger on the problem – besides using channel.request_pty it was also necessary to request a shell. The following finally works as expected:
Net::SSH.start('host', 'user', :password => "pass", :auth_methods => ["password"]) do |session|
session.open_channel do |channel|
channel.request_pty do |ch, success|
raise "Error requesting pty" unless success
ch.send_channel_request("shell") do |ch, success|
raise "Error opening shell" unless success
end
end
channel.on_data do |ch, data|
STDOUT.print data
end
channel.on_extended_data do |ch, type, data|
STDOUT.print "Error: #{data}\n"
end
channel.send_data( "1\n" )
session.loop
end
end
I'm not terribly familiar with the Net::SSH libs so I can't help with that per-se but it sounds like you could achieve what you want using Capistrano.
For example I have a capistrano task which connects to a remote server, runs a command which expects input and then continues. Capistrano takes care of the remote i/o. Maybe that could be a solution for you?
Hope it helps!
If I execute "1\n" in a shell the reply I get is: bash: 1: command not found
If I execute echo "1" I get: 1
Are you sure you want to try to execute the text you are sending? Maybe you were looking for something like:
output = ""
Net::SSH.start('host', 'user', :password => "pass") do |ssh|
output = ssh.exec! "echo 1"
end
puts output
I'm not proficient with that lib, but on SSH you can open multiple channels. Maybe the server only responds to the first default channel and if you open another one you get a fresh shell.