I have done a program that sends requests to a url and saves them in a file. The program is this, and is working perfectly:
require 'open-uri'
n = gets.to_i
out = gets.chomp
output = File.open( out, "w" )
for i in 1..n
response = open('http://slowapi.com/delay/10').read
output << (response +"\n")
puts response
end
output.close
I want to modify it so that I can execute it from command line. I must run it like this:
fle --test abc -n 300 -f output
What must I do?
Something like this should do the trick:
#!/usr/bin/env ruby
require 'open-uri'
require 'optparse'
# Prepare the parser
options = {}
oparser = OptionParser.new do |opts|
opts.banner = "Usage: fle [options]"
opts.on('-t', '--test [STRING]', 'Test string') { |v| options[:test] = v }
opts.on('-n', '--count COUNT', 'Number of times to send request') { |v| options[:count] = v.to_i }
opts.on('-f', '--file FILE', 'Output file', :REQUIRED) { |v| options[:out_file] = v }
end
# Parse our options
oparser.parse! ARGV
# Check if required options have been filled, print help and exit otherwise.
if options[:count].nil? || options[:out_file].nil?
$stderr.puts oparser.help
exit 1
end
File::open(options[:out_file], 'w') do |output|
options[:count].times do
response = open('http://slowapi.com/delay/10').read
output.puts response # Puts the response into the file
puts response # Puts the response to $stdout
end
end
Here's a more idiomatic way of writing your code:
require 'open-uri'
n = gets.to_i
out = gets.chomp
File.open(out, 'w') do |fo|
n.times do
response = open('http://slowapi.com/delay/10').read
fo.puts response
puts response
end
end
This uses File.open with a block, which allows Ruby to close the file once the block exits. It's a much better practice than assigning the file handle to a variable and use that to close the file later.
How to handle passing in variables from the command-line as options is handled in the other answers.
The first step would be to save you program in a file, add #!/usr/bin/env ruby at the top and chmod +x yourfilename to be able to execute your file.
Now you are able to run your script from the command line.
Secondly, you need to modify your script a little bit to pick up command line arguments. In Ruby, the command line arguments are stored inside ARGV, so something like
ARGV.each do|a|
puts "Argument: #{a}"
end
allows you to retrieve command line arguments.
Related
This is homework and I do not expect you to give me the complete answer.
I'm trying to parse a command line entry such as:
./apacheReport.rb -u testlog.txt
When I enter this:
./apacheReport.rb -u testlog.txt
I get:
Argument required
My code is:
require_relative 'CommonLog'
require 'optparse'
# puts ARGV.inspect
optparser = OptionParser.new
optU = false
optI = false
optS = false
optH = false
optparser.banner = "apacheReport.rb [options] filename"
optparser.parse!
rescue => m
puts m.message
puts optparser
exit
end
if ARGV.length < 1
puts "Argument required"
exit
end
userInputFile = ARGV[0]
userInputFile.to_s
file = CommonLog.new(userInputFile)
It should parse the leftover portion of the command into ARGV[0] then should store it as userInputFile and then create a CommonLog object using the file as the constructor. At that point I call the methods that were specified in the command.
It seems that for some reason my ARGV is not being returned. I'm not sure what the issue is.
Ruby's OptionParser is easy to use, but you have to puzzle through the documentation. Here's a little example that'd be useful for your code:
require 'optparse'
options = {}
OptionParser.new do |opt|
opt.on('-u', '--use_this FILE', 'Use this file') { |o| options[:use_this] = o }
end.parse!
options will contain the flags. In this case, if you pass in -u foo, options[:use_this] will be foo.
Save that and try running it without and with a parameter. Also try running it with just a -h flag.
You can search StackOverflow for more answers where I was dealing with OptionParser.
It's hard to tell what's wrong since you code doesn't seem to be working at the moment. The problem may be that the parse! method removes found options from ARGV. So when you write:
optparser.parse!
It removes your two parameters (-u testlog.txt) and this code always fails:
if ARGV.length < 1
puts "Argument required"
exit
end
Instead of looking at ARGV, you need to set up optparser correctly. Perhaps something like:
optparser = OptionParser.new do |opts|
opts.banner = "apacheReport.rb [options] filename"
opts.on("-u", "--u-short-for-this", "Whatever u stands for") do |u|
optU = u
end
end
Then optU will be true only if the user passed -u and the filename will be in ARGV[0].
I wounder how I can save the parsed data to a txt file. My script is only saving the last parsed. Do i need to add .each do ? kind of lost right now
here is my code and if maybe somebody could explain to me how save the parsed info on a new line
here is the code
require 'rubygems'
require 'nokogiri'
require 'open-uri'
url = "http://www.clearsearch.se/foretag/-/q_advokat/1/"
doc = Nokogiri::HTML(open(url))
doc.css(".gray-border-bottom").each do |item|
title = item.css(".medium").text.strip
phone = item.css(".grayborderwrapper > .bold").text.strip
adress = item.css(".grayborder span").text.strip
www = item.css(".click2www").map { |link| link['href'] }
puts "#{title} ; \n"
puts "#{phone} ; \n"
puts "#{adress} ; \n"
puts "#{www} ; \n\n\n"
puts "Writing"
company = "#{title}; #{phone}; #{adress}; #{www} \n\n"
puts "saving"
file = File.open("exporterad.txt", "w")
file.write(company)
file.close
puts "done"
end
puts "done"
Calling File.open inside your loop truncates the file to zero length with each invocation. Instead, open the file outside your loop (using the block form):
File.open("exporterad.txt", "w") do |file|
doc.css(".gray-border-bottom").each do |item|
# ...
file.write(company)
# ...
end
end # <- file is closed automatically at the end of the block
I want to bulid a multi threaded application. If i do not use threads, everything works fine. When i try to use threads, then nothing is displayed on the browser. when i use the syntax 'puts "%s" %io.read' then it displays on the command prompt and not on the browser. Any help would be appreciated.
require 'sinatra'
require 'thread'
set :environment, :production
get '/price/:upc/:rtype' do
Webupc = "#{params[:upc]}"
Webformat = "#{params[:rtype]}"
MThread = Thread.new do
puts "inside thread"
puts "a = %s" %Webupc
puts "b = %s" %Webformat
#call the price
Maxupclen = 16
padstr = ""
padupc = ""
padlen = (Maxupclen - Webupc.length)
puts "format type: #{params[:rtype]}"
puts "UPC: #{params[:upc]}"
puts "padlen: %s" %padlen
if (Webformat == 'F')
puts "inside format"
if (padlen == 0 ) then
IO.popen("tstprcpd.exe #{Webupc}")
{ |io|
"%s" %io.read
}
elsif (padlen > 0 ) then
for i in 1 .. padlen
padstr = padstr + "0"
end
padupc = padstr + Webupc
puts "padupc %s" %padupc
IO.popen("tstprcpd.exe #{padupc}") { |io|
"%s" %io.read
}
elsif (padlen < 0 ) then
IO.popen("date /T") { |io|
"UPC length must be 16 digits or less." %io.read
}
end
end
end
end
Your code has several problems:
It is not formatted properly
You are using Uppercase names for variables; that makes them constants!
puts will not output to the browser, but to the console. The browser will recieve the return value of the block, i.e. the return value of the last statement in the block. Therefore, you need to build your output differently (see below).
You are never joining the thread
Here's a minimal sinatra app that uses a thread. However, the thread makes no sense in this case because you must wait for its termination anyway before you can output the result to the browser. In order to build the output I have used StringIO, which you can use with puts to build a multiline string conveniently. However, you could also simply initialize res with an empty string with res = "" and then append your lines to this string with res << "new line\n".
require 'sinatra'
require 'thread'
require 'stringio'
get '/' do
res = StringIO.new
th = Thread.new do
res.puts 'Hello, world!'
end
th.join
res.string
end
I am using Popen3 to run some Perl scrips and then dump their output into a text file. Once in the text file, I search for the result of the Perl script. I get the error after running for about 40 minutes, which is about 220 files.
ruby/1.8/open3.rb:49:in `pipe': Too many open files (Errno::EMFILE)
from /ruby/1.8/open3.rb:49:in `popen3'
from ./RunAtfs.rb:9
from ./RunAtfs.rb:8:in `glob'
from ./RunAtfs.rb:8
The script is below.
require 'logger'
require 'open3'
atfFolder = ARGV[0]
testResult = ARGV[1]
res = "result.txt"
open('result.txt', 'w') { }
Dir.glob(atfFolder+'/*.pl') do |atfTest|
Open3.popen3("atf.pl -c run-config.pl -t #{atfTest}") do |i, o, e, t|
while line = e.gets
$testFile = testResult + line[/^[0-9]+$/].to_s + "testOutput.txt"
log = Logger.new($testFile)
log.info(line)
end
log.close
end
lastLine = `tail +1 #{$testFile}`
file = File.open(res, 'a')
if(lastLine.include? "(PASSED)")
file.puts("Test #{atfTest} --> Passed")
file.close
File.delete($testFile)
else
file.puts("Test #{atfTest} --> Failed!")
file.close
end
end
This script is processing 4900 Perl files so I don't know if that is just too many files for popen3 or I am not using it correctly.
Thanks for helping me!
I refactored my script after some very helpful pointers! The code is working great!
require 'open3'
atf_folder, test_result = ARGV[0, 2]
File.open('result.txt', 'w') do |file| end
Dir.glob("#{ atf_folder }/*.pl") do |atf_test|
test_file = atf_test[/\/\w+.\./][1..-2].to_s + ".txt"
comp_test_path = test_result + test_file
File.open(comp_test_path, 'w') do |file| end
Open3.popen3("atf.pl -c run-config.pl -t #{ atf_test }") do |i, o, e, t|
while line = e.gets
File.open(comp_test_path, 'a') do |file|
file.puts(line)
end
end
end
last_line = `tail +1 #{comp_test_path}`
File.open('result.txt', 'a') do |file|
output_str = if (last_line.include? "(PASSED)")
File.delete(comp_test_path)
"Passed"
else
"Failed!"
end
file.puts "Test #{ atf_test } --> #{ output_str }"
end
end
Consider this:
require 'logger'
require 'open3'
atf_folder, test_result = ARGV[0, 2]
Dir.glob("#{ atf_folder }/*.pl") do |atf_test|
Open3.popen3("atf.pl -c run-config.pl -t #{ atf_test }") do |i, o, e, t|
while line = e.gets
$testFile = test_result + line[/^[0-9]+$/].to_s + "testOutput.txt"
log = Logger.new($testFile)
log.info(line)
log.close
end
end
lastLine = `tail +1 #{ $testFile }`
File.open('result.txt', 'a') do |file|
output_str = if (lastLine.include? "(PASSED)")
File.delete($testFile)
"Passed"
else
"Failed!"
end
file.puts "Test #{ atf_test } --> #{ output_str }"
end
end
It's untested, of course, since there's no sample data, but it's written more idomatically for Ruby.
Things to note:
atf_folder, test_result = ARGV[0, 2] slices the ARGV array and uses parallel assignment to retrieve both parameters at once. You should test to see that you got values for them. And, as you move to more complex scripts, take advantage of the OptionParser class that comes in Ruby's STDLIB.
Ruby lets us pass a block to File.open, which automatically closes the file when the block exits. This is a major strength of Ruby and helps reduce errors like you're seeing. Logger doesn't do that, so extra care has to be take to avoid leaving hanging file-handles, like you're doing. Instead, use:
log = Logger.new($testFile)
log.info(line)
log.close
to immediately close the handle. You are doing it outside the loop, not inside it, so you had a bunch of open handles.
Also consider whether you need Logger, or if a regular File.open would suffice. Logger has additional overhead.
Your use of $testFile is questionable. $variables are globals, and their use is generally an indicator you're doing something wrong, at least until you understand why and when you should use them. I'd refactor the code using that.
In Ruby, variables and methods are in snake_case, not CamelCase, which is used for classes and modules. That doesn't seem like much until_you_run_into CodeDoingTheWrongThing and_have_to_read_it. (Notice how your brain bogged down deciphering the camel-case?)
In general, I question whether this is the fastest way to do what you want. I suspect you could write a shell script using grep or tail that would at least keep up, and maybe run faster. You might sit down with your sysadmin and do some brain-pickin'.
I have a Ruby app which uses readline with command completion.
After the first string (the command) was typed, I would like to be able to complete its arguments. The arguments list should be based on the chosen command.
Does someone have a quick example?
These are the commands:
COMMANDS = [
'collect', 'watch'
].sort
COLLECT = [
'stuff', 'otherstuff'
].sort
comp = proc do |s|
COMMANDS.grep( /^#{Regexp.escape(s)}/ )
end
Readline.completion_proc = comp
Each time I press TAB, the proc block is executed and a command from the COMMANDS array is matched.
After one of the commands was fully matched I would like to start searching for the argument only in the COLLECT array.
Since your question popped up first every time I looked for something like this I want to share my code for any one else.
#!/usr/bin/env ruby
require 'readline'
module Shell
PROMPT = "shell> "
module InputCompletor
CORE_WORDS = %w[ clear help show exit export]
SHOW_ARGS = %w[ list user ]
EXPORT_ARGS = %w[ file ]
COMPLETION_PROC = proc { |input|
case input
when /^(show|export) (.*)/
command = $1
receiver = $2
DISPATCH_TABLE[$1].call($2)
when /^(h|s|c|e.*)/
receiver = $1
CORE_WORDS.grep(/^#{Regexp.quote(receiver)}/)
when /^\s*$/
puts
CORE_WORDS.map{|d| print "#{d}\t"}
puts
print PROMPT
end
}
def self.show(receiver)
if SHOW_ARGS.grep(/^#{Regexp.quote(receiver)}/).length > 1
SHOW_ARGS.grep(/^#{Regexp.quote(receiver)}/)
elsif SHOW_ARGS.grep(/^#{Regexp.quote(receiver)}/).length == 1
"show #{SHOW_ARGS.grep(/^#{Regexp.quote(receiver)}/).join}"
end
end
def self.export(receiver)
if EXPORT_ARGS.grep(/^#{Regexp.quote(receiver)}/).length > 1
EXPORT_ARGS.grep(/^#{Regexp.quote(receiver)}/)
elsif EXPORT_ARGS.grep(/^#{Regexp.quote(receiver)}/).length == 1
"export #{EXPORT_ARGS.grep(/^#{Regexp.quote(receiver)}/).join}"
end
end
DISPATCH_TABLE = {'show' => lambda {|x| show(x)} ,
'export' => lambda {|x| export(x)}}
end
class CLI
Readline.completion_append_character = ' '
Readline.completer_word_break_characters = "\x00"
Readline.completion_proc = Shell::InputCompletor::COMPLETION_PROC
def initialize
while line = Readline.readline("#{PROMPT}",true)
Readline::HISTORY.pop if /^\s*$/ =~ line
begin
if Readline::HISTORY[-2] == line
Readline::HISTORY.pop
end
rescue IndexError
end
cmd = line.chomp
case cmd
when /^clear/
system('clear')
when /^help/
puts 'no help here'
when /show list/
puts 'nothing to show'
when /^show\s$/
puts 'missing args'
when /export file/
puts 'nothing to export'
when /^export\s$/
puts 'missing args'
when /^exit/
exit
end
end
end
end
end
Shell::CLI.new
After thinking a while, the solution was very simple:
comp = proc do |s|
if Readline.line_buffer =~ /^.* /
COLLECT.grep( /^#{Regexp.escape(s)}/ )
else
COMMANDS.grep( /^#{Regexp.escape(s)}/ )
end
end
Now I just need to turn it into something more flexible/usable.