I'm making a command line tool using Ruby. It will print a lot of text on screen. Currently, I'm using shell pipeline (may_app | more) to do so. But I think it's better to has a default pager.
It's just like what you see when execute git log . One can disable pager by using git --nopager log.
I've done quite much google work and find one gem: hirb , but it seems a little overkill.
After many tries, I'm current using shell wrapper to do so:
#!/bin/bash
# xray.rb is the core script
# doing the main logic and will
# output many rows of text on
# screen
XRAY=$HOME/fdev-xray/xray.rb
if [ "--nopager" == "$1" ]; then
shift
$XRAY $*
else
$XRAY $* | more
fi
It works. But is there a better way?
You are doing it right. But instead using more you'd better get a pager from $PAGER environment variable, if any.
Some people prefer less to more for example, and others have their favorite parser options set in this var.
You can use the pipe in Ruby via a call to system and provide the options (along with a nice help interface) like so:
require 'optparse'
pager = ENV['PAGER'] || 'more'
option_parser = OptionParser.new do |opts|
opts.on("--[no-]pager",
"[don't] page output using #{pager} (default on)") do |use_pager|
pager = nil unless use_pager
end
end
option_parser.parse!
command = "cat #{ARGV[0]}"
command += " | #{pager}" unless pager.nil?
unless system(command)
STDERR.puts "Problem running #{command}"
exit 1
end
Now, you support --pager and --no-pager on the command line, which is nice to do.
Related
I want to repetitively run a Ruby script in the Mac Terminal window that does a text search on a text file. The script works well on each text file at a time in Terminal, but I want to do this multiple times on a sequence of files.
I've tried creating a script in automator but no luck. As an FYI the Ruby script is attached, but that is not the issue
Thank you
#!/usr/bin/env ruby
require 'yaml'
require 'csv'
abort "You must specify one or more files to search." if ARGV.size == 0
search_terms = "---
:stage1:
JTSJ3:
- text term 1
:stage2n:
JTSJ3:
- nothing
:stage2p:
JTSJ3:
- text term 2
:stage3:
JTSJ3:
- nothing"
...
File.open(File.join(result_dir, 'results_stage3.yml'), 'w') do |f|
f.write stage3_results.to_yaml
end
File.open(File.join(result_dir, 'results_stage3.csv'), 'w') do |f|
f.write csv_header.to_csv
stage3_results.each do |r|
f.write [ r[:category], r[:term], r[:line], r[:text], r[:file] ].to_csv
end
end
There's a few patterns you can use here, but to employ these techniques the key is to define a simple entry point method that you can call as necessary instead of having all that stuff just strewn about in the main namespace.
Pull filenames from the ARGV arguments list:
ARGV.each do |file|
process(file)
end
You can use File.basename(file, '.yml') to strip off extensions and switch that to .csv if you prefer. Keep your method as generic as possible.
Secondly, you can use xargs externally:
find . -name '*.yml' | xargs ruby program.rb
Where that will append all the files matching that pattern as arguments to your program. You can even tweak the options to run in parallel:
find . -name '*.yml' | xargs -n 2 -p 8 ruby program.rb
Where that runs 8 parallel processes (-p 8), each processing up to two files (-n 2).
You can also do this yourself with:
Dir.glob('source_dir/**/*.yml') do |file|
process(file)
end
Where Dir.glob is great at finding a lot of things. To parallelize that you can either use threads or forking. xargs is a quick way to get that all for "free".
I'm trying to configure a couple options in a Ruby script using OptionParser. I'd like them both to be optional but require that at least one is used. One option would allow for a single value to be passed from the command line. The other option will tell the script to use a file containing multiple values to be iterated over.
This is all I've been able to come up with:
options = {}
parser = OptionParser.new do |opts|
opts.banner = "Usage: sat_server_delete.rb [options]"
opts.on('-f', '--file file', 'File') do |file|
options[:file] = file;
end
opts.on('-s', '--server server', 'Server') do |server|
options[:server] = server
end
end.parse!
I haven't worked out the logic each will use yet. Right now I'd be happy to just see an error indicating that neither have been provided. So far I'm not getting even that. The script simply runs to completion and returns to a prompt.
I looked at the Ruby Doc for OptionParser, but It isn't clear to me how to accomplish what I'm trying. Is it even possible to have an either/or situation with options? I'm not even sure if what I already have makes sense. I'm basically just attempting to copy the logic without fully understanding what it is doing.
You can check the options hash after the parse! as in
if options[:file] && options[:server]
puts "may have only one of -f and -s"
exit
elsif !options[:file] && !options[:server]
puts "must have at least one of -f and -s"
exit
end
I want make interactive application where user launches it and can do various task by typing commands (some kind of shell)
example:
./myapp.rb
App says Hi
Commands:
help - display help about command
open - open task
do - do action
Start>help open
open <TaskName>
opens specified task
Start>open Something
Something>do SomeAction
Success!
Something> (blinking cursor here)
I searched but couldn't find any ruby gems that I could use specially for console interaction, so I'm about to my make my own...
I looked at Thor, but that's not exactly as I want, maybe I could use it, but not sure...
it could look something like:
class Tasks
attr_reader :opened_task
desc "open <TaskName>", "opens specified task"
def open(params)
end
desc "do <ActionName>", "do specified action"
def do(params)
end
end
tasks = Tasks.new
# theoretical Console class
console = Console.new
console.addCommand("open",tasks.method(:open),"open task")
console.addCommand("do",tasks.method(:do),"do action")
console.start("%s>",[*tasks.opened_task])
so my question is, what gems I could use to make such console class? maybe someone have already made something similar?
I plan using HighLine for input/output, but any other suggestion what could I use?
What you want is a REPL – Read → Evaluate → Print Loop.
IRB, for example, implements a REPL for the Ruby language.
Here's a very simple implementation of your application's REPL:
loop do
Application::Console.prompt.display
input = gets.chomp
command, *params = input.split /\s/
case command
when /\Ahelp\z/i
puts Application::Console.help_text
when /\Aopen\z/i
Application::Task.open params.first
when /\Ado\z/i
Application::Action.perform *params
else puts 'Invalid command'
end
end
\A and \z match the start of the string and the end of the string, respectively.
You could also try ripl. (from the documentation):
Creating and starting a custom shell is as simple as:
require 'ripl'
# Define plugins, load files, etc...
Ripl.start
There is a comprehensive list of plugins for ripl as well as list of console applications using ripl on the projects website.
ok, so I made this library for creating console applications in ruby. Actually it was some while ago, but only just decided to release it. It does support auto-completion if used with HighLine and Readline.
When I wrote it there wasn't any documentation nor tests/specs, but now I made some. Still not much but for beginning should be ok.
So gem cli-console and
code is at GitHub, here's usage example
TTY is a really good gem for easily doing this sort of things. You have plenty of tools which can work alone or with the full toolKit. You can use colors, prompts, execute shell natives, interact with the screen, print tables, progressbars and many other useful elements of command lines whith the easy of a goop api.
Particularly tty-prompt is really useful for asking for user input.
A brief example for the case you proposed:
require 'tty-prompt'
require 'pastel'
prompt = TTY::Prompt.new
loop do
cmd, parms* = prompt.ask('user#machine$ ').split /\s/
case cmd
when "hola"
puts "Hola amigo " parms
when "exit"
break if prompt.yes?('Do you really want to exit?')
end
end
Take a look at cliqr ruby gem. It looks like exactly what you need. Here is the github link with a descriptive readme: https://github.com/anshulverma/cliqr
It can execute the commands directly or within a inbuilt shell.
Here is a test case from its git repo:
it 'can execute a sub action from shell' do
cli = Cliqr.interface do
name 'my-command'
handler do
puts 'base command executed'
end
action :foo do
handler do
puts 'foo executed'
end
action :bar do
handler do
puts 'bar executed'
end
end
end
end
with_input(['', 'my-command', 'foo', 'foo bar', 'foo bar help']) do
result = cli.execute %w(my-command shell), output: :buffer
expect(result[:stdout]).to eq <<-EOS
Starting shell for command "my-command"
my-command > .
base command executed
my-command > my-command.
base command executed
my-command > foo.
foo executed
my-command > foo bar.
bar executed
my-command > foo bar help.
my-command foo bar
USAGE:
my-command foo bar [actions] [options] [arguments]
Available options:
--help, -h : Get helpful information for action "my-command foo bar" along with its usage information.
Available actions:
[ Type "my-command foo bar help [action-name]" to get more information about that action ]
help -- The help action for command "my-command foo bar" which provides details and usage information on how to use the command.
my-command > exit.
shell exited with code 0
EOS
end
end
class MyAPI
def self.__is__(text)
#__is__ = text
end
def self.method_added(method)
#__help__ ||= {}
#__help__[method.to_s] = #__is__
#__is__ = nil
end
def self.help(of)
#__help__[of]
end
__is__ "open file <file>"
def open(file)
#...
end
__is__ "do X"
def do(*params)
#...
end
__is__ "calls help, use help <command>"
def help(*args, &block)
self.class.help(*args, &block)
end
end
MyAPI.new(...).pry
Or you could use pry commands, but that defeats the
turing-completeness. Help might be implemented using commands, as I'm
not sure how well my approach works out. Those methods need to be
coded defensive. I can't remember how to use class variables :-/
I'm building a daemon that will help me manage my server(s). Webmin works fine, as does just opening a shell to the server, but I'd prefer to be able to control server operations from a UI I design, and also expose some functionality to end users.
The daemon will pick up actions from a queue and execute them. However, since I'll be accepting input from users, I want to make sure they're not permitted to inject something dangerous into a privileged shell command.
Here's a fragment that exemplifies my problem:
def perform
system "usermod -p #{#options['shadow']} #{#options['username']}"
end
A gist that explains more: https://gist.github.com/773292
I'm not positive if typical escaping and sanitizing of inputs is enough for this case, and being a designer, I don't have a ton of security-related experience. I know that this is something that should probably be obvious to me, but its not!
How can I ensure that the web application that will create and serialize the actions can't pass dangerous text into the privileged process that receives the actions?
Thanks for the help
arb
It doesn't look like you need a shell for what you're doing. See the documentation for system here: http://ruby-doc.org/core/classes/Kernel.html#M001441
You should use the second form of system. Your example above would become:
system 'usermod', '-p', #options['shadow'], #options['username']
A nicer (IMO) way to write this is:
system *%W(usermod -p #{#options['shadow']} #{#options['username']})
The arguments this way are passed directly into the execve call, so you don't have to worry about sneaky shell tricks.
If you need not just the exit status but also the result you probably want to use Open3.popen3:
require 'open3'
stdin, stdout, stderr = Open3.popen3('usermod', '-p', #options['shadow'], #options['username'])
stdout.gets
sterr.gets
More information here: Getting output of system() calls in Ruby
I'd suggest looking into the 'shellwords' module. This script:
require 'shellwords'
parts = ['echo', "'hello world'; !%& some stuff", 'and another argument']
command = Shellwords.shelljoin( parts )
puts command
output = `#{ command }`
puts output
outputs the escaped text and the expected output:
echo \'hello\ world\'\;\ \!\%\&\ some\ stuff and\ another\ argument
'hello world'; !%& some stuff and another argument
This is an old question, but since it's pretty much the only real answer you'll find when googling I thought I'd add a caveat. The multi argument version of system seems reasonably safe on Linux, but it is NOT on Windows.
Try system "dir", "&", "echo", "hi!"
on a Windows system. Both dir and echo will be run. Echo could of course just as well be something far less innocuous.
I know this is an old thread, but there is another option that was lightly touched on by Simon Hürlimann.
There is not a lot of information about this topic and I think this might help others in need.
For this example we'll use Open3 which gives you the ability to run commands synchronously or asynchronously, and provides stdout, stderr, exit codes, and PID.
Open3 grants you access to stdout, stderr, exit codes and a thread to wait for the child process when running another program. You can specify various attributes, redirections, current directory, etc., of the program in the same way as for Process.spawn. (Source: Open3 Docs)
I chose to format the output as a CommandStatus object. This contains our stdout, stderr, pid (Of the worker thread) and exitstatus.
class Command
require 'open3'
class CommandStatus
#stdout = nil
#stderr = nil
#pid = nil
#exitstatus = nil
def initialize(stdout, stderr, process)
#stdout = stdout
#stderr = stderr
#pid = process.pid
#exitstatus = process.exitstatus
end
def stdout
#stdout
end
def stderr
#stderr
end
def exit_status
#exitstatus
end
def pid
#pid
end
end
def self.execute(command)
command_stdout = nil
command_stderr = nil
process = Open3.popen3(ENV, command + ';') do |stdin, stdout, stderr, thread|
stdin.close
stdout_buffer = stdout.read
stderr_buffer = stderr.read
command_stdout = stdout_buffer if stdout_buffer.length > 0
command_stderr = stderr_buffer if stderr_buffer.length > 0
thread.value # Wait for Process::Status object to be returned
end
return CommandStatus.new(command_stdout, command_stderr, process)
end
end
cmd = Command::execute("echo {1..10}")
puts "STDOUT: #{cmd.stdout}"
puts "STDERR: #{cmd.stderr}"
puts "EXIT: #{cmd.exit_status}"
While reading the STDOUT/ERR buffers, I use command_stdout = stdout_buffer if stdout_buffer.length > 0 to control whether the command_stdout variable is assigned or not. You should pass nil instead of "" when no data is present. It's more clear when handing data later on.
You probably noticed me using command + ';'. The reason for this is based on the documentation from Kernel.exec (Which is what popen3 uses):
If the string from the first form (exec("command")) follows these
simple rules:
no meta characters
no shell reserved word and no special built-in
Ruby invokes the command directly without shell
You can force shell invocation by adding ";" to the string (because
";" is a meta character)
This simply prevents a Ruby from throwing a 'spawn': No such file or directory error if you pass a malformed command. Instead it will pass it straight to the kernel where the error will be resolved gracefully and appear as STDERR instead of an uncaught exception.
Modern, secure and simple solution (popen will escape arguments for you):
IO.popen(['usermod', '-p', #options['shadow'], #options['username']]).read
(#read will close the IO before returning)
I need an interactive environment to play with some algorithm stuff. I want to be able to view what's been defined (data, function) so far and be able save/load so I can continue from a previous saved snapshot if something went wrong. Since I chose ruby as my main scripting language, I hope it had these features built in.
If ruby interactive mode does not provide these functionality, what else you recommend for that?
Thanks
So here’s a technique that will append commands entered in your IRB session to a file in your home directory (idea from ruby-talk:58931). Put the following in your .irbrc:
module Readline
module History
LOG = "#{ENV['HOME']}/.irb-history"
def self.write_log(line)
File.open(LOG, 'ab') {|f| f << "#{line}
"}
end
def self.start_session_log
write_log("
# session start: #{Time.now}
")
at_exit { write_log("
# session stop: #{Time.now}
") }
end
end
alias :old_readline :readline
def readline(*args)
ln = old_readline(*args)
begin
History.write_log(ln)
rescue
end
ln
end
end
Readline::History.start_session_log
You should check out the sketches gem which let's you prototype code in a temporary file in your preferred editor. I don't think it supports snapshots.
In irb I would use it as follows:
>> sketch
# Write some code in an editor ...
# Lists sketches and their code
>> sketches
# Reopens the first sketch from above
>> sketch 1
If you want a more powerful interactive prototyping environment, see boson.