I want to repetitively run a Ruby script in the Mac Terminal window that does a text search on a text file. The script works well on each text file at a time in Terminal, but I want to do this multiple times on a sequence of files.
I've tried creating a script in automator but no luck. As an FYI the Ruby script is attached, but that is not the issue
Thank you
#!/usr/bin/env ruby
require 'yaml'
require 'csv'
abort "You must specify one or more files to search." if ARGV.size == 0
search_terms = "---
:stage1:
JTSJ3:
- text term 1
:stage2n:
JTSJ3:
- nothing
:stage2p:
JTSJ3:
- text term 2
:stage3:
JTSJ3:
- nothing"
...
File.open(File.join(result_dir, 'results_stage3.yml'), 'w') do |f|
f.write stage3_results.to_yaml
end
File.open(File.join(result_dir, 'results_stage3.csv'), 'w') do |f|
f.write csv_header.to_csv
stage3_results.each do |r|
f.write [ r[:category], r[:term], r[:line], r[:text], r[:file] ].to_csv
end
end
There's a few patterns you can use here, but to employ these techniques the key is to define a simple entry point method that you can call as necessary instead of having all that stuff just strewn about in the main namespace.
Pull filenames from the ARGV arguments list:
ARGV.each do |file|
process(file)
end
You can use File.basename(file, '.yml') to strip off extensions and switch that to .csv if you prefer. Keep your method as generic as possible.
Secondly, you can use xargs externally:
find . -name '*.yml' | xargs ruby program.rb
Where that will append all the files matching that pattern as arguments to your program. You can even tweak the options to run in parallel:
find . -name '*.yml' | xargs -n 2 -p 8 ruby program.rb
Where that runs 8 parallel processes (-p 8), each processing up to two files (-n 2).
You can also do this yourself with:
Dir.glob('source_dir/**/*.yml') do |file|
process(file)
end
Where Dir.glob is great at finding a lot of things. To parallelize that you can either use threads or forking. xargs is a quick way to get that all for "free".
Related
I want to write a script that does some work on some files specified on a command line, but I want the user to be able to call
fooscript *
even if * expands to include some directory names
I know I can use ARGF to handle the files
ARGF.each do |f|
puts f
end
But this gives me an error for the case of a directory. I'd like to skip over directoryies, or perhaps handle them specially. What's the most idiomatic way to accomplish this is Ruby?
You could just filter them from ARGV before reading ARGF :
ARGV.reject! {|f| File.directory? f }
# Now there are no dirs in ARGF...
ARGF.each do |l|
#... etc.
I want a ruby script that will dump all the existing cron jobs to a text file using "crontab -l" or anything else that will achieve the same objective. Also the text file should be possible to use with crontab txtfile to create the cron jobs again.
Below is the code I already wrote:
def dump_pre_cron_jobs(file_path)
begin
cron_list = %x[crontab -l]
if(cron_list.size > 0)
cron_list.each do |crl|
mymethod_that_writes_tofile(file_path, crl) unless crl.chomp.include?("myfilter")
end
end
rescue Exception => e
raise(e.message)
end
end
Why does this need to be a Ruby script?
As you say, you can dump the crontab to a file with crontab -l > crontab.txt.
To read them back in again, simply use crontab crontab.txt, or cat crontab.txt | crontab -
I agree with #Vortura that you do not need to create a Ruby script to do this.
If you really want to, here is a probable way:
File.open('crontab.txt', 'w') do |crontab|
crontab << `crontab -l`
end
NOTE: Running this as root, or using sudo should capture all the cron jobs on a system, not just a single users' jobs. Run it as yourself or as that user and it might capture just those jobs. I haven't test that aspect of it.
Trying to run crontab -l to capture crontab files for all the users and packages seems the indirect way to do the task and could have the hassle of dealing with password requests hanging your code. I'd write code to comb through the directories that store them, rather than mess with prompts. Run the code using sudo and you shouldn't have any problems accessing the files.
Take a look at the discussion at: http://www.linuxquestions.org/questions/linux-newbie-8/etc-crontab-vs-etc-cron-d-vs-var-spool-cron-crontabs-853881/ for information on where the actual cron tab files are stored on disk.
Also https://superuser.com/questions/389116/how-to-recover-crontab-jobs-from-filesystem/389137 has similar information.
Mac OS varies a little from Linux in where Apple puts the cron files. Run man cron at the command-line for the definitive details on either OS.
Here's slightly-tested code for how I'd back up the files. How you restore them is for you to figure out, but it shouldn't be hard to figure out:
require 'fileutils'
BACKUP_PATH = '/path/to/some/safe/storage/directory'
CRONTAB_DIRS = %w[
/usr/lib/cron/tabs
/var/spool/cron
/etc/anacrontab
/etc/cron.d
]
CRONTAB_FILES = %w[
/etc/cron_list
]
def dump_pre_cron_jobs(file_path)
full_backup_path = File.join(
BACKUP_PATH,
File.dirname(file_path)
)
FileUtils.mkdir_p(full_backup_path) unless Dir.exist?(full_backup_path)
File.write(
File.join(
full_backup_path,
file_path
),
File.read(file_path)
)
rescue Exception => e
STDERR.puts e.message
end
CRONTAB_DIRS.each do |ct|
next unless Dir.exist?(ct)
begin
Dir.entries(File.join(ct, '*')).each { |fn| dump_pre_cron_jobs(fn) }
rescue Errno::EACCES => e
STDERR.puts e.message
end
end
CRONTAB_FILES.each do |fn|
dump_pre_cron_jobs(fn)
end
You'll need to run this as root via sudo to access the directories and files as they're usually locked down from unauthorized prying eyes.
The code creates a repository of crontabs, in BACKUP_PATH, based on their original file paths. No changes are made to the file contents so they can be restored as-is by copying them back via cp or writing code to reverse this process.
I'm trying to write a program which substitutes a string.
require File.join(APP_ROOT, 'lib', 'main.rb')
files_names = Dir.entries("../DeSpacer")
files_names.each do |file_name|
File.open("#{file_name}", "w") do |text|
text.each {|line| line.gsub!(/\.\s{2,}/, "\.\s")}
end
end
I keep getting a
Permission denied -. (ERRNO::EACCES)
Can you explain what I am doing wrong?
The initial problem is that you're only opening the file for writing ('w'), and not reading, and thus receiving the exception.
As the comments above mention, there are other issues with the code as well.
This answer gives a more typical way to do what you're trying to do.
As mentioned in another answer to the same question, Ruby also has a command line shortcut inherited from Perl which makes things like this trivial:
ruby -pi.bak -e "gsub(/oldtext/, 'newtext')" *.txt
This will edit a file or files in place, backing up the previous version with a suffix of '.bak'.
From Programming Ruby:
-i [extension}
' Edits ARGV files in place. For each file named in ARGV, anything you write to
standard output will be saved back as the contents of that file. A backup copy of
the file will be made if extension is supplied.
% ruby -pi.bak -e "gsub(/Perl/, 'Ruby')" *.txt
I'm making a command line tool using Ruby. It will print a lot of text on screen. Currently, I'm using shell pipeline (may_app | more) to do so. But I think it's better to has a default pager.
It's just like what you see when execute git log . One can disable pager by using git --nopager log.
I've done quite much google work and find one gem: hirb , but it seems a little overkill.
After many tries, I'm current using shell wrapper to do so:
#!/bin/bash
# xray.rb is the core script
# doing the main logic and will
# output many rows of text on
# screen
XRAY=$HOME/fdev-xray/xray.rb
if [ "--nopager" == "$1" ]; then
shift
$XRAY $*
else
$XRAY $* | more
fi
It works. But is there a better way?
You are doing it right. But instead using more you'd better get a pager from $PAGER environment variable, if any.
Some people prefer less to more for example, and others have their favorite parser options set in this var.
You can use the pipe in Ruby via a call to system and provide the options (along with a nice help interface) like so:
require 'optparse'
pager = ENV['PAGER'] || 'more'
option_parser = OptionParser.new do |opts|
opts.on("--[no-]pager",
"[don't] page output using #{pager} (default on)") do |use_pager|
pager = nil unless use_pager
end
end
option_parser.parse!
command = "cat #{ARGV[0]}"
command += " | #{pager}" unless pager.nil?
unless system(command)
STDERR.puts "Problem running #{command}"
exit 1
end
Now, you support --pager and --no-pager on the command line, which is nice to do.
I want to be able to read a currently open file. The test.rb is sending its output to test.log which I want to be able to read and ultimately send via email.
I am running this using cron:
*/5 * * * /tmp/test.rb > /tmp/log/test.log 2>&1
I have something like this in test.rb:
#!/usr/bin/ruby
def read_file(file_name)
file = File.open(file_name, "r")
data = file.read
file.close
return data
end
puts "Start"
puts read_file("/tmp/log/test.log")
puts "End"
When I run this code, it only gives me this output:
Start
End
I would expect the output to be something like this:
Start
Start (from the reading of the test.log since it should have the word start already)
End
Ok, you're trying to do several things at once, and I suspect you didn't systematically test before moving from one step to the next.
First we're going to clean up your code:
def read_file(file_name)
file = File.open(file_name, "r")
data = file.read
file.close
return data
end
puts "Start"
puts read_file("/tmp/log/test.log")
puts "End"
can be replaced with:
puts "Start"
puts File.read("./test.log")
puts "End"
It's plain and simple; There's no need for a method or anything complicated... yet.
Note that for ease of testing I'm working with a file in the current directory. To put some content in it I'll simply do:
echo "foo" > ./test.log
Running the test code gives me...
Greg:Desktop greg$ ruby test.rb
Start
foo
End
so I know the code is reading and printing correctly.
Now we can test what would go into the crontab, before we deal with its madness:
Greg:Desktop greg$ ruby test.rb > ./test.log
Greg:Desktop greg$
Hmm. No output. Something is broken with that. We knew there was content in the file previously, so what happened?
Greg:Desktop greg$ cat ./test.log
Start
End
Cat'ing the file shows it has the "Start" and "End" output of the code, but the part that should have been read and output is now missing.
What happening is that the shell truncated "test.log" just before it passed control to Ruby, which then opened and executed the code, which opened the now empty file to print it. In other words, you're asking the shell to truncate (empty) it just before you read it.
The fix is to read from a different file than you're going to write to, if you're trying to do something with the contents of it. If you're not trying to do something with its contents then there's no point in reading it with Ruby just to write it to a different file: We have cp and/or mv to do those things for us witout Ruby being involved. So, this makes more sense if we're going to do something with the contents:
ruby test.rb > ./test.log.out
I'll reset the file contents using echo "foo" > ./test.log, and cat'ing it showed 'foo', so I'm ready to try the redirection test again:
Greg:Desktop greg$ ruby test.rb > ./test.log.out
Greg:Desktop greg$ cat test.log.out
Start
foo
End
That time it worked. Trying it again has the same result, so I won't show the results here.
If you're going to email the file you could add that code at this point. Replacing the puts in the puts File.read('./test.log') line with an assignment to a variable will store the file's content:
contents = File.read('./test.log')
Then you can use contents as the body of a email. (And, rather than use Ruby for all of this I'd probably do it using mail or mailx or pipe it directly to sendmail, using the command-line and shell, but that's your call.)
At this point things are in a good position to add the command to crontab, using the same command as used on the command-line. Because it's running in cron, and errors can happen that we'd want to know about, we'd add the 2>&1 redirect to capture STDERR also, just as you did before. Just remember that you can NOT write to the same file you're going to read from or you'll have an empty file to read.
That's enough to get your app working.
class FileLineRead
File.open("file_line_read.txt") do |file|
file.each do |line|
phone_number = line.gsub(/\n/,'')
user = User.find_by_phone_number(line)
user.destroy unless user.nil?
end
end
end
open file
read line
DB Select
DB Update
In the cron job you have already opened and cleared test.log (via redirection) before you have read it in the Ruby script.
Why not do both the read and write in Ruby?
It may be a permissions issue or the file may not exist.
f = File.open("test","r")
puts f.read()
f.close()
The above will read the file test. If the file exists in the current directory
The problem is, as I can see, already solved by Slomojo. I'll only add:
to read and print a text file in Ruby, just:
puts File.read("/tmp/log/test.log")