I am new to Ruby. I have used shell in the past. I am converting a shell program to ruby. I have the following command
cmd="cat -n " + infile + " | grep '127.0.0.1 '" + site
f = %x[#{cmd}]
The shell cat command returns the line I am looking for and the line number. I would like to be able to do something similar in ruby without using shell. May need to port this program to windows. I can use the grep function in ruby but how do I return the line number without having to loop through a million lines in a file. Thanks.
Here's a little ruby function that'll do what you're asking for. You do have to loop through each line, but grep was doing that too - you just didn't have to do it yourself. Fortunately, ruby makes looping easy:
def mygrep(filename, regex)
result = []
File.open(filename) do |f|
f.each_with_index do |l, i|
result << [i, l] if regex =~ l
end
end
return result
end
Cheers!
Related
Im triying to use this code but not work
ruby -a -F';' -ne if $F[2]<4 'puts $F[3]' ppp.txt
this is my file
mmm;2;nsfnjd
sadjjasjnsd;6;gdhjsd
gsduhdssdj;3;gsdhjhjsd
what is doing worng Please help me
First of all, instead of treating Ruby like some kind of fancy Perl and writing scripts like that, let's expand it into the Ruby code equivalent for clarity:
$; = ';'
while gets
$F = $_.split
if $F[2]<4
puts $F[3]
end
end
Your original code doesn't work, it can't possibly work because it's not valid Ruby code, and further, you're not properly quoting it to pass through the -e evaluation term. Trying to run it I get:
-bash: 4: No such file or directory
You're also presuming the array is 1-indexed, but it's not. It's 0-indexed. Additionally Ruby treats integer values as completely different from strings, never equivalent, not auto-converted. As such you need to call .to_i to convert.
Here's a re-written program that does the job:
File.open(ARGV[0]) do |fi|
fi.readlines.each do |line|
parts = line.chomp.split(';')
if parts[1].to_i < 4
puts parts[2]
end
end
end
I solved with this
ruby -a -F';' -ne ' if $F[1] < "4" ;puts $F[2] end ' ppp.txt
I'm using a Ruby script to do a lot of manipulation and cleaning to get this, and a bunch of other files, ready for import.
I have a really large file with some data that I'm trying to import into a database. There are some data issues with newline characters being in the data where they should not be, messing with the import.
I was able to solve this problem with sed using this:
sed -i '.original' -e ':a' -e 'N' -e '$!ba' -e 's/Oversight Bd\n/Oversight Bd/g' -e 's/Sciences\n/Sciences/g' combined_old_individual.txt"
However, I can't call that command from inside a Ruby script, because Ruby messes up interpreting the newline characters and won't run that command. sed needs the non-escaped newline character but when calling a system command from Ruby it needs a string, where the newline character needs to be escaped.
I also tried doing this using Ruby's file method, but it's not working either:
File.open("combined_old_individual.txt", "r") do |f|
File.open("combined_old_individual_new.txt","w") do |new_file|
to_combine = nil
f.each_line do |line|
if(/Oversight Bd$/ =~ line || /Sciences$/ =~ line)
to_combine = line
else
if to_combine.nil?
new_file.puts line
else
combined_line = to_combine + line
new_file.puts combined_line
to_combine = nil
end
end
end
end
end
Any ideas how I can join lines where the first line ends with "Bd" or "Sciences", from within a Ruby script, would be very helpful.
Here's an example of what might go in a testfile.txt:
random line
Oversight Bd
should be on the same line as the above, but isn't
last line
and the result should be
random line
Oversight Bdshould be on the same line as the above, but isn't
last line
With ruby (My first attempt at a ruby answer):
File.open("combined_old_individual.txt", "r") do |f|
File.open("combined_old_individual_new.txt","w") do |new_file|
f.each_line do |line|
if(/(Oversight Bd|Sciences)$/ =~ line)
new_file.print line.strip
else
new_file.puts line
end
end
end
end
You have to realize that sed normally works line by line, so you cannot match for \n in your initial pattern. You can however match for the pattern on the first line and then pull in the next line with the N command and then run the substitute command on the buffer to remove the newline like so:
sed -i -e '/Oversight Bd/ {;N;s/\n//;}' /your/file
Run from Ruby (without -i so that the output goes to stdout):
> cat test_text
aaa
bbb
ccc
aaa
bbb
ccc
> cat test.rb
cmd="sed -e '/aaa/ {;N;s/\\n//;}' test_text"
system(cmd)
> ruby test.rb
aaabbb
ccc
aaabbb
ccc
Since you are asking in bash, here is a pure-bash solution:
$ r="(Oversight Bd|Sciences)$"
$ while read -r; do printf "%s" "$REPLY"; [[ $REPLY =~ $r ]] || echo; done < combined_old_individual.txt
random line
Oversight Bdshould be on the same line as the above, but isn't
last line
$
In Bash, I want to compare the fields of 2 different CSVs (field 2 of file1 and field 3 of file2):
diff <(cut -d, -f2 file1) <(cut -d, -f3 file2)
I tried to implement this more generally in Ruby:
def identical_files?(file1, field1, file2, field2)
%x{diff <(cut -d, -f#{field1} #{file1}) <(cut -d, -f#{field2} #{file2})}.blank?
end
Printing the output of the %x{} block, I see sh: Syntax error: "(" unexpected. Does I/O redirection not work when running shell commands within Ruby? Is this because it's only supported by bash but not sh?
It doesn’t work because, as the error you’re getting indicates, Ruby shells out to sh, not Bash. And, of course, sh does not support that syntax.
You can instead call Bash explicitly:
`bash -c 'cat <(echo foo)'` #=> "foo"
Is this because it's only supported by bash but not sh?
Yes.
Process substitution is not supported by sh, even when sh is actually bash (for compatibility).
Don't try to use something as simple as cut to process fields in a CSV file. CSV files can have embedded commas inside fields, which will fool cut, causing your code to do the wrong thing.
Instead, use something designed specifically to process CSV files, such as Ruby's CSV class. Something like this untested code will get you started:
require 'csv'
csv_file1 = CSV.open('file1')
csv_file2 = CSV.open('file2')
until (csv_file1.eof? || csv_file2.eof?) do
row1 = csv_file1.shift
row2 = csv_file2.shift
# do something to diff the fields
puts "#{ csv_file1.lineno }: #{ row1[1] } == #{ row2[2] } --> #{ row1[1] == row2[2] }"
end
[
[csv_file1, 'file1'],
[csv_file2, 'file2']
].each do |f, fn|
puts "Hit EOF for #{ fn }" if f.eof?
f.close
end
I have Ruby code similar to:
ok.rb
hasil = "input operator salah"
puts hasil
exec("sort ok.rb > output.txt") if fork.nil?
It just wrote all code into output.txt. However, I only want hasil result to be written to output.txt. How should I modify the code for such an end result?
You've executed the sort command taking ok.rb as the input. Instead, you want to run ok.rb and take its output as the input to sort.
Without knowing Ruby, I'd expect this to be something like:
exec("ruby ok.rb | sort > output.txt") if fork.nil?
I've just tried this from my Linux desktop and it worked fine:
ok.rb:
hasil = "input operator salah"
puts hasil
other.rb:
exec("ruby ok.rb | sort > output.txt") if fork.nil?
Execute:
$ ruby other.rb
$ cat output.txt
input operator salah
(You've only provided a single line of output so there wasn't exactly a lot to sort.)
The cleanest way would be to change the preceding code to not produce output to stdout directly, but instead to only build the string and then sort this from ruby and print it to the file. Like this for example:
hasil = "input operator salah"
File.open("output.txt", "w") do |f|
f.puts hasil.split("\n").sort.join("\n")
end
If replacing unix sort with ruby sort is not an option (maybe because sort was just an example and in reality you're piping to a different application that can't easily be replaced with ruby), you can write your code to the application directly instead of writing to stdout. You could even write your code to be general enough to write to any IO.
def generate_output(out)
hasil = "input operator salah"
out.puts hasil
end
# If you decide to output the text directly to stdout (without sorting)
generate_output(stdout)
# If you instead want to pipe to sort:
IO.popen("sort > output.txt", "w") do |sort|
generate_output(sort)
end
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
The community reviewed whether to reopen this question last month and left it closed:
Original close reason(s) were not resolved
Improve this question
I want to deal with the command line input in Ruby:
> cat input.txt | myprog.rb
> myprog.rb < input.txt
> myprog.rb arg1 arg2 arg3 ...
What is the best way to do it? In particular I want to deal with blank STDIN, and I hope for an elegant solution.
#!/usr/bin/env ruby
STDIN.read.split("\n").each do |a|
puts a
end
ARGV.each do |b|
puts b
end
Following are some things I found in my collection of obscure Ruby.
So, in Ruby, a simple no-bells implementation of the Unix command
cat would be:
#!/usr/bin/env ruby
puts ARGF.read
— https://web.archive.org/web/20080725055721/http://www.oreillynet.com/ruby/blog/2007/04/trivial_scripting_with_ruby.html#comment-565558
ARGF is your friend when it comes to input; it is a virtual file that gets all input from named files or all from STDIN.
ARGF.each_with_index do |line, idx|
print ARGF.filename, ":", idx, ";", line
end
# print all the lines in every file passed via command line that contains login
ARGF.each do |line|
puts line if line =~ /login/
end
Thank goodness we didn’t get the diamond operator in Ruby, but we did
get ARGF as a replacement. Though obscure, it actually turns out to
be useful. Consider this program, which prepends copyright headers
in-place (thanks to another Perlism, -i) to every file mentioned on
the command-line:
#!/usr/bin/env ruby -i
Header = DATA.read
ARGF.each_line do |e|
puts Header if ARGF.pos - e.length == 0
puts e
end
__END__
#--
# Copyright (C) 2007 Fancypants, Inc.
#++
— http://blog.nicksieger.com/articles/2007/10/06/obscure-and-ugly-perlisms-in-ruby
Credit to:
https://web.archive.org/web/20080725055721/http://www.oreillynet.com/ruby/blog/2007/04/trivial_scripting_with_ruby.html#comment-565558
http://blog.nicksieger.com/articles/2007/10/06/obscure-and-ugly-perlisms-in-ruby
Ruby provides another way to handle STDIN: The -n flag. It treats your entire program as being inside a loop over STDIN, (including files passed as command line args). See e.g. the following 1-line script:
#!/usr/bin/env ruby -n
#example.rb
puts "hello: #{$_}" #prepend 'hello:' to each line from STDIN
#these will all work:
# ./example.rb < input.txt
# cat input.txt | ./example.rb
# ./example.rb input.txt
I am not quite sure what you need, but I would use something like this:
#!/usr/bin/env ruby
until ARGV.empty? do
puts "From arguments: #{ARGV.shift}"
end
while a = gets
puts "From stdin: #{a}"
end
Note that because ARGV array is empty before first gets, Ruby won't try to interpret argument as text file from which to read (behaviour inherited from Perl).
If stdin is empty or there is no arguments, nothing is printed.
Few test cases:
$ cat input.txt | ./myprog.rb
From stdin: line 1
From stdin: line 2
$ ./myprog.rb arg1 arg2 arg3
From arguments: arg1
From arguments: arg2
From arguments: arg3
hi!
From stdin: hi!
Something like this perhaps?
#/usr/bin/env ruby
if $stdin.tty?
ARGV.each do |file|
puts "do something with this file: #{file}"
end
else
$stdin.each_line do |line|
puts "do something with this line: #{line}"
end
end
Example:
> cat input.txt | ./myprog.rb
do something with this line: this
do something with this line: is
do something with this line: a
do something with this line: test
> ./myprog.rb < input.txt
do something with this line: this
do something with this line: is
do something with this line: a
do something with this line: test
> ./myprog.rb arg1 arg2 arg3
do something with this file: arg1
do something with this file: arg2
do something with this file: arg3
while STDIN.gets
puts $_
end
while ARGF.gets
puts $_
end
This is inspired by Perl:
while(<STDIN>){
print "$_\n"
}
Quick and simple:
STDIN.gets.chomp == 'YES'
You can also use STDIN.each_line, and STDIN.each_line.to_a to get it as an array.
e.g.
STDIN.each_line do |line|
puts line
end
I'll add that in order to use ARGF with parameters, you need to clear ARGV before calling ARGF.each. This is because ARGF will treat anything in ARGV as a filename and read lines from there first.
Here's an example 'tee' implementation:
File.open(ARGV[0], 'w') do |file|
ARGV.clear
ARGF.each do |line|
puts line
file.write(line)
end
end
I do something like this :
all_lines = ""
ARGV.each do |line|
all_lines << line + "\n"
end
puts all_lines
It seems most answers are assuming the arguments are filenames containing content to be cat'd to the stdin. Below everything is treated as just arguments. If STDIN is from the TTY, then it is ignored.
$ cat tstarg.rb
while a=(ARGV.shift or (!STDIN.tty? and STDIN.gets) )
puts a
end
Either arguments or stdin can be empty or have data.
$ cat numbers
1
2
3
4
5
$ ./tstarg.rb a b c < numbers
a
b
c
1
2
3
4
5