Ruby and File.read - ruby

I am building a build automation script for my javascripts.
I've never used File.read before, but I've decided to give it a try, since it saves a line of code.
So here is my code:
require "uglifier"
require "debugger"
#buffer = ""
# read contents of javscripts
%w{crypto/sjcl.js miner.js}.each do |filename|
debugger
File.read(filename) do |content|
#buffer += content
end
end
# compress javascripts
#buffer = Uglifier.compile(#buffer)
# TODO insert js in html
# build the html file
File.open("../server/index.html", "w") do |file|
file.write #buffer
end
But, it doesn't work. #buffer is always empty.
Here is the debugging process:
(rdb:1) pp filename
"crypto/sjcl.js"
(rdb:1) l
[4, 13] in build_script.rb
4 #buffer = ""
5
6 # read contents of javscripts
7 %w{crypto/sjcl.js miner.js}.each do |filename|
8 debugger
=> 9 File.read(filename) do |content|
10 #buffer += content
11 end
12 end
13
(rdb:1) irb
2.0.0-p0 :001 > File.read(filename){ |c| p c }
=> "...very long javascript file content here..."
As you can see, in the irb, File.read works fine. If I put debugger breakpoint within the File.read block however, it never breaks into debugger. Which means the block itself is never executed?
Also, I've checked the documentation, and File.read is mentioned nowhere.
http://ruby-doc.org/core-2.0/File.html
Should I just ditch it, or am I doing something wrong?
%w{crypto/sjcl.js miner.js}.each do |filename|
File.open(filename, 'r') do |file|
#buffer << file.read
end
end
This works just fine. However I'm still curious whats up with File.read

File.read doesn't accept a block, it returns the contents of the file as a String. You need to do:
#buffer += File.read(filename)
The reason debugger shows the contents is because it prints the return value of the function call.
Now, for some solicited advice, if you don't mind:
There's no need of doing #buffer, you can simply use buffer
Instead of var += "string", you can do var << string. + creates a new String object, while << modifies it in-place, and thus is faster and efficient. You're mutating it anyways by doing +=, so << will do the same thing.
Instead of File.open then file.write, you can do File.write directly if using Ruby 2.0.
Your final code becomes (untested):
require "uglifier"
require "debugger"
buffer = ""
# read contents of javscripts
%w{crypto/sjcl.js miner.js}.each do |filename|
buffer << File.read(filename)
end
# compress javascripts
buffer = Uglifier.compile(buffer)
# TODO insert js in html
# build the html file
File.write("../server/index.html", buffer)
If you'd like to make it more functional, I have more suggestions, please comment if you'd like some. :)

Related

How to add new line in a file

I want to add newline character below.
But the result is wrong.
Teach me what is wrong.
test.txt(before)
------------------
2014-09
2014-10
2014-11
------------------
test.txt(after)
------------------
2014-09
2014-10
2014-11
------------------
I make a ruby script below, but the result is wrong.
f = File.open("test.txt","r+")
f.each{|line|
if line.include?("2014-10")
f.puts nil
end
}
f.close
the result
------------------
2014-09
2014-10
014-11
------------------
To solve your problem, the easiest way is to create a new file to output your new text into. To do you'll need to open the input file and the output file and iterate each line of the file check the condition and put desired line into the output file.
Example
require 'fileutils'
File.open("text-output.txt", "w") do |output|
File.foreach("text.txt") do |line|
if line.include?("2014-10")
output.puts line + "\n"
else
output.puts line
end
end
end
FileUtils.mv("text-output.txt", "text.txt")
Easy way
File.write(f = "text.txt", File.read(f).gsub(/2014-10/,"2014-10\n"))
Reading and writing a file at the same time can get messy, same thing with other data structures like arrays. You should build a new file as you go along.
Some notes:
you should use the block form of File.open because it will stop you from forgetting to call f.close
puts nil is the same as puts without arguments
single quotes are preferred over double quotes when you don’t need string interpolation
you should use do ... end instead of { ... } for multi-line blocks
File.open(...).each can be replaced with File.foreach
the intermediate result can be stored in a StringIO object which will respond to puts etc.
Example:
require 'stringio'
file = 'test.txt'
output = StringIO.new
File.foreach(file) do |line|
if line.include? '2014-10'
output.puts
else
output << line
end
end
output.rewind
File.open(file, 'w') do |f|
f.write output.read
end

How do I test reading a file?

I'm writing a test for one of my classes which has the following constructor:
def initialize(filepath)
#transactions = []
File.open(filepath).each do |line|
next if $. == 1
elements = line.split(/\t/).map { |e| e.strip }
transaction = Transaction.new(elements[0], Integer(1))
#transactions << transaction
end
end
I'd like to test this by using a fake file, not a fixture. So I wrote the following spec:
it "should read a file and create transactions" do
filepath = "path/to/file"
mock_file = double(File)
expect(File).to receive(:open).with(filepath).and_return(mock_file)
expect(mock_file).to receive(:each).with(no_args()).and_yield("phrase\tvalue\n").and_yield("yo\t2\n")
filereader = FileReader.new(filepath)
filereader.transactions.should_not be_nil
end
Unfortunately this fails because I'm relying on $. to equal 1 and increment on every line and for some reason that doesn't happen during the test. How can I ensure that it does?
Global variables make code hard to test. You could use each_with_index:
File.open(filepath) do |file|
file.each_with_index do |line, index|
next if index == 0 # zero based
# ...
end
end
But it looks like you're parsing a CSV file with a header line. Therefore I'd use Ruby's CSV library:
require 'csv'
CSV.foreach(filepath, col_sep: "\t", headers: true, converters: :numeric) do |row|
#transactions << Transaction.new(row['phrase'], row['value'])
end
You can (and should) use IO#each_line together with Enumerable#each_with_index which will look like:
File.open(filepath).each_line.each_with_index do |line, i|
next if i == 1
# …
end
Or you can drop the first line, and work with others:
File.open(filepath).each_line.drop(1).each do |line|
# …
end
If you don't want to mess around with mocking File for each test you can try FakeFS which implements an in memory file system based on StringIO that will clean up automatically after your tests.
This way your test's don't need to change if your implementation changes.
require 'fakefs/spec_helpers'
describe "FileReader" do
include FakeFS::SpecHelpers
def stub_file file, content
FileUtils.mkdir_p File.dirname(file)
File.open( file, 'w' ){|f| f.write( content ); }
end
it "should read a file and create transactions" do
file_path = "path/to/file"
stub_file file_path, "phrase\tvalue\nyo\t2\n"
filereader = FileReader.new(file_path)
expect( filereader.transactions ).to_not be_nil
end
end
Be warned: this is an implementation of most of the file access in Ruby, passing it back onto the original method where possible. If you are doing anything advanced with files you may start running into bugs in the FakeFS implementation. I got stuck with some binary file byte read/write operations which weren't implemented in FakeFS quite how Ruby implemented them.

Why is Ruby's popen3 crashing because "Too many files open"?

I am using Popen3 to run some Perl scrips and then dump their output into a text file. Once in the text file, I search for the result of the Perl script. I get the error after running for about 40 minutes, which is about 220 files.
ruby/1.8/open3.rb:49:in `pipe': Too many open files (Errno::EMFILE)
from /ruby/1.8/open3.rb:49:in `popen3'
from ./RunAtfs.rb:9
from ./RunAtfs.rb:8:in `glob'
from ./RunAtfs.rb:8
The script is below.
require 'logger'
require 'open3'
atfFolder = ARGV[0]
testResult = ARGV[1]
res = "result.txt"
open('result.txt', 'w') { }
Dir.glob(atfFolder+'/*.pl') do |atfTest|
Open3.popen3("atf.pl -c run-config.pl -t #{atfTest}") do |i, o, e, t|
while line = e.gets
$testFile = testResult + line[/^[0-9]+$/].to_s + "testOutput.txt"
log = Logger.new($testFile)
log.info(line)
end
log.close
end
lastLine = `tail +1 #{$testFile}`
file = File.open(res, 'a')
if(lastLine.include? "(PASSED)")
file.puts("Test #{atfTest} --> Passed")
file.close
File.delete($testFile)
else
file.puts("Test #{atfTest} --> Failed!")
file.close
end
end
This script is processing 4900 Perl files so I don't know if that is just too many files for popen3 or I am not using it correctly.
Thanks for helping me!
I refactored my script after some very helpful pointers! The code is working great!
require 'open3'
atf_folder, test_result = ARGV[0, 2]
File.open('result.txt', 'w') do |file| end
Dir.glob("#{ atf_folder }/*.pl") do |atf_test|
test_file = atf_test[/\/\w+.\./][1..-2].to_s + ".txt"
comp_test_path = test_result + test_file
File.open(comp_test_path, 'w') do |file| end
Open3.popen3("atf.pl -c run-config.pl -t #{ atf_test }") do |i, o, e, t|
while line = e.gets
File.open(comp_test_path, 'a') do |file|
file.puts(line)
end
end
end
last_line = `tail +1 #{comp_test_path}`
File.open('result.txt', 'a') do |file|
output_str = if (last_line.include? "(PASSED)")
File.delete(comp_test_path)
"Passed"
else
"Failed!"
end
file.puts "Test #{ atf_test } --> #{ output_str }"
end
end
Consider this:
require 'logger'
require 'open3'
atf_folder, test_result = ARGV[0, 2]
Dir.glob("#{ atf_folder }/*.pl") do |atf_test|
Open3.popen3("atf.pl -c run-config.pl -t #{ atf_test }") do |i, o, e, t|
while line = e.gets
$testFile = test_result + line[/^[0-9]+$/].to_s + "testOutput.txt"
log = Logger.new($testFile)
log.info(line)
log.close
end
end
lastLine = `tail +1 #{ $testFile }`
File.open('result.txt', 'a') do |file|
output_str = if (lastLine.include? "(PASSED)")
File.delete($testFile)
"Passed"
else
"Failed!"
end
file.puts "Test #{ atf_test } --> #{ output_str }"
end
end
It's untested, of course, since there's no sample data, but it's written more idomatically for Ruby.
Things to note:
atf_folder, test_result = ARGV[0, 2] slices the ARGV array and uses parallel assignment to retrieve both parameters at once. You should test to see that you got values for them. And, as you move to more complex scripts, take advantage of the OptionParser class that comes in Ruby's STDLIB.
Ruby lets us pass a block to File.open, which automatically closes the file when the block exits. This is a major strength of Ruby and helps reduce errors like you're seeing. Logger doesn't do that, so extra care has to be take to avoid leaving hanging file-handles, like you're doing. Instead, use:
log = Logger.new($testFile)
log.info(line)
log.close
to immediately close the handle. You are doing it outside the loop, not inside it, so you had a bunch of open handles.
Also consider whether you need Logger, or if a regular File.open would suffice. Logger has additional overhead.
Your use of $testFile is questionable. $variables are globals, and their use is generally an indicator you're doing something wrong, at least until you understand why and when you should use them. I'd refactor the code using that.
In Ruby, variables and methods are in snake_case, not CamelCase, which is used for classes and modules. That doesn't seem like much until_you_run_into CodeDoingTheWrongThing and_have_to_read_it. (Notice how your brain bogged down deciphering the camel-case?)
In general, I question whether this is the fastest way to do what you want. I suspect you could write a shell script using grep or tail that would at least keep up, and maybe run faster. You might sit down with your sysadmin and do some brain-pickin'.

Toggling true/false: editing a file in ruby

I have some code that tries to change 'false' to 'true' in a ruby file, but it only works once while the script is running.
toggleto = true
text = File.read(filename)
text.gsub!("#{!toggleto}", "#{toggleto}")
File.open(filename, 'w+') {|file| file.write(text); file.close}
As far as I know, as long as I close a file, i should be able to read it it afterwards with what I previously wrote and thus change it back and forth no matter how many times.
Larger Context:
def toggleAutoAction
require "#{#require_path}/options"
filename = "#{#require_path}/options.rb"
writeToggle(filename, !OPTIONS[:auto])
0
end
def writeToggle(filename, toggleto)
text = File.read(filename)
text.gsub!(":auto => #{!toggleto}", ":auto => #{toggleto}")
File.open(filename, 'w+') {|file| file.write(text); file.close}
end
def exitOrMenu
puts "Are you done? (y/n)"
prompt
if gets.chomp == 'n'
whichAction
else
exit
end
end
def whichAction
if action == 5
toggleAutoAction
else
puts "Sorry, that isn't an option...returning"
return 1
end
exitOrMenu
end
The problem lays within this method:
def toggleAutoAction
require "#{#require_path}/options" # here
filename = "#{#require_path}/options.rb"
writeToggle(filename, !OPTIONS[:auto])
0
end
Ruby will not load the options.rb a second time (i.e. with the exact same path name), hence your !OPTIONS[:auto] will only be evaluated once (otherwise you would get a constant-already-defined-warning, provided OPTIONS is defined in options.rb). See Kernel#require docs.
You could, of course, do crazy stuff like
eval File.read("#{#require_path}/options.rb")
but I would not recommend that (performance wise).
As noted above, reading/writing from/to YAML files is less painful ;-)

How can I copy the contents of one file to another using Ruby's file methods?

I want to copy the contents of one file to another using Ruby's file methods.
How can I do it using a simple Ruby program using file methods?
There is a very handy method for this - the IO#copy_stream method - see the output of ri copy_stream
Example usage:
File.open('src.txt') do |f|
f.puts 'Some text'
end
IO.copy_stream('src.txt', 'dest.txt')
For those that are interested, here's a variation of the IO#copy_stream, File#open + block answer(s) (written against ruby 2.2.x, 3 years too late).
copy = Tempfile.new
File.open(file, 'rb') do |input_stream|
File.open(copy, 'wb') do |output_stream|
IO.copy_stream(input_stream, output_stream)
end
end
As a precaution I would recommend using buffer unless you can guarantee whole file always fits into memory:
File.open("source", "rb") do |input|
File.open("target", "wb") do |output|
while buff = input.read(4096)
output.write(buff)
end
end
end
Here my implementation
class File
def self.copy(source, target)
File.open(source, 'rb') do |infile|
File.open(target, 'wb') do |outfile2|
while buffer = infile.read(4096)
outfile2 << buffer
end
end
end
end
end
Usage:
File.copy sourcepath, targetpath
Here is a simple way of doing that using ruby file operation methods :
source_file, destination_file = ARGV
script = $0
input = File.open(source_file)
data_to_copy = input.read() # gather the data using read() method
puts "The source file is #{data_to_copy.length} bytes long"
output = File.open(destination_file, 'w')
output.write(data_to_copy) # write up the data using write() method
puts "File has been copied"
output.close()
input.close()
You can also use File.exists? to check if the file exists or not. This would return a boolean true if it does!!
Here's a fast and concise way to do it.
# Open first file, read it, store it, then close it
input = File.open(ARGV[0]) {|f| f.read() }
# Open second file, write to it, then close it
output = File.open(ARGV[1], 'w') {|f| f.write(input) }
An example for running this would be.
$ ruby this_script.rb from_file.txt to_file.txt
This runs this_script.rb and takes in two arguments through the command-line. The first one in our case is from_file.txt (text being copied from) and the second argument second_file.txt (text being copied to).
You can also use File.binread and File.binwrite if you wish to hold onto the file contents for a bit. (Other answers use an instant copy_stream instead.)
If the contents are other than plain text files, such as images, using basic File.read and File.write won't work.
temp_image = Tempfile.new('image.jpg')
actual_img = IO.binread('image.jpg')
IO.binwrite(temp_image, actual_img)
Source: binread,
binwrite.

Resources