I am trying to read in a JSON file with Ruby and the output is extremely strange. Here is the code that I am using:
require 'rubygems'
class ServiceCalls
def initialize ()
end
def getFile()
Dir.entries('./json').each do |mFile|
if mFile[0,1] != "."
self.sendServiceRequest(mFile)
end
end
end
def sendServiceRequest(mFile)
currentFile = File.new("./json/" + mFile, "r")
puts currentFile.read
currentFile.close
end
end
mServiceCalls = ServiceCalls.new
mServiceCalls.getFile
And here is the output:
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094AA9DAB6D??/??is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
bookmark88?A[DT>??A?#
ApplicationsMAMPhtdocsServiceTestAutomationMDXservicecatalog-verizon.json$4T??
`?
U?????l??????
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094?is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
UIEvolutions-MacBook-Pro-109:MDXServiceTesting Banderson$ ruby testmdxservices.rb
bookmark88?A?,P>??A?#
ApplicationsMAMPhtdocsServiceTestAutomationMDXservicecatalog-adaptation.json$4T??
`?
U?????l??????
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094AA9DAB6D??/?<icns<?TOC his32?s8mic118il32?l8mic1?ic07ic13#ic08#ic14^?ic09_ic1?is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
IHDR szz?iCCPICC Profile(?T?k?P??e???:g >h?ndStC??kW??Z?6?!H??m\??$?~?ًo:?w?>?
كo{?a???"L?"???4M'S??????9'??^??qZ?/USO???????^C+?hM??J&G#Ӳy???lt?o߫?c՚?
? ??5?"?Y?i\??'&??.?<?ER/?dE?oc?ግ#?f45#? ??B:K?#8?i??
??s??_???雭??m?N?|??9}p?????_?A??pX6?5~B?$?&???ti??e??Y)%$?bT?3li?
??????P???4?43Y???P??1???KF????ۑ??5>?)?#????r??y??????[?:V???ͦ#??wQ?HB??d(??B
a?cĪ?L"J??itTy?8?;(???Gx?_?^?[???????%??ŷ??Q???麲?ua??n?7???
Q???H^e?O?Q?u6?S??u
?2??%vX
???^?*l
O?????ޭˀq,>??S???%?L??d????B???1CZ??$M??9??P
'w????\/????]???.r#???E|!?3?>_?o?a?۾?d?1Z?ӑ???z???'?=??????~+??cjJ?tO%mN?????
|??-???bW?O+
o?
^?
I?H?.?;???S?]?i_s9?*p???.7U^??s.?3u?
Can someone please tell me what I am doing wrong? Do I need to specify what type of encoding I'm using? I have tried to read the file with gets, sysread, and another I can't remember.
I am not completely sure why but I believe it is the './json' path that is causing the issue. I tried the script on my Windows XP machine and got similar results.
However, when I rewrote the script to include File.dirname(__FILE__) instead of './' it worked. I also cleaned up some of the code.
class ServiceCalls
def get_file
dirname = File.join(File.dirname(__FILE__), 'json')
Dir.entries(dirname).each do |file|
unless file.start_with? '.'
File.open(File.join(dirname, file), 'r') {|f| puts f.read}
end
end
end
end
sc = ServiceCalls.new
sc.get_file
__FILE__ is the path of the current script. File.join uses system independent path separators. File.open, if you pass it a block, will actually close the file for you when it completes the block. String#start_with? is a cleaner way than using [0,1] to get the first element of a string.
try this:
Dir.entries('./json').each do |mFile|
next if ['.', '..'].include?(mFile)
self.sendServiceRequest(mFile)
Related
I am doing the EventManager tutorial from Jumpstart Labs. Originally I could not get it my .rb file to read a .erb file, and I think I may have solved that, but I am not sure as I do not know what I SHOULD be seeing if all is running correctly and unfortunately the tutorial doesn't tell you. Here is my original question
Now after a simple change, I am no longer getting the error - but I am also not getting any indication that the code is working as expected. The tutorial says that this code should be creating a new directory and storing a copy of each 'thank you' letter into a file called 'output' in that new directory. But when I run it all I see is the => EventManager initialized from the terminal, which tells me that my .rb is being read and (I think) that the .erb is finally being read...but I do not see any new directories/files in the file structure, nor any indication that anything was created - so I can't tell if it is actually doing anything.
I am kind of expecting to see some kind of message telling me the directory has been created, perhaps with a file path or something.
I have never done anything like this and I'm not sure what I should be seeing...can anyone tell me how I would know that this code is preforming as expected? And if it is not, why?
require "csv"
require "sunlight/congress"
require "erb"
Sunlight::Congress.api_key = "e179a6973728c4dd3fb1204283aaccb5"
def save_thank_you_letters(id, form_letter)
Dir.mkdir("output") unless Dir.exists? ("output")
filename = "output/thanks_#{id}.html"
File.open(filename, 'w') do |file|
file.puts form_letter
end
end
def legislators_by_zipcode(zipcode)
legislators = Sunlight::Congress::Legislator.by_zipcode(zipcode)
end
def clean_zipcode(zipcode)
zipcode.to_s.rjust(5,"0")[0..4]
end
puts "EventManager initialized."
contents = CSV.open "event_attendees.csv", headers: true, header_converters: :symbol
template_letter = File.read( "event_manager/form_letter.erb")
erb_template = ERB.new template_letter
contents.each do |row|
id = row[0]
name = row[:first_name]
zipcode = clean_zipcode(row[:zipcode])
legislators = legislators_by_zipcode(zipcode)
form_letter = erb_template.result(binding)
save_thank_you_letters(id, form_letter)
end
I've (ever so slightly) modified your save_thank_you_letters method to spit out some helpful information as it writes files:
def save_thank_you_letters(id, form_letter)
Dir.mkdir("output") unless Dir.exists? ("output")
filename = "output/thanks_#{id}.html"
File.open(filename, 'w') do |file|
file.puts form_letter
puts "Wrote ID: #{id} to #{filename}"
end
end
The line puts "Wrote ID: #{id} to #{filename}" will print the ID and file path of the message it has written. You can place additional puts "Your text here..." throughout your Ruby logic to print more information to the console as you see fit.
Side note: in general, it's a super bad idea to post your personal API keys to any public forums. If this key is private/unique to you, delete it and request a new one. Anyone can now impersonate your account at Sunlight Labs.
I'm writing a test for one of my classes which has the following constructor:
def initialize(filepath)
#transactions = []
File.open(filepath).each do |line|
next if $. == 1
elements = line.split(/\t/).map { |e| e.strip }
transaction = Transaction.new(elements[0], Integer(1))
#transactions << transaction
end
end
I'd like to test this by using a fake file, not a fixture. So I wrote the following spec:
it "should read a file and create transactions" do
filepath = "path/to/file"
mock_file = double(File)
expect(File).to receive(:open).with(filepath).and_return(mock_file)
expect(mock_file).to receive(:each).with(no_args()).and_yield("phrase\tvalue\n").and_yield("yo\t2\n")
filereader = FileReader.new(filepath)
filereader.transactions.should_not be_nil
end
Unfortunately this fails because I'm relying on $. to equal 1 and increment on every line and for some reason that doesn't happen during the test. How can I ensure that it does?
Global variables make code hard to test. You could use each_with_index:
File.open(filepath) do |file|
file.each_with_index do |line, index|
next if index == 0 # zero based
# ...
end
end
But it looks like you're parsing a CSV file with a header line. Therefore I'd use Ruby's CSV library:
require 'csv'
CSV.foreach(filepath, col_sep: "\t", headers: true, converters: :numeric) do |row|
#transactions << Transaction.new(row['phrase'], row['value'])
end
You can (and should) use IO#each_line together with Enumerable#each_with_index which will look like:
File.open(filepath).each_line.each_with_index do |line, i|
next if i == 1
# …
end
Or you can drop the first line, and work with others:
File.open(filepath).each_line.drop(1).each do |line|
# …
end
If you don't want to mess around with mocking File for each test you can try FakeFS which implements an in memory file system based on StringIO that will clean up automatically after your tests.
This way your test's don't need to change if your implementation changes.
require 'fakefs/spec_helpers'
describe "FileReader" do
include FakeFS::SpecHelpers
def stub_file file, content
FileUtils.mkdir_p File.dirname(file)
File.open( file, 'w' ){|f| f.write( content ); }
end
it "should read a file and create transactions" do
file_path = "path/to/file"
stub_file file_path, "phrase\tvalue\nyo\t2\n"
filereader = FileReader.new(file_path)
expect( filereader.transactions ).to_not be_nil
end
end
Be warned: this is an implementation of most of the file access in Ruby, passing it back onto the original method where possible. If you are doing anything advanced with files you may start running into bugs in the FakeFS implementation. I got stuck with some binary file byte read/write operations which weren't implemented in FakeFS quite how Ruby implemented them.
I have a gem that has code like this inside:
def read(file)
#file = File.new file, "r"
end
Now the problem is, say you have a directory structure like so:
app/main.rb
app/templates/example.txt
and main.rb has the following code:
require 'mygem'
example = MyGem.read('templates/example.txt')
It comes up with File Not Found: templates/example.txt. It would work if example.txt was in the same directory as main.rb but not if it's in a directory. To solve this problem I've added an optional argument called relative_to in read(). This takes an absolute path so the above could would need to be:
require 'mygem'
example = MyGem.read('templates/example.txt', File.dirname(__FILE__))
That works fine, but I think it's a bit ugly. Is there anyway to make it so the class knows what file read() is being called in and works out the path based on that?
There is an interesting library - i told you it was private. One can protect their methods with it from being called from outside. The code finds the caller method's file and removes it. The offender is found using this line:
offender = caller[0].split(':')[0]
I guess you can use it in your MyGem.read code:
def read( file )
fpath = Pathname.new(file)
if fpath.relative?
offender = caller[0].split(':')[0]
fpath = File.join( File.dirname( offender ), file )
end
#file = File.new( fpath, "r" )
end
This way you can use paths, relative to your Mygem caller and not pwd. Exactly the way you tried in your app/main.rb
Well, you can use caller, and a lot more reliably than what the other people said too.
In your gem file, outside of any class or module, put this:
c = caller
req_file = nil
c.each do |s|
if(s =~ /(require|require_relative)/)
req_file = File.dirname(File.expand_path(s.split(':')[0])) #Does not work for filepaths with colons!
break
end
end
REQUIRING_FILE_PATH = req_file
This will work 90% of the time, unless the requiring script executed a Dir.chdir. The File.expand_path depends on that. I'm afraid that unless your requirer passes their __FILE__, there's nothing you can do if they change the working dir.
Also you may check for caller:
def read(file)
if /^(?<file>.+?):.*?/ =~ caller(1).first
caller_dir, caller_file = Pathname.new(Regexp.last_match[:file]).split
file_with_path = File.join caller_dir, file
#file = File.new "#{file_with_path}", "r"
end
end
I would not suggest you to do so (the code above will break being called indirectly, because of caller(1), see reference to documentation on caller). Furthermore, the regex above should be tuned more accurately if the caller path is intended to contain colons.
This should work for typical uses (I'm not sure how resistant it is to indirect use, as mentioned by madusobwa above):
def read_relative(file)
#file = File.new File.join(File.dirname(caller.first), file)
end
On a side note, consider adding a block form of your method that closes the file after yielding. In the current form you're forcing clients to wrap their use of your gem with an ensure.
Accept a file path String as an argument. Convert to a Pathname object. Check if the path is relative. If yes, then convert to absolute.
def read(file)
fpath = Pathname.new(file)
if fpath.relative?
fpath = File.expand_path(File.join(File.dirname(__FILE__),file))
end
#file = File.new(fpath,"r")
end
You can make this code more succinct (less verbose).
I want to copy the contents of one file to another using Ruby's file methods.
How can I do it using a simple Ruby program using file methods?
There is a very handy method for this - the IO#copy_stream method - see the output of ri copy_stream
Example usage:
File.open('src.txt') do |f|
f.puts 'Some text'
end
IO.copy_stream('src.txt', 'dest.txt')
For those that are interested, here's a variation of the IO#copy_stream, File#open + block answer(s) (written against ruby 2.2.x, 3 years too late).
copy = Tempfile.new
File.open(file, 'rb') do |input_stream|
File.open(copy, 'wb') do |output_stream|
IO.copy_stream(input_stream, output_stream)
end
end
As a precaution I would recommend using buffer unless you can guarantee whole file always fits into memory:
File.open("source", "rb") do |input|
File.open("target", "wb") do |output|
while buff = input.read(4096)
output.write(buff)
end
end
end
Here my implementation
class File
def self.copy(source, target)
File.open(source, 'rb') do |infile|
File.open(target, 'wb') do |outfile2|
while buffer = infile.read(4096)
outfile2 << buffer
end
end
end
end
end
Usage:
File.copy sourcepath, targetpath
Here is a simple way of doing that using ruby file operation methods :
source_file, destination_file = ARGV
script = $0
input = File.open(source_file)
data_to_copy = input.read() # gather the data using read() method
puts "The source file is #{data_to_copy.length} bytes long"
output = File.open(destination_file, 'w')
output.write(data_to_copy) # write up the data using write() method
puts "File has been copied"
output.close()
input.close()
You can also use File.exists? to check if the file exists or not. This would return a boolean true if it does!!
Here's a fast and concise way to do it.
# Open first file, read it, store it, then close it
input = File.open(ARGV[0]) {|f| f.read() }
# Open second file, write to it, then close it
output = File.open(ARGV[1], 'w') {|f| f.write(input) }
An example for running this would be.
$ ruby this_script.rb from_file.txt to_file.txt
This runs this_script.rb and takes in two arguments through the command-line. The first one in our case is from_file.txt (text being copied from) and the second argument second_file.txt (text being copied to).
You can also use File.binread and File.binwrite if you wish to hold onto the file contents for a bit. (Other answers use an instant copy_stream instead.)
If the contents are other than plain text files, such as images, using basic File.read and File.write won't work.
temp_image = Tempfile.new('image.jpg')
actual_img = IO.binread('image.jpg')
IO.binwrite(temp_image, actual_img)
Source: binread,
binwrite.
In my app, I have the following code:
File.open "filename", "w" do |file|
file.write("text")
end
I want to test this code via RSpec. What are the best practices for doing this?
I would suggest using StringIO for this and making sure your SUT accepts a stream to write to instead of a filename. That way, different files or outputs can be used (more reusable), including the string IO (good for testing)
So in your test code (assuming your SUT instance is sutObject and the serializer is named writeStuffTo:
testIO = StringIO.new
sutObject.writeStuffTo testIO
testIO.string.should == "Hello, world!"
String IO behaves like an open file. So if the code already can work with a File object, it will work with StringIO.
For very simple i/o, you can just mock File. So, given:
def foo
File.open "filename", "w" do |file|
file.write("text")
end
end
then:
describe "foo" do
it "should create 'filename' and put 'text' in it" do
file = mock('file')
File.should_receive(:open).with("filename", "w").and_yield(file)
file.should_receive(:write).with("text")
foo
end
end
However, this approach falls flat in the presence of multiple reads/writes: simple refactorings which do not change the final state of the file can cause the test to break. In that case (and possibly in any case) you should prefer #Danny Staple's answer.
This is how to mock File (with rspec 3.4), so you could write to a buffer and check its content later:
it 'How to mock File.open for write with rspec 3.4' do
#buffer = StringIO.new()
#filename = "somefile.txt"
#content = "the content fo the file"
allow(File).to receive(:open).with(#filename,'w').and_yield( #buffer )
# call the function that writes to the file
File.open(#filename, 'w') {|f| f.write(#content)}
# reading the buffer and checking its content.
expect(#buffer.string).to eq(#content)
end
You can use fakefs.
It stubs filesystem and creates files in memory
You check with
File.exists? "filename"
if file was created.
You can also just read it with
File.open
and run expectation on its contents.
For someone like me who need to modify multiple files in multiple directories (e.g. generator for Rails), I use temp folder.
Dir.mktmpdir do |dir|
Dir.chdir(dir) do
# Generate a clean Rails folder
Rails::Generators::AppGenerator.start ['foo', '--skip-bundle']
File.open(File.join(dir, 'foo.txt'), 'w') {|f| f.write("write your stuff here") }
expect(File.exist?(File.join(dir, 'foo.txt'))).to eq(true)
end
end