I'm enjoying a very interesting problem where File.write... only works occasionally, i.e., seemingly when it's at the end of the method. For some weird reason this works:
def update(id)
r = HTTParty.get("#{user_api_url}/#{id}?token=#{token}").parsed_response
file_path = "/Users/#{server_user}/server/resources/users/"
File.write("#{file_path}#{id}#{extension(r['user_file_name'])}", open("#{r['user_url']}").read, { mode: 'wb' })
File.write("#{file_path}#{id}_logo#{extension(r['logo_file_name'])}", open("#{r['user_logo_url']}").read, { mode: 'wb' })
end
And this doesn't:
def update(id)
r = HTTParty.get("#{user_api_url}/#{id}?token=#{token}").parsed_response
file_path = "/Users/#{server_user}/server/resources/users/"
r['shared_resources'].map do |key, value|
file = "#{file_path}shared_resources/#{value.split('/')[-1]}"
p "#{timestamp}: Saving #{key} from #{api_server}#{value} into #{file}"
File.write(file, open("#{api_server}#{value}").read, mode: 'wb')
end
File.write("#{file_path}#{id}#{extension(r['user_file_name'])}", open("#{r['user_url']}").read, mode: 'wb')
File.write("#{file_path}#{id}_logo#{extension(r['logo_file_name'])}", open("#{r['user_logo_url']}").read, mode: 'wb')
end
The second method doesn't generate any errors at all, but no files are written. The paths and the URL are both correct. Makes me think I'm not opening or closing something correctly but I don't know what. Any ideas?
UPDATE
Getting this error:
Errno::ENOENT: No such file or directory # rb_sysopen - /Users/user123/server/resources/users/shared_resources/cqap_logo_cmyk-6a82ebd2e336c92188a58cacf26792cf9f43b6d296ae51c2b3fe...
Which is from File.write in the loop.
UPDATE 2
The r['shared_resources'] outputs this:
{\"logo\"=>\"/assets/server/user123-6a82ebd2e336c92188a58cacf26792cf9f43b6d296ae51c2b3fe05a0c1802794.jpg\"}"
But nothing in the loop seems to do anything.
It looks like shared_resources folder does not exist. First snippet writes directly in existing file_path, while second one tries to write into subfolder. Put the following right after you have file_path defined:
Dir.mkdir File.join file_path, 'shared_resources' # unless exists?
Related
I'm writing a test for one of my classes which has the following constructor:
def initialize(filepath)
#transactions = []
File.open(filepath).each do |line|
next if $. == 1
elements = line.split(/\t/).map { |e| e.strip }
transaction = Transaction.new(elements[0], Integer(1))
#transactions << transaction
end
end
I'd like to test this by using a fake file, not a fixture. So I wrote the following spec:
it "should read a file and create transactions" do
filepath = "path/to/file"
mock_file = double(File)
expect(File).to receive(:open).with(filepath).and_return(mock_file)
expect(mock_file).to receive(:each).with(no_args()).and_yield("phrase\tvalue\n").and_yield("yo\t2\n")
filereader = FileReader.new(filepath)
filereader.transactions.should_not be_nil
end
Unfortunately this fails because I'm relying on $. to equal 1 and increment on every line and for some reason that doesn't happen during the test. How can I ensure that it does?
Global variables make code hard to test. You could use each_with_index:
File.open(filepath) do |file|
file.each_with_index do |line, index|
next if index == 0 # zero based
# ...
end
end
But it looks like you're parsing a CSV file with a header line. Therefore I'd use Ruby's CSV library:
require 'csv'
CSV.foreach(filepath, col_sep: "\t", headers: true, converters: :numeric) do |row|
#transactions << Transaction.new(row['phrase'], row['value'])
end
You can (and should) use IO#each_line together with Enumerable#each_with_index which will look like:
File.open(filepath).each_line.each_with_index do |line, i|
next if i == 1
# …
end
Or you can drop the first line, and work with others:
File.open(filepath).each_line.drop(1).each do |line|
# …
end
If you don't want to mess around with mocking File for each test you can try FakeFS which implements an in memory file system based on StringIO that will clean up automatically after your tests.
This way your test's don't need to change if your implementation changes.
require 'fakefs/spec_helpers'
describe "FileReader" do
include FakeFS::SpecHelpers
def stub_file file, content
FileUtils.mkdir_p File.dirname(file)
File.open( file, 'w' ){|f| f.write( content ); }
end
it "should read a file and create transactions" do
file_path = "path/to/file"
stub_file file_path, "phrase\tvalue\nyo\t2\n"
filereader = FileReader.new(file_path)
expect( filereader.transactions ).to_not be_nil
end
end
Be warned: this is an implementation of most of the file access in Ruby, passing it back onto the original method where possible. If you are doing anything advanced with files you may start running into bugs in the FakeFS implementation. I got stuck with some binary file byte read/write operations which weren't implemented in FakeFS quite how Ruby implemented them.
Ruby is giving me this error:
C:/Ruby/new.rb:11:in `read': No such file or directory - m.txt (Errno::ENOENT)
from C:/Ruby/new.rb:11:in `<main>'
But I'm sure that there is such file, Here is my code:
text = File.read("m.txt").split('\n')
text.each do |x|
x.to_i
File.open("m.txt", "w") do |file|
file.gsub(x, x *10)
end
end
The line that is generating this error:
text = File.read("m.txt").split('\n')
I have checked several examples, like this: How can I read a file with Ruby?
And tried things like:
File.open("m.txt", "r+") do |infile|
while (line = infile.gets)
line.to_i.gsub(line, line *10)
end
end
But I'm still getting this error.
What I'm trying to do is: I have some numbers in text file like
12.2
432.3
3.43
.342
...
And I want to multiply each one by 10. Note I'm sure about the file and that it exists.
You have to provide the absolute path:
text = File.read("C:/Ruby/m.txt").split('\n')
since your current directory is not the same as your script's directory.
Alternatively, you should navigate to that specific folder and then run the script.
You can do it this way:
text = File.read("C:/Ruby/m.txt").split('\n')
File.open("C:/Ruby/m.txt", "w") do |file|
text.each do |x|
file.puts x.to_f * 10
end
end
I am trying to read in a JSON file with Ruby and the output is extremely strange. Here is the code that I am using:
require 'rubygems'
class ServiceCalls
def initialize ()
end
def getFile()
Dir.entries('./json').each do |mFile|
if mFile[0,1] != "."
self.sendServiceRequest(mFile)
end
end
end
def sendServiceRequest(mFile)
currentFile = File.new("./json/" + mFile, "r")
puts currentFile.read
currentFile.close
end
end
mServiceCalls = ServiceCalls.new
mServiceCalls.getFile
And here is the output:
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094AA9DAB6D??/??is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
bookmark88?A[DT>??A?#
ApplicationsMAMPhtdocsServiceTestAutomationMDXservicecatalog-verizon.json$4T??
`?
U?????l??????
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094?is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
UIEvolutions-MacBook-Pro-109:MDXServiceTesting Banderson$ ruby testmdxservices.rb
bookmark88?A?,P>??A?#
ApplicationsMAMPhtdocsServiceTestAutomationMDXservicecatalog-adaptation.json$4T??
`?
U?????l??????
Macintosh H??=A?v?P$66267945-2481-3907-B88A-1094AA9DAB6D??/?<icns<?TOC his32?s8mic118il32?l8mic1?ic07ic13#ic08#ic14^?ic09_ic1?is32???????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz?????????????????????????????????????????????????????????????????????????????????????????????vvz???????????????????????????????????????????????????????????s8m+88888888???????89????????99?????????9:??????????:;??????????;=??????????=>??????????>????????????#??????????#A??????????AC??????????CD??????????DE??????????EE??????????E6OXdknnkdXO6ic118?PNG
IHDR szz?iCCPICC Profile(?T?k?P??e???:g >h?ndStC??kW??Z?6?!H??m\??$?~?ًo:?w?>?
كo{?a???"L?"???4M'S??????9'??^??qZ?/USO???????^C+?hM??J&G#Ӳy???lt?o߫?c՚?
? ??5?"?Y?i\??'&??.?<?ER/?dE?oc?ግ#?f45#? ??B:K?#8?i??
??s??_???雭??m?N?|??9}p?????_?A??pX6?5~B?$?&???ti??e??Y)%$?bT?3li?
??????P???4?43Y???P??1???KF????ۑ??5>?)?#????r??y??????[?:V???ͦ#??wQ?HB??d(??B
a?cĪ?L"J??itTy?8?;(???Gx?_?^?[???????%??ŷ??Q???麲?ua??n?7???
Q???H^e?O?Q?u6?S??u
?2??%vX
???^?*l
O?????ޭˀq,>??S???%?L??d????B???1CZ??$M??9??P
'w????\/????]???.r#???E|!?3?>_?o?a?۾?d?1Z?ӑ???z???'?=??????~+??cjJ?tO%mN?????
|??-???bW?O+
o?
^?
I?H?.?;???S?]?i_s9?*p???.7U^??s.?3u?
Can someone please tell me what I am doing wrong? Do I need to specify what type of encoding I'm using? I have tried to read the file with gets, sysread, and another I can't remember.
I am not completely sure why but I believe it is the './json' path that is causing the issue. I tried the script on my Windows XP machine and got similar results.
However, when I rewrote the script to include File.dirname(__FILE__) instead of './' it worked. I also cleaned up some of the code.
class ServiceCalls
def get_file
dirname = File.join(File.dirname(__FILE__), 'json')
Dir.entries(dirname).each do |file|
unless file.start_with? '.'
File.open(File.join(dirname, file), 'r') {|f| puts f.read}
end
end
end
end
sc = ServiceCalls.new
sc.get_file
__FILE__ is the path of the current script. File.join uses system independent path separators. File.open, if you pass it a block, will actually close the file for you when it completes the block. String#start_with? is a cleaner way than using [0,1] to get the first element of a string.
try this:
Dir.entries('./json').each do |mFile|
next if ['.', '..'].include?(mFile)
self.sendServiceRequest(mFile)
post '/upload' do
unless params[:file] && (tmpfile = params[:file][:tempfile]) && (name = params[:file][:filename])
return haml(:upload)
end
time = Time.now.to_s
time.gsub!(/\s/, '')
name = time + name
while blk = tmpfile.read(65536)
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") { |f| f.write(tmpfile.read) }
end
'success'
end
Everything goes where expected the files just end up being corrupted.
This bit looks really funky:
while blk = tmpfile.read(65536)
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") { |f| f.write(tmpfile.read) }
end
I'm guessing you're trying to read your tempfile a 65536-byte block at a time, and then write those blocks successively to your destination file. But you never write blk, which is the first block you read; you write the rest of the file (tempfile.read) instead. And even if this loop did write blocks like it should, it opens the file anew for each block, overwriting the old contents! Anyway, I suspect you meant something like this:
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") do |f|
while(blk = tempfile.read(65536))
f.write(blk)
end
end
That said, if you've got the file as a temp file (presumably already on your local file system), maybe all you need to do is move that file? It'll go way faster if that's the case - if the source and destination are on the same disk, it's just a matter of swapping some file system pointers, rather than copying all that data.
Hope that helps!
The code opens and replaces the file during every iteration of the loop, which causes part of the problem. The code also reads the tmpfile into blk then throws that data away. Time.now.to_s contains colons, which is the path separator on Mac OS X, and could cause a problem on OS X. The user-supplied filename could contain some bad stuff like .. which may allow users to overwrite files. Try this instead:
require 'pathname'
require 'zaru'
post '/upload' do
unless tmpfile = params[:file].try(:[], :tempfile)
return haml(:upload)
end
name = Zaru.sanitize!("#{Time.now.to_i}#{params[:file][:filename]}")
Pathname.pwd.join("public/uploads", name).open("wb") do |f|
while blk = tmpfile.read(65536)
f.write(blk)
end
end
'success'
end
You should also make sure that the filename doesn't end in something nefarious, like .js or .css, which could be exploited.
I want to copy the contents of one file to another using Ruby's file methods.
How can I do it using a simple Ruby program using file methods?
There is a very handy method for this - the IO#copy_stream method - see the output of ri copy_stream
Example usage:
File.open('src.txt') do |f|
f.puts 'Some text'
end
IO.copy_stream('src.txt', 'dest.txt')
For those that are interested, here's a variation of the IO#copy_stream, File#open + block answer(s) (written against ruby 2.2.x, 3 years too late).
copy = Tempfile.new
File.open(file, 'rb') do |input_stream|
File.open(copy, 'wb') do |output_stream|
IO.copy_stream(input_stream, output_stream)
end
end
As a precaution I would recommend using buffer unless you can guarantee whole file always fits into memory:
File.open("source", "rb") do |input|
File.open("target", "wb") do |output|
while buff = input.read(4096)
output.write(buff)
end
end
end
Here my implementation
class File
def self.copy(source, target)
File.open(source, 'rb') do |infile|
File.open(target, 'wb') do |outfile2|
while buffer = infile.read(4096)
outfile2 << buffer
end
end
end
end
end
Usage:
File.copy sourcepath, targetpath
Here is a simple way of doing that using ruby file operation methods :
source_file, destination_file = ARGV
script = $0
input = File.open(source_file)
data_to_copy = input.read() # gather the data using read() method
puts "The source file is #{data_to_copy.length} bytes long"
output = File.open(destination_file, 'w')
output.write(data_to_copy) # write up the data using write() method
puts "File has been copied"
output.close()
input.close()
You can also use File.exists? to check if the file exists or not. This would return a boolean true if it does!!
Here's a fast and concise way to do it.
# Open first file, read it, store it, then close it
input = File.open(ARGV[0]) {|f| f.read() }
# Open second file, write to it, then close it
output = File.open(ARGV[1], 'w') {|f| f.write(input) }
An example for running this would be.
$ ruby this_script.rb from_file.txt to_file.txt
This runs this_script.rb and takes in two arguments through the command-line. The first one in our case is from_file.txt (text being copied from) and the second argument second_file.txt (text being copied to).
You can also use File.binread and File.binwrite if you wish to hold onto the file contents for a bit. (Other answers use an instant copy_stream instead.)
If the contents are other than plain text files, such as images, using basic File.read and File.write won't work.
temp_image = Tempfile.new('image.jpg')
actual_img = IO.binread('image.jpg')
IO.binwrite(temp_image, actual_img)
Source: binread,
binwrite.