I wrote this little test script in Ruby (on MacOS):
#!/usr/bin/ruby
require 'digest/sha2'
def calc_sha(file)
# calc hash
hash = Digest::SHA2.new
File.open(file, 'r') do |fh|
fh.each_line do |l|
hash << l
end
end
hash.to_s
end
puts calc_sha('dont-panic.jpeg')
puts '40075d8441ab6a9abeceb7039552704320f471667b8f9ac3c18b9b5b0a1fee7e'
puts calc_sha('dont-panic.jpeg') == '40075d8441ab6a9abeceb7039552704320f471667b8f9ac3c18b9b5b0a1fee7e'
Which outputs (on MacOS):
~/shatest $ ./sha.rb
40075d8441ab6a9abeceb7039552704320f471667b8f9ac3c18b9b5b0a1fee7e
40075d8441ab6a9abeceb7039552704320f471667b8f9ac3c18b9b5b0a1fee7e
true
Then I run the exact same Script in Windows XP:
F:\shatest>ruby sha.rb
9c787b71392551238b24915c888dbd44f4ff465c8e8aadca7af3bb6aaf66a3ca
40075d8441ab6a9abeceb7039552704320f471667b8f9ac3c18b9b5b0a1fee7e
false
Can anyone tell me whats the Problem here?
You're opening a JPEG (i.e. a binary file) and then reading every line of text from it. Don't do that. Any time you treat binary data as text, you're just asking for odd behaviour.
I don't know much about Ruby at all, but I'd generally expect to open the file, and repeatedly read chunks of binary data from it, updating the hash with that. Don't do anything which talks about "lines" or uses text at all.
Related
I'm trying to read from a file, and because my more complex code doesn't work, I came back to basics to see whether it even reads properly.
My code:
MyParser.new(STDIN.read).run.lines.each do |line|
p line.chomp
end
I use
ruby program
(it's placed in a bin directory and I saved it without .rb )
Now program is waiting for me to write something. I type:
../examples/file.txt
and use CTRL + Z (I'm on Windows 10). It produces ^Z and I hit enter.
Now I have an error:
Invalid argument # rb_sysopen - ../examples/file.txt (Errno::EINVAL)
MyParser class and its whole logic works fine. I'll be grateful for any hints.
Without knowing, what your MyParser expect, it is hard to know, what you need.
But maybe this helps:
MyParser.new(STDIN.gets.strip).run.lines.each do |line|
p line.chomp
end
I would extend it by a message what you need:
puts "Please enter the filename"
STDOUT.flush
MyParser.new(STDIN.gets.strip).run.lines.each do |line|
p line.chomp
end
With STDOUT.flush the user gets the message, before STDIN.getswaits for a message.
In your case I would take a look on ARGV and call the program with:
ruby program ../examples/file.txt
Your program should then use:
MyParser.new(ARGV.first).run.lines.each do |line|
p line.chomp
end
actually i'm writing a ruby script which accesses an API based on HTTP-POST calls.
The API returns a zip file containing textdocuments when i call it with specific POST-Parameters. At the moment i'm doing that with the Net::HTTP Package.
Now my problem:
It seems to return the zip-file as a string as far as i know. I can see "PK" (which i suppose is part of the PK-Header of zip-files) and the text from the documents.
And the Content-Type Header is telling me "application/x-zip-compressed; name="somename.zip"".
When i save the zip file like so:
result = comodo.get_cert("<somenumber>")
puts result['Content-Type']
puts result.inspect
puts result.body
File.open("test.zip", "w") do |file|
file.write result.body
end
I can unzip it on my macbook without further problems. But when i run the same code on my Win10 PC it tells me that the file is corrupt or not a ZIP-file.
Has it something to do with the encoding? Can i change it, so it's working on both?
Or is it a complete wrong approach on how to recieve a zip-file from a POST-request?
PS:
My ruby-version on Mac:
ruby 2.2.3p173
My ruby-version on Windows:
ruby 2.2.4p230
Many thanks in advance!
The problem is due to the way Windows handles line endings (\r\n for Windows, whereas OS X and other Unix based operating systems use just \n). When using File.open, using the mode of just w makes the file subject to line ending changes, so any occurrences of byte 0x0A (or \n) are converted into bytes 0x0D 0x0A (or \r\n), which effectively breaks the zip.
When opening the file for write, use the mode wb instead, as this will suppress any line ending changes.
http://ruby-doc.org/core-2.2.0/IO.html#method-c-new-label-IO+Open+Mode
Many thanks! Just as you posted the solution i found it out myself..
So much trouble because of one missing 'b' :/
Thank you very much!
The solution (see Ben Y's answer):
result = comodo.get_cert("<somenumber>")
puts result['Content-Type']
puts result.inspect
puts result.body
File.open("test.zip", "wb") do |file|
file.write result.body
end
I'm still learning ruby, so I'm sure I'm doing something wrong here, but using ruby 1.9.3 on windows, I'm having a problem writing a file with random ascii garbage to be a specific size. I need to be able to write these files for a test on an application I'm QAing. On Mac and on *nix, the file size is written correctly every time. But on windows, it generates files of random size, generally between 1,024 bytes and 1,031 bytes.
I'm sure the problem is one of the characters that the rstr is generating is counting as two characters but... it seems like this shouldn't happen.
Here is my code:
num = 10
k = 1
for i in 1..num
fname = "f#{i}.txt"
f = File.new(fname, "w")
for k in 1..size
rstr = "#{(1..1024).map{rand(255).chr}.join}"
f.write rstr
print " #{rstr.size} " # this returns 1024 every time.
rstr = ""
end
f.close
end
Also tried:
opts = {}
opts[:encoding] = "UTF-8"
fname = "f#{i}.txt"
f = File.new(fname, "w", opts)
By default files open in Windows are open with text mode meaning that line endings and other details are adjusted.
If you want the files be written byte-to-byte exactly as you want, you need to open the files in binary mode:
File.new("foo", "wb") do |f|
# ...
end
The b is a ignored on POSIX operating systems, so your scripts are now cross-platform compatible.
Note: I used block syntax to manage the file so it properly closes and disposes the file handler once the block is executed. You no longer need to worry about closing the file ;-)
Hope this helps.
There is not any 255 ASCII. The values goes from 0~254.
If you try to printf 255.chr, you'll get a multibyte character.
As Windows does not standard utf-8, you'll get incorrect values. Hence the problem you're facing!
Try adding #coding: utf-8 at the top of your file. It should get things working.
Is there a Gem or means of securely erasing a file in Ruby? I'd like to avoid external programs that may not be present on the system.
By "secure erase" I'm referring to overwriting the file contents.
If you are on *nix, a pretty good way would be to just call shred using exec/open3/open4:
`shred -fxuz #{filename}`
http://www.gnu.org/s/coreutils/manual/html_node/shred-invocation.html
Check this similar post:
Writing a file shredder in python or ruby?
Something like this will get you started:
#!/usr/bin/env ruby
abort "Missing filename" if (ARGV.empty?)
ARGV.each do |filename|
filesize = File.size(filename)
[0x00, 0xff].each do |byte|
File.open(filename, 'wb') do |fo|
filesize.times { fo.print(byte.chr) }
end
end
end
It should get you close.
For more thoroughness, you could also use 0xaa and 0x55 for alternating 0 and 1 bits in the byte. Random.rand(0xff) will give you a random value from 0 to 255.
just
open the file
write some garbage at least in amount equal to current file size
flush() and close()
repeat N times, mixing garbage with zeroes and 0xff's on different passes
I need an easy way to take a tar file and convert it into a string (and vice versa). Is there a way to do this in Ruby? My best attempt was this:
file = File.open("path-to-file.tar.gz")
contents = ""
file.each {|line|
contents << line
}
I thought that would be enough to convert it to a string, but then when I try to write it back out like this...
newFile = File.open("test.tar.gz", "w")
newFile.write(contents)
It isn't the same file. Doing ls -l shows the files are of different sizes, although they are pretty close (and opening the file reveals most of the contents intact). Is there a small mistake I'm making or an entirely different (but workable) way to accomplish this?
First, you should open the file as a binary file. Then you can read the entire file in, in one command.
file = File.open("path-to-file.tar.gz", "rb")
contents = file.read
That will get you the entire file in a string.
After that, you probably want to file.close. If you don’t do that, file won’t be closed until it is garbage-collected, so it would be a slight waste of system resources while it is open.
If you need binary mode, you'll need to do it the hard way:
s = File.open(filename, 'rb') { |f| f.read }
If not, shorter and sweeter is:
s = IO.read(filename)
To avoid leaving the file open, it is best to pass a block to File.open. This way, the file will be closed after the block executes.
contents = File.open('path-to-file.tar.gz', 'rb') { |f| f.read }
how about some open/close safety.
string = File.open('file.txt', 'rb') { |file| file.read }
Ruby have binary reading
data = IO.binread(path/filaname)
or if less than Ruby 1.9.2
data = IO.read(path/file)
on os x these are the same for me... could this maybe be extra "\r" in windows?
in any case you may be better of with:
contents = File.read("e.tgz")
newFile = File.open("ee.tgz", "w")
newFile.write(contents)
You can probably encode the tar file in Base64. Base 64 will give you a pure ASCII representation of the file that you can store in a plain text file. Then you can retrieve the tar file by decoding the text back.
You do something like:
require 'base64'
file_contents = Base64.encode64(tar_file_data)
Have look at the Base64 Rubydocs to get a better idea.
Ruby 1.9+ has IO.binread (see #bardzo's answer) and also supports passing the encoding as an option to IO.read:
Ruby 1.9
data = File.read(name, {:encoding => 'BINARY'})
Ruby 2+
data = File.read(name, encoding: 'BINARY')
(Note in both cases that 'BINARY' is an alias for 'ASCII-8BIT'.)
If you can encode the tar file by Base64 (and storing it in a plain text file) you can use
File.open("my_tar.txt").each {|line| puts line}
or
File.new("name_file.txt", "r").each {|line| puts line}
to print each (text) line in the cmd.