How do I parse a PNG in string format (like below) into a file in Ruby?
\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x01,\x00\x00\
Adding more details I left in the comments.
If the PNG existed on the file system, I could open the file with File.open. I want the same object File.open would create, but I need to create it from the string, not the file system.
Ultimately, I want to assign this to a Paperclip attachment and have it recognize the object as a png.
File is just an implementation of IO. Ruby has another IO implementation that can read/write strings called, obviously, StringIO.
file = StringIO.new("\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x01,\x00\x00\")
Your comment suggests you need this to work with paperclip. In that case paperclip will usually (depends on version) want a file name and mime type, so add them before assigning the file to your paperclip attachment attribute.
file.content_type = "image/png"
file.original_filename = "image.png"
object.attachment = file
The above works for the most recent paperclip. Still better than writing to a temp file.
You can use StringIO:
s = "\x89PNG\r\n..."
file = StringIO.new(s)
Alternatively, you can use Tempfile (if you want real file object):
require 'tempfile'
file = Tempfile.new('png')
file.write "\x89PNG\r\n..."
file.rewind # move position pointer to the beginning of the file
Related
I'm trying to save and load the states of Matrices (using Matrix) during the execution of my program with the functions dump and load from Marshal. I can serialize the matrix and get a ~275 KB file, but when I try to load it back as a string to deserialize it into an object, Ruby gives me only the beginning of it.
# when I want to save
mat_dump = Marshal.dump(#mat) # serialize object - OK
File.open('mat_save', 'w') {|f| f.write(mat_dump)} # write String to file - OK
# somewhere else in the code
mat_dump = File.read('mat_save') # read String from file - only reads like 5%
#mat = Marshal.load(mat_dump) # deserialize object - "ArgumentError: marshal data too short"
I tried to change the arguments for load but didn't find anything yet that doesn't cause an error.
How can I load the entire file into memory? If I could read the file chunk by chunk, then loop to store it in the String and then deserialize, it would work too. The file has basically one big line so I can't even say I'll read it line by line, the problem stays the same.
I saw some questions about the topic:
"Ruby serialize array and deserialize back"
"What's a reasonable way to read an entire text file as a single string?"
"How to read whole file in Ruby?"
but none of them seem to have the answers I'm looking for.
Marshal is a binary format, so you need to read and write in binary mode. The easiest way is to use IO.binread/write.
...
IO.binwrite('mat_save', mat_dump)
...
mat_dump = IO.binread('mat_save')
#mat = Marshal.load(mat_dump)
Remember that Marshaling is Ruby version dependent. It's only compatible under specific circumstances with other Ruby versions. So keep that in mind:
In normal use, marshaling can only load data written with the same major version number and an equal or lower minor version number.
I am trying to identify when a file is PNG or JPG to apply it as a wallpaper. I am using the SHGetFileInfo to get the type name with the .szTypeName variable, but I just realized that it changes if the OS is in another language.
This is my code:
SHFILEINFOW fileInfo;
UINT sizeFile = sizeof(fileInfo);
UINT Flags = SHGFI_TYPENAME | SHGFI_USEFILEATTRIBUTES;
// Getting file info to find out if it has JPG or PNG format
SHGetFileInfoW(argv[1], 0, &fileInfo, sizeFile, Flags);
This is how I am validating:
if (wcscmp(fileInfo.szTypeName, L"JPG File") == 0)
{
//Code here
}
When the OS is in spanish, the value changes to "Archivo JPG" so I would have to validate against all language, and does not make sense.
Any idea what other function I can use?
This API is meant to be used to produce a user-facing string representation for known file types1). It is not meant to be used to implement code logic.
More importantly, it doesn't try to parse the file contents. It works off of the file extension alone. If you rename an Excel workbook MySpreadsheet.xlsx to MySpreadsheet.png, it will happily report, that this is a "PNG File".
The solution to your problem is simple: You don't have to do anything, other than filtering on the file extension. Use PathFindExtension (or PathCchFindExtension for Windows 8 and above) to get the file extension from a fully qualified path name.
This can fail, in case the user appended the wrong file extension. Arguably, this isn't something your application should fix, though.
As an aside, you pass SHGFI_USEFILEATTRIBUTES to SHGetFileInfoW but decided to not pass any file attributes (second argument) to the call. This is a bug. See What does SHGFI_USEFILEATTRIBUTES mean? for details.
1) It is the moral equivalent of SHGFI_DISPLAYNAME. The only thing you can do with display names is display them.
I'm working with the Podio application, and I made a tiny little script to grab data from my Podio database and put in an xlsx file, and convert it to a csv using roo.
xlsx = Podio::Item.xlsx( <app_number>, options = {} )
fname = "blah.xlsx"
somefile = File.open(fname, "w")
somefile.puts xlsx
somefile.close
xlsx_data = Roo:Spreadsheet.open(fname)
csv = xlsx_data.to_csv
puts csv
So this works, but printing to a file, only to grab the data back again and convert it to a csv seems...sloppy. Is there a way to not print it to a file, put it in a variable, and then convert it to a csv so I'm not printing files?
I tried using StringIO.new to the output from Podio::Item.xlsx, but I got this as a response:
file = StringIO.new(xlsx)
`extname': no implicit conversion of StringIO into String (TypeError)
Didn't see anything about this in the Podio docs: https://developers.podio.com/doc/items/get-items-as-xlsx-63233 It just mentioned how to export an xlsx file, not what to do with it afterwards.
Any thoughts?
Roo::Excelx fakes support of streams by creating temporary file. So you won’t gain any profit avoiding it and passing StringIO instance to it.
I was unable to receive the error you received, but stream support in Roo is kinda cumbersome and I failed to load a stream directly, yielding it from:
StreamIO.new(File.read('/tmp/my_xlsx.xlsx'))
In your case I would go with Tempfile instance, just for the sake of code readability.
I have a function in my code that takes a string representing the url of an image and creates a File object from that string, to be attached to a Tweet. This seems to work about 90% of the time, but occasionally fails.
require 'open-uri'
attachment_url = "https://s3.amazonaws.com/FirmPlay/photos/images/000/002/443/medium/applying_too_many_jobs_-_daniel.jpg?1448392757"
image = File.new(open(attachment_url))
If I run the above code it returns TypeError: no implicit conversion of StringIO into String. If I change open(attachment_url) to open(attachment_url).read I get ArgumentError: string contains null byte. I also tried stripping out the null bytes from the file like so, but that also made no difference.
image = File.new(open(attachment_url).read.gsub("\u0000", ''))
Now if I try the original code with a different image, such as the one below, it works fine. It returns a File object as expected:
attachment_url = "https://s3.amazonaws.com/FirmPlay/photos/images/000/002/157/medium/mike_4.jpg"
I thought maybe it had something to do with the params in the original url, so I stripped those out, but it made no difference. If I open the images in Chrome they appear to be fine.
I'm not sure what I'm missing here. How can I resolve this issue?
Thanks!
Update
Here is the working code I have in my app:
filename = self.attachment_url.split(/[\/]/)[-1].split('?')[0]
stream = open(self.attachment_url)
image = File.open(filename, 'w+b') do |file|
stream.respond_to?(:read) ? IO.copy_stream(stream, file) : file.write(stream)
open(file)
end
Jordan's answer works except that calling File.new returns an empty File object, whereas File.open returns a File object containing the image data from stream.
The reason you're getting TypeError: no implicit conversion of StringIO into String is that open sometimes returns a String object and sometimes returns a StringIO object, which is unfortunate and confusing. Which it does depends on the size of the file. See this answer for more information: open-uri returning ASCII-8BIT from webpage encoded in iso-8859 (Although I don't recommend using the ensure-encoding gem mentioned therein, since it hasn't been updated since 2010 and Ruby has had significant encoding-related changes since then.)
The reason you're getting ArgumentError: string contains null byte is that you're trying to pass the image data as the first argument to File.new:
image = File.new(open(attachment_url))
The first argument of File.new should be a filename, and null bytes aren't allowed in filenames on most systems. Try this instead:
image_data = open(attachment_url)
filename = 'some-filename.jpg'
File.new(filename, 'wb') do |file|
if image_data.respond_to?(:read)
IO.copy_stream(image_data, file)
else
file.write(image_data)
end
end
The above opens the file (creating it if it doesn't exist; the b in 'wb' tells Ruby that you're going to write binary data), then writes the data from image_data to it using IO.copy_stream if it's a StreamIO object or File#write otherwise, then closes the file again.
If you use Paperclip, they have a method to copy to disk.
def raw_image_data
attachment.copy_to_local_file.read
end
change attachment to what ever variable you used of course.
I have a ruby function that accesses files in my unix filesystem.
I have 2050 files each representing an hashed value in a dedicated directory.
The function reads a file containing email addresses and performs a hashing function, finds out the file id and prints.
Usually I do those things in Java but I wanna start doing it in Ruby. My problem is, that within my function, I try to open the correct file for reading, but I see that open works the same as new when no code block is provided. From the IO class, method ::openWith no associated block, IO.open is a synonym for ::new.
What I simply need to do is, open the file, set the reader pointer to the first available line, write and flush.
For simplicity I will put every code statement in one line. The file should be opened with its current status (see the HERE comment).
def dispatch
while (id=IDS_FILE.gets)
bucket="#{BUCKETS}" << (PERFORM HERE THE HASH CALCULATION) ".txt"
#HERE
bucket_file=File.open("#{bucket}","w")
bucket_file.write(id)
bucket_file.close
end
log "Writing #{id.chomp!} to #{bucket_file.to_path}"
end
end
To get "the file to be opened with its current status" you'll need the "append mode", as documented here:
"a" Write-only, starts at end of file if file exists,
otherwise creates a new file for writing.
So your code should read like so:
File.open(bucket, "a") do |f|
f.write id
end