Ruby open-uri open method loses file extension opening images - ruby

I'm using ruby 1.9.2 along with Rails 3.1.4 and Paperclip 2.4.5.
My issue is trying to save a paperclip attachment from a URI loses the file extension and saves the file without one resulting in issues with things like fancybox that require an extension.
Some example code:
uri = "http://featherfiles.aviary.com/2012-06-13/bbe5f0de1/0c5a672b88ea47ecb4631ac173e27430.png"
open(uri)
#=> #<File:/var/folders/zc/d69gxhzx10x_bvjrkqgyjgxr0000gn/T/open-uri20120613-27204-i6cldv>
Because there is no extension on the temp file paperclip is saving the file without one resulting in issues.
Has anyone run into this issue? I've seen multiple answers about using paperclip to store images from a URI but none seem to address the same problem we're running

Don't use the temporary file! It's there as a placeholder as the file is read from the port, and should be considered a private resource for OpenURI. Instead, use open(url).read and work with the resulting content by saving it.
Do something like:
require 'uri'
require 'open-uri'
url = 'http://www.iana.org/domains/example/index.html'
filename = File.basename(URI.parse(url).path)
File.open(filename, 'wb') do |fo|
fo.write(open(url).read)
end
Temporarily spooling to disk during an operation, especially a network operation, is common. Once the file's content has been accumulated, then it is available to be passed off to the app. read is blocking, so your code will stop there until the file is returned to you. Then you can play with it.

Extension isn't important for temporary file, but if you want use this file in code or save to another place. You can do it:
temp_file = open(params[:url])
def temp_file.original_filename; File.basename(base_uri.path); end
Now, you can save this temporary file to permanent space or use it in code; Original filename will be used automatically.

Im not sure if this will help in your case, but I was noticing similar issues in my project.
The issue turned out to be not caused by Paperclip nor open-uri, but the receiver of the paperclip file (in my case Spree Commerce). Check that you are assigning the paperclip object to the right object, and that it is being interpreted correctly.
The fix that worked for me was to change:
#product.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
to
#product.master.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
Good luck with your issue

Have you inclued the :extension in your path/url option?
For example:
has_attached_file :image,
...
:url => '/images/highlights/:id_partition/:style_:id.:extension',
:path => ':rails_root/files/images/highlights/:id_partition/:style_:id.:extension'
This will probably solve your problem.
You can force an extension there, but I don't think that's recommended.

Update – Paperclip can do this on its own!
Posted by Aditya Sanghi (thanks a lot!):
current_comments.pictures.create!(file: URI.parse(image_url))
Although keep in mind, that you still need to handle 500, 404, etc
errors (Paperclip can raise them).
Thanks to: https://mensfeld.pl/2013/12/rails-paperclip-open-uri-downloading-files-from-the-internet-and-saving-them-with-paperclip/

Yes, it is a problem but we can get around this with fancybox.
In the link tag(for image) add :type => 'image'
- #images.each do |image|
= link_to image_tag(image.attachment.url), image.attachment.url, class: "fancybox", type: 'image'
By specifying 'type', Fancybox overrides the type as image
https://groups.google.com/forum/?fromgroups=#!topic/fancybox/QgjquBCLynU

Related

Download a GitHub file with Octokit::Client in Ruby

I am trying to download an XML file from GitHub via Octokit::Client in Ruby. This is in a Dangerfile, so I have access to the client via github.api.
I have got something working with the following code:
listing = client.contents('erik-allen/RepoName', :path => 'Root/Path')
download = open(listing[0].download_url)
I can then call Nokogiri::XML(download) and parse the XML with no issues.
However, it only works because it is the only file in the directory. It also does not feel like the correct way to do things.
I have tried a few other ways:
download = client.contents('erik-allen/RepoName', :path => 'Root/Path/File.xml')
That returned a Sawyer::Resource but I have yet to find a way to get any data from that. I tried combinations of .get.data and .data but neither worked. Calling Base64.decode64() on the result did not yield anything either.
I am suspecting I may need an "accept" header, but I am not sure how I would do that with Octokit::Client.
Does anyone have any suggestions. I would have assumed this would be a common task, but I can find no examples.
I was able to eventually figure things out. There is a content property on the Sawyer::Resource that gives the Base64 data. So the final solution is:
contents = client.contents('erik-allen/RepoName', :path => 'Root/Path/File.xml')
download = Base64.decode64(contents.content)
I can then call Nokogiri::XML(download) and parse the XML with no issues.

Updating content-type after file upload on Amazon S3 with Amazon-SDK Ruby gem

I'm running a script that updates a metadata field on some of my S3 objects after they have already been uploaded to the S3 bucket. On initialization, I am setting the content-type by checking the file name.
def save_to_amazon(file, s3_object, file_name, meta_path)
puts "uploaded #{file} to Amazon S3"
content_type = set_content_type(file_name)
s3_object.write(file.get_input_stream.read, :metadata => { :folders => meta_path}, :content_type => content_type)
end
At this point, the S3 content-type works fine for these objects. The problem arises when I update the metadata later on. I run something like this:
s3_object.metadata['folders'] = "some string"
At this point, I get an empty string returned when I run s3_objects.content_type after updating the metadata.
s3_object.content_type = is not available.
As far as I can tell from reading the Rdoc there isn't a way to assign content-type after uploading the S3 file. I have tried using the metadata method like
s3.object.metadata['content_type'] = "some string"
s3.object.metadata['content-type'] = "some string"
Both of these appear to assign a new custom metadata attribute instead of updating the object's mime type.
Is there a way to set this, or do I need to completely re-upload the file again?
To elaborate on tkotisis reponse, here is what I did to update the content-type using copy_to. You can use s3object.head[:metadata] to pull out the existing metadata to copy it over as referenced here.
amazon_bucket.objects.each do |ob|
metadata = ob.head[:metadata]
content_type = "foo/bar"
ob.copy_to(ob.key, :metadata => metadata, :content_type => content_type)
end
EDIT
amazon_bucket.objects.each do |ob|
metadata = ob.metadata
content_type = "foo/bar"
ob.copy_to(ob.key, :metadata{:foo => metadata[:foo]}, :content_type => content_type)
end
Your example code only modifies your in-memory object.
To modify the metadata of the actual S3 object, issue a copy request with destination key the one of your current object.
EDIT
According to the documentation
Using the copy operation, you can rename objects by copying them and
deleting the original ones.
When copying an object, you might decide to update some of the
metadata values. For example, if your source object is configured to
use standard storage, you might choose to use reduced redundancy
storage for the object copy. You might also decide to alter some of
the user-defined metadata values present on the source object. Note
that if you choose to update any of the object's user configurable
metadata (system or user-defined) during the copy, then you must
explicitly specify all the user configurable metadata, even if you are
only changing only one of the metadata values, present on the source
object in your request.
I haven't tried it, but using the Ruby SDK this is probably achieved through the
- (S3Object) copy_to(target, options = {})
method.
I'm using a gem "aws-sdk", "~> 2" (2.2.3)
Assume that you have a current file without set content-type (Content-type will be set as a "binary/octet-stream" by default)
How to check a content-type file?
If you use the RestClient as follows:
object mean Aws::S3::Object
bucket = Aws::S3::Bucket.new(bucket_name)
object = bucket.object(key)
RestClient.head(object.presigned_url(:head)) do |resp|
puts resp.headers
puts resp.headers[:content_type]
end
How to change a content-type file?
In my case, I wanna change a content-type to 'image/jpeg' which current object is 'binary/octet-stream' so you can
object.copy_from(
object,
content_type: 'image/jpeg',
metadata_directive: 'REPLACE'
)
Make sure you set the ACL to :public read, otherwise your files will be unavailable after copying.
This did the trick for me:
bucket.objects.with_prefix('my_assets').each do |obj|
metadata = obj.head[:metadata]
content_type = "application/pdf"
obj.copy_to(obj.key, :metadata => metadata, :content_type => content_type)
obj.acl = :public_read
end
Although not Ruby I found this project which automatically guessing the mime type based on the extension and resets is via the same copy method that the other answers refers to. It's not terribly quick since it has to copy the blob. If you needed to make it happen faster you could probably divide up the work and copy in parallel via something like IronWorker. I did a similar thing for resetting permissions.

Watir-Webdriver: Uploading files isn't working correctly on firefox

Using this code I cannot get a file to upload correctly to the website.
browser.form(:index, 2).file_field(:name, "filedata").set(""+folderName+"/iTunesArtwork")
Once the code is run it shows that it's uploading by displaying a spinning wheel but it never actually uploads and the wheel just keeps spinning. If I upload it on my own though it uploads fine.
Here's a link for the html:
http://f.cl.ly/items/3v3o1p1g0t2S1q3q3Q1h/Text%202011.09.03%2011:40:06%20PM.html
As you can see the html is in a form tag. If I try to access the file_field without going through the form first it will give me an error saying element can't be interacted with because it's not visible. Anyone have a clue what's going on?
Your local file you are uploading mustn't exist locally.
I have uploaded your HTML to: http://dl.dropbox.com/u/18859962/uploader.html
I tried this:
ruby-1.9.2-p290 :001 > require "watir-webdriver"
=> true
ruby-1.9.2-p290 :002 > b = Watir::Browser.start "http://dl.dropbox.com/u/18859962/uploader.html"
=> #<Watir::Browser:0x..fdea53ebfe3940b9a url="http://dl.dropbox.com/u/18859962/uploader.html" title="untitled">
ruby-1.9.2-p290 :003 > b.file_field.exists?
=> true
ruby-1.9.2-p290 :004 > local_file = "/users/me/ie.html"
=> "/users/me/ie.html"
ruby-1.9.2-p290 :005 > File.exists? local_file
=> true
ruby-1.9.2-p290 :006 > raise "error" unless File.exists? local_file
=> nil
ruby-1.9.2-p290 :007 > b.file_field.set local_file
=> "/users/me/ie.html
Please run your script against this hosted html file and report back.
Also, you should raise an exception if the local file doesn't exist, so that eliminates that error.
I wasn't able to get file_field.set to work until I started giving it an absolute path to the file.
My code looks something like:
relative_path = 'image.png'
full_path = File.expand_path relative_path
browser.file_field(:id, 'file_field_id').set full_path
Hope this helps!
In terms of this part of your question:
As you can see the html is in a form tag. If I try to access the
file_field without going through the form first it will give me an
error saying element can't be interacted with because it's not
visible. Anyone have a clue what's going on?
The most likely answer is that there may be another input field in the DOM that has the same name and is not currently visible to the user. Without being able to look at the entire page and see all the HTML, I can't say for sure, but that error is typical of when I've tried to interact with controls that had the same name or text as others, but were currently hidden from view. searching the page code for the name or text I was using would usually discover the other element. Changing how I was addressing things (either by specifying a unique container as you are doing, or adding an :index value along with the existing identifier will then allow you to get the 'right' instance of the element.
You could switch to using :id to identify the field (it has one), and see if that works without having to specify the outer (form) container element first. Selecting by ID is generally preferred anyway as the value should be unique on the page if it's valid HTML.
That doesn't address the 'why doesn't it upload' aspect of your question however, which I suspect is the real problem you need help with. but it does address that aspect of your question.
As to the rest, without being able to interact with the site myself, it's difficult to say why it is behaving the way it is. (and if the site is itunes, you should check what their terms of service has to say about using automation to access the site)

Convert file upload contents to a binary file without saving (Rails)

I have a rails 3 app where I am using the 'face' gem to reference the Face.com API. The api method takes a parameter of the form:
:file => File.new(path_to_file, 'rb')
which works.
I am trying to change the flow of the app so that the file can be uploaded via a form, do some work with RMagick and then make the API call, all without saving the file to disk.
I can generate the RMagick 'Image' with
image = Magick::Image.from_blob(upload_image_field.read)
I can then manipulate the file with RMagick and even save the results into the database with:
self.data = image.to_blob #normally 'upload_image_field.read' if not using RMagick
My problem is that I can't change the image file (or the blob) into something that the API will recognize (without saving it to disk and then referencing the file on disk).
For example using this in the API method fails:
:file => image.to_blob
How do I convert he blob into the same format as
File.new(path_to_file, 'rb')
Thanks
OK, I could be wrong on this one... but I wanted to dig this up. Unfortunately, you just have to live with saving it as a file. The reason is because the API makes an HTTP POST. Unfortunately, this needs to be a file.
References from: [https://github.com/rociiu/face/tree/master/lib/face]:
recognition.rb:
def faces_detect(opts={})
opts.assert_valid_keys(:urls, :file, :detector, :attributes, :callback, :callback_url)
make_request(:faces_detect, opts)
end
utils.rb:
def make_request(api_method, opts={})
....
response = JSON.parse( RestClient.post(API_METHODS[ api_method ], opts.merge(api_crendential)).body )
....
end
So, why is it a problem to save to a file then?

CarrierWave with ActiveResource

Does anyone have any insights into using CarrierWave with an ActiveResource model (in Rails 3)? I've got an ActiveResource model with field for the filename, and I want to save the file to the remote filesystem.
I've tried a few things without much success (or conviction that I was doing anything remotely correctly), so I'd appreciate suggestions from anyone who's successfully implemented CarrierWave without using the ORM modules already included in the gem.
I'm probably late for this as the original author has moved on, but this question comes up at the top when someone searches for "carrierwave activeresource", so I thought it was still worth answering.
For the sake of discussion, let's assume we have a model named Artist with a picture named artist_picture mounted as a CarrierWave uploader. With ActiveRecord, you would assign this picture to a File:
artist.artist_picture=File.open('ravello.jpg')
And when you save artist:
artist.save!
the picture will be saved, also.
Now, let's say I create a resource based on this:
class Artist < ActiveResource::Base
end
If I subsequently read in an artist:
artist = Artist.find(1)
and look at it, I'll find this in there:
#<Artist:0x39432039 #attributes={"id"=>1, "name"=>"Ravello", "artist_picture"=>#<ArtistPicture:0x282347249243 #attributes={"url"=>"/uploads/artists/artist_picture/1/ravello.jpg"}, #prefix_options={}, #persisted=false>, #prefix_options={}, #persisted=false>
Interestingly, artist_picture is itself a model and we could declare it and play around with it if we wanted. As it is, you can use the url to grab the picture if you want. But let's talk instead about uploading another picture.
We can add this little bit of code to the Artist model on the server side:
def artist_picture_as_base64=(picsource)
tmpfile = Tempfile.new(['artist','.jpg'], Rails.root.join('tmp'), :encoding => 'BINARY')
begin
tmpfile.write(Base64.decode64(picsource.force_encoding("BINARY")))
file = CarrierWave::SanitizedFile.new(tmpfile)
file.content_type = 'image/jpg'
self.artist_picture = file
ensure
tmpfile.close!
end
end
I'm just showing a simple example - you should probably pass the original filename, also. Anyway, on the resource side:
class Artist < ActiveResource::Base
def artist_picture=(filename)
self.artist_picture_as_base64=Base64.encode64(File.read(filename))
end
end
At this point, on the resource side you need only set "artist_picture" to a filename and it will be encoded and sent when the resource is saved. On the server side, the file will be decoded and saved. Presumably you could skip base64 encoding by just forcing the string to binary encoding, but it craps when I do that and I don't have the patience to track it down. Encoding as base64 works.

Resources