I have a rails 3 app where I am using the 'face' gem to reference the Face.com API. The api method takes a parameter of the form:
:file => File.new(path_to_file, 'rb')
which works.
I am trying to change the flow of the app so that the file can be uploaded via a form, do some work with RMagick and then make the API call, all without saving the file to disk.
I can generate the RMagick 'Image' with
image = Magick::Image.from_blob(upload_image_field.read)
I can then manipulate the file with RMagick and even save the results into the database with:
self.data = image.to_blob #normally 'upload_image_field.read' if not using RMagick
My problem is that I can't change the image file (or the blob) into something that the API will recognize (without saving it to disk and then referencing the file on disk).
For example using this in the API method fails:
:file => image.to_blob
How do I convert he blob into the same format as
File.new(path_to_file, 'rb')
Thanks
OK, I could be wrong on this one... but I wanted to dig this up. Unfortunately, you just have to live with saving it as a file. The reason is because the API makes an HTTP POST. Unfortunately, this needs to be a file.
References from: [https://github.com/rociiu/face/tree/master/lib/face]:
recognition.rb:
def faces_detect(opts={})
opts.assert_valid_keys(:urls, :file, :detector, :attributes, :callback, :callback_url)
make_request(:faces_detect, opts)
end
utils.rb:
def make_request(api_method, opts={})
....
response = JSON.parse( RestClient.post(API_METHODS[ api_method ], opts.merge(api_crendential)).body )
....
end
So, why is it a problem to save to a file then?
Related
I'm running a script that updates a metadata field on some of my S3 objects after they have already been uploaded to the S3 bucket. On initialization, I am setting the content-type by checking the file name.
def save_to_amazon(file, s3_object, file_name, meta_path)
puts "uploaded #{file} to Amazon S3"
content_type = set_content_type(file_name)
s3_object.write(file.get_input_stream.read, :metadata => { :folders => meta_path}, :content_type => content_type)
end
At this point, the S3 content-type works fine for these objects. The problem arises when I update the metadata later on. I run something like this:
s3_object.metadata['folders'] = "some string"
At this point, I get an empty string returned when I run s3_objects.content_type after updating the metadata.
s3_object.content_type = is not available.
As far as I can tell from reading the Rdoc there isn't a way to assign content-type after uploading the S3 file. I have tried using the metadata method like
s3.object.metadata['content_type'] = "some string"
s3.object.metadata['content-type'] = "some string"
Both of these appear to assign a new custom metadata attribute instead of updating the object's mime type.
Is there a way to set this, or do I need to completely re-upload the file again?
To elaborate on tkotisis reponse, here is what I did to update the content-type using copy_to. You can use s3object.head[:metadata] to pull out the existing metadata to copy it over as referenced here.
amazon_bucket.objects.each do |ob|
metadata = ob.head[:metadata]
content_type = "foo/bar"
ob.copy_to(ob.key, :metadata => metadata, :content_type => content_type)
end
EDIT
amazon_bucket.objects.each do |ob|
metadata = ob.metadata
content_type = "foo/bar"
ob.copy_to(ob.key, :metadata{:foo => metadata[:foo]}, :content_type => content_type)
end
Your example code only modifies your in-memory object.
To modify the metadata of the actual S3 object, issue a copy request with destination key the one of your current object.
EDIT
According to the documentation
Using the copy operation, you can rename objects by copying them and
deleting the original ones.
When copying an object, you might decide to update some of the
metadata values. For example, if your source object is configured to
use standard storage, you might choose to use reduced redundancy
storage for the object copy. You might also decide to alter some of
the user-defined metadata values present on the source object. Note
that if you choose to update any of the object's user configurable
metadata (system or user-defined) during the copy, then you must
explicitly specify all the user configurable metadata, even if you are
only changing only one of the metadata values, present on the source
object in your request.
I haven't tried it, but using the Ruby SDK this is probably achieved through the
- (S3Object) copy_to(target, options = {})
method.
I'm using a gem "aws-sdk", "~> 2" (2.2.3)
Assume that you have a current file without set content-type (Content-type will be set as a "binary/octet-stream" by default)
How to check a content-type file?
If you use the RestClient as follows:
object mean Aws::S3::Object
bucket = Aws::S3::Bucket.new(bucket_name)
object = bucket.object(key)
RestClient.head(object.presigned_url(:head)) do |resp|
puts resp.headers
puts resp.headers[:content_type]
end
How to change a content-type file?
In my case, I wanna change a content-type to 'image/jpeg' which current object is 'binary/octet-stream' so you can
object.copy_from(
object,
content_type: 'image/jpeg',
metadata_directive: 'REPLACE'
)
Make sure you set the ACL to :public read, otherwise your files will be unavailable after copying.
This did the trick for me:
bucket.objects.with_prefix('my_assets').each do |obj|
metadata = obj.head[:metadata]
content_type = "application/pdf"
obj.copy_to(obj.key, :metadata => metadata, :content_type => content_type)
obj.acl = :public_read
end
Although not Ruby I found this project which automatically guessing the mime type based on the extension and resets is via the same copy method that the other answers refers to. It's not terribly quick since it has to copy the blob. If you needed to make it happen faster you could probably divide up the work and copy in parallel via something like IronWorker. I did a similar thing for resetting permissions.
I have an application that reads the content of a file and indexes it. I was storing them in the disk itself, but now I'm using Amazon S3, so the following method doesn't work anymore.
It was something like this:
def perform(docId)
#document = Document.find(docId)
if #document.file?
#You should't create a new version
#document.versionless do |doc|
#document.file_content = Cloudoc::Extractor.new.extract(#document.file.file)
#document.save
end
end
end
#document.file returns the FileUploader, and doc.file.file returns the CarrierWave::Storage::Fog::File class.
How can I get the real file?
Calling #document.file.read will get you the contents of the file from S3 in Carrierwave.
I'm using ruby 1.9.2 along with Rails 3.1.4 and Paperclip 2.4.5.
My issue is trying to save a paperclip attachment from a URI loses the file extension and saves the file without one resulting in issues with things like fancybox that require an extension.
Some example code:
uri = "http://featherfiles.aviary.com/2012-06-13/bbe5f0de1/0c5a672b88ea47ecb4631ac173e27430.png"
open(uri)
#=> #<File:/var/folders/zc/d69gxhzx10x_bvjrkqgyjgxr0000gn/T/open-uri20120613-27204-i6cldv>
Because there is no extension on the temp file paperclip is saving the file without one resulting in issues.
Has anyone run into this issue? I've seen multiple answers about using paperclip to store images from a URI but none seem to address the same problem we're running
Don't use the temporary file! It's there as a placeholder as the file is read from the port, and should be considered a private resource for OpenURI. Instead, use open(url).read and work with the resulting content by saving it.
Do something like:
require 'uri'
require 'open-uri'
url = 'http://www.iana.org/domains/example/index.html'
filename = File.basename(URI.parse(url).path)
File.open(filename, 'wb') do |fo|
fo.write(open(url).read)
end
Temporarily spooling to disk during an operation, especially a network operation, is common. Once the file's content has been accumulated, then it is available to be passed off to the app. read is blocking, so your code will stop there until the file is returned to you. Then you can play with it.
Extension isn't important for temporary file, but if you want use this file in code or save to another place. You can do it:
temp_file = open(params[:url])
def temp_file.original_filename; File.basename(base_uri.path); end
Now, you can save this temporary file to permanent space or use it in code; Original filename will be used automatically.
Im not sure if this will help in your case, but I was noticing similar issues in my project.
The issue turned out to be not caused by Paperclip nor open-uri, but the receiver of the paperclip file (in my case Spree Commerce). Check that you are assigning the paperclip object to the right object, and that it is being interpreted correctly.
The fix that worked for me was to change:
#product.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
to
#product.master.images << Spree::Image.create({
:attachment => open(image_url)
}, :without_protection => true)
Good luck with your issue
Have you inclued the :extension in your path/url option?
For example:
has_attached_file :image,
...
:url => '/images/highlights/:id_partition/:style_:id.:extension',
:path => ':rails_root/files/images/highlights/:id_partition/:style_:id.:extension'
This will probably solve your problem.
You can force an extension there, but I don't think that's recommended.
Update – Paperclip can do this on its own!
Posted by Aditya Sanghi (thanks a lot!):
current_comments.pictures.create!(file: URI.parse(image_url))
Although keep in mind, that you still need to handle 500, 404, etc
errors (Paperclip can raise them).
Thanks to: https://mensfeld.pl/2013/12/rails-paperclip-open-uri-downloading-files-from-the-internet-and-saving-them-with-paperclip/
Yes, it is a problem but we can get around this with fancybox.
In the link tag(for image) add :type => 'image'
- #images.each do |image|
= link_to image_tag(image.attachment.url), image.attachment.url, class: "fancybox", type: 'image'
By specifying 'type', Fancybox overrides the type as image
https://groups.google.com/forum/?fromgroups=#!topic/fancybox/QgjquBCLynU
Does anyone have any insights into using CarrierWave with an ActiveResource model (in Rails 3)? I've got an ActiveResource model with field for the filename, and I want to save the file to the remote filesystem.
I've tried a few things without much success (or conviction that I was doing anything remotely correctly), so I'd appreciate suggestions from anyone who's successfully implemented CarrierWave without using the ORM modules already included in the gem.
I'm probably late for this as the original author has moved on, but this question comes up at the top when someone searches for "carrierwave activeresource", so I thought it was still worth answering.
For the sake of discussion, let's assume we have a model named Artist with a picture named artist_picture mounted as a CarrierWave uploader. With ActiveRecord, you would assign this picture to a File:
artist.artist_picture=File.open('ravello.jpg')
And when you save artist:
artist.save!
the picture will be saved, also.
Now, let's say I create a resource based on this:
class Artist < ActiveResource::Base
end
If I subsequently read in an artist:
artist = Artist.find(1)
and look at it, I'll find this in there:
#<Artist:0x39432039 #attributes={"id"=>1, "name"=>"Ravello", "artist_picture"=>#<ArtistPicture:0x282347249243 #attributes={"url"=>"/uploads/artists/artist_picture/1/ravello.jpg"}, #prefix_options={}, #persisted=false>, #prefix_options={}, #persisted=false>
Interestingly, artist_picture is itself a model and we could declare it and play around with it if we wanted. As it is, you can use the url to grab the picture if you want. But let's talk instead about uploading another picture.
We can add this little bit of code to the Artist model on the server side:
def artist_picture_as_base64=(picsource)
tmpfile = Tempfile.new(['artist','.jpg'], Rails.root.join('tmp'), :encoding => 'BINARY')
begin
tmpfile.write(Base64.decode64(picsource.force_encoding("BINARY")))
file = CarrierWave::SanitizedFile.new(tmpfile)
file.content_type = 'image/jpg'
self.artist_picture = file
ensure
tmpfile.close!
end
end
I'm just showing a simple example - you should probably pass the original filename, also. Anyway, on the resource side:
class Artist < ActiveResource::Base
def artist_picture=(filename)
self.artist_picture_as_base64=Base64.encode64(File.read(filename))
end
end
At this point, on the resource side you need only set "artist_picture" to a filename and it will be encoded and sent when the resource is saved. On the server side, the file will be decoded and saved. Presumably you could skip base64 encoding by just forcing the string to binary encoding, but it craps when I do that and I don't have the patience to track it down. Encoding as base64 works.
I wrote a simple Sinatra app that generate an image using rmagick from some user inputs. The image is saved in the ./public directory with a unique file name. The unique file name is used in the HTML generated by Sinatra so that each user gets the correct image. Once a day a script deletes files older than one hour. This is clearly a terrible hack but I have no web experience!
Is there any way to serve the rmagick image in sinatra without first saving it to disk?
Use the Image#to_blob method to turn the in-memory image into a string:
get '/' do
content_type 'image/png'
img = Magick::Image.read('logo:')[0]
img.format = 'png'
img.to_blob
end