I want to upload a stringIO as a file to a server.
I'm not using rails to make the request, just ruby and some gems.
I have created a stringIO like below:
require 'rest-client'
require 'zip'
... some code
stream = Zip::OutputStream::write_buffer do |zip|
... code that download files and packs them
end
stream.rewind
RestClient.post('somepath', { file: stream.read }, headers)
RestClient is just some handy gem that makes it a bit easier to send http request.
so my problem is that on the server side in the controller when I receive the request I get an error
ArgumentError - invalid byte sequence in UTF-8:
activesupport (4.2.6) lib/active_support/core_ext/object/blank.rb:117:in `blank?'
carrierwave (0.10.0) lib/carrierwave/sanitized_file.rb:127:in `is_path?'
carrierwave (0.10.0) lib/carrierwave/sanitized_file.rb:93:in `size'
carrierwave (0.10.0) lib/carrierwave/sanitized_file.rb:136:in `empty?'
carrierwave (0.10.0) lib/carrierwave/uploader/cache.rb:119:in `cache!'
carrierwave (0.10.0) lib/carrierwave/mount.rb:329:in `cache'
carrierwave (0.10.0) lib/carrierwave/mount.rb:163:in `export='
carrierwave (0.10.0) lib/carrierwave/orm/activerecord.rb:39:in `export='
app/controllers/exports_controller.rb:17:in `upload'
(I'm using carrierwave to store the files)
I think that I'm not handling the stringIO properly but honestly don't exactly know how should I pass it to the post request so that it behaves like a file
any ideas?
turns out this was happening because I was not saving the zip as a file but used write_buffer that responds with stringIO.
stringIO read method returns a string that was uploaded to my controller and then I tried to save the file using carrierwave roughly like that:
... code in my controller
myObject = ObjectModel.find(params[:id])
myObject.fileUploader = StringIO.new(params[:file]) => this was the string i received from stringIO read method
myObject.save
but it won't work because StringIO objects don't have original_filename method anymore since Rails3 (As far as I understood from the carrierwave issue page)
The solutions was to create a wrapper around StringIO and add either an attribute or a method that is called original_filename afterwards file was being saved correctly.
I'm posting link to carrierwave wiki that deals with this:
https://github.com/carrierwaveuploader/carrierwave/wiki/How-to:-Upload-from-a-string-in-Rails-3-or-later
and a sample wrapper like object
class StringIOWrapper < StringIO
attr_accessor :original_filename
end
afterwards this will work
myObject.fileUploader = StringIOWrapper.new(params[:file])
myObject.save
Related
I am working on a project with Sitespec, which is a static site generator using RSpec and uses any Rack app. I chose Padrino as a Rack app for Sitespec.
Then, I get the following errors when I do rspec at some URLs of Content-type: image/png type.
Failures:
1) Sitespec GET public/images/simple-image1.png Generate a static image public/images/simple-image1.png
Failure/Error: pathname.write(response.body)
Encoding::UndefinedConversionError:
"\x89" from ASCII-8BIT to UTF-8
Pathname.write(response.body) is going to try to convert it to UTF-8.
But when I use Sinatra instead of Padrino, the error does not occur.
I made simple sample projects for comparison:
Using Padrino https://github.com/Nyoho/sitespec-padrino-sample2
Using Sinatra https://github.com/Nyoho/simple-sitespec-sinatra-test
Question: Why does the Padrino project fail and how do I fix the errors?
By the way, doing monkey patch
module Sitespec
class Artifact
def generate_file
pathname.binwrite(response.body)
end
end
end
eliminates the error. (Replacing Pathname#write to Pathname#binwrite.)
I am trying to download a track from Soundcloud using the ruby sdk (soundcloud 0.2.0 gem) with an app. I have registered the app on soundcloud and the client_secret is correct. I know this because I can see my profile info and tracks using the app.
Now when I try to download a track using the following code
#track = current_user.soundcloud_client.get(params[:track_uri])
data = current_user.soundcloud_client.get(#track.download_url)
File.open("something.mp3","wb"){|f|f.write(data)}
and when I open the file it has nothing in it. I've tried many approaches including the following one,
data = current_user.soundcloud_client.get(#track.download_url)
file = File.read(data)
And this one gives me an error
can't convert nil into String
on line 13 which is in
app/controllers/store_controller.rb:13:in `read'
that is the File.read function.
I have double checked that the track I am trying to download is public and downloadable.
I tried to test the download_url that is being used explicitly by copying it from console and sending a request using Postman and it worked. I am not sure why it is not working with the app when other things are working so well.
What I want to do is to successfully be able to either download or at least get the data which I could store somewhere.
Version details : -
ruby 1.9.3p194 (2012-04-20 revision 35410) [x86_64-linux]
Rails 3.2.18
soundcloud 0.2.0
There are few assumptions that you have to understand before doing this thing.
Not every track on SoundClound can be downloaded! Only tracks that are flagged as downloadable can be downloaded - your code has to consider that option!
Your track URL has to be "resolved" before you get to download_url and after you get download_url you have to use your client_id to get the final download URL.
Tracks can be big, and downlowding them requires time! You should never do tasks like this straight from your Rails app in your controller or model. If the tasks runs longer you always use some background worker or some other kind of background processing "thing" - Sidekiq for example.
Command-line client example
This is example of working client, that you can use to download tracks from SoundClound. Its using official Official SoundCloud API Wrapper for Ruby, assumes that you are using Ruby 1.9.x and its not dependent on Rails in any way.
# We use Bundler to manage our dependencies
require 'bundler/setup'
# We store SC_CLIENT_ID and SC_CLIENT_SECRET in .env
# and dotenv gem loads that for us
require 'dotenv'; Dotenv.load
require 'soundcloud'
require 'open-uri'
# Ruby 1.9.x has a problem with following redirects so we use this
# "monkey-patch" gem to fix that. Not needed in Ruby >= 2.x
require 'open_uri_redirections'
# First there is the authentication part.
client = SoundCloud.new(
client_id: ENV.fetch("SC_CLIENT_ID"),
client_secret: ENV.fetch("SC_CLIENT_SECRET")
)
# Track URL, publicly visible...
track_url = "http://soundcloud.com/forss/flickermood"
# We call SoundCloud API to resolve track url
track = client.get('/resolve', url: track_url)
# If track is not downloadable, abort the process
unless track["downloadable"]
puts "You can't download this track!"
exit 1
end
# We take track id, and we use that to name our local file
track_id = track.id
track_filename = "%s.aif" % track_id.to_s
download_url = "%s?client_id=%s" % [track.download_url, ENV.fetch("SC_CLIENT_ID")]
File.open(track_filename, "wb") do |saved_file|
open(download_url, allow_redirections: :all) do |read_file|
saved_file.write(read_file.read)
end
end
puts "Your track was saved to: #{track_filename}"
Also note that files are in AIFF (Audio Interchange File Format). To convert them to mp3 you do something like this with ffmpeg.
ffmpeg -i 293.aif final-293.mp3
I have a custom gem and am encountering a really weird LoadError when I install it as a gem and attempt to require it in irb.
Everything works fine with my rspec tests inside the project folder. This only occurs when using it as an actual gem in irb.
The file it throws a LoadError exception at (/lib/mws/api/order_response.rb) does in fact exist. I've tried renaming the file and updating the file that requires it (/lib/mws.rb). I've tried recreating the file thinking maybe there was a permissions issue. Nothing works.
If I comment out the require line for that specific file, everything works. There's nothing special about the file. There's 4 other files nearly identical to it (*_response.rb).
I feel like I'm taking crazy pills. I must being overlooking something but I sure don't see it.
Trace:
chris#Samus:~$ irb
1.9.3p194 :001 > require 'mws'
LoadError: cannot load such file -- mws/api/order_response
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /Users/chris/.rvm/gems/ruby-1.9.3-p194/gems/mws-0.1.18/lib/mws.rb:14:in `<top (required)>'
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `require'
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `rescue in require'
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
from (irb):1
from /Users/chris/.rvm/rubies/ruby-1.9.3-p194/bin/irb:16:in `<main>'
File with the requires (/lib/mws.rb)
require 'mws/base'
require 'mws/connection'
require 'mws/utility'
require 'mws/api/seller'
require 'mws/api/product'
require 'mws/api/order'
require 'mws/api/report'
require 'mws/api/general_response'
require 'mws/api/product_response'
require 'mws/api/report_response'
require 'mws/api/seller_response'
require 'mws/api/order_response' # <--- the offending line
module MWS
# #see Base#initialize MWS::Base for instantiation details.
# #return [Base] returns MWS::Base object.
def self.new(merchant_id, access_key, secret_key)
MWS::Base.new(merchant_id, access_key, secret_key)
end
end
# The below is for documentation generation purposes.
# MWS is a wrapper for the Amazon Marketplace Web Service (MWS) API.
module MWS
# API handles all the Amazon MWS API specific stuff.
module API
end
# Utilities contains various functions needed throughout MWS. Utilities is a mixin to multiple classes.
module Utilities
end
end
File I'm requiring (/lib/mws/api/order_response.rb):
module MWS
module API
# Class for parsing Amazon's XML responses into managable objects.
class OrderResponse
# Include GeneralResponse instance methods as class methods
extend GeneralResponse
end
end
end
And my file structure
For anyone interested, I was using Jeweler to handle building this gem. As it turns out, Jeweler uses your Git repository when building a gemspec.
If you haven't added all required files to your git repository, Jeweler's gemspec rake task will not include them when generating a new gemspec file.
Can should check in /Users/chris/.rvm/gems/ruby-1.9.3-p194/gems/mws-0.1.18/lib/mws/api if the file lies there (and doesn't have obscure permissions).
If that's not the case, you probably forgot to add it in your gemspec.
If it is there, please try requiring/loading it with the absolute path (for debugging purpose).
I want to access request in the Rack::ResponseHeaders. I am using Sinatra in my app.
Below is my code:
use Rack::ResponseHeaders do |headers|
# Manipulation of request variables.
# Setting request headers.
end
The question is that in order to manipulate variables in request, I need to have the request variable first.
Please suggest.
First thing is, you need to install the gem rack-contrib via rubygems:
$ gem install rack-contrib
This gem contains contributed rack utilities. Then you need to require this gem in your app:
require 'rack/contrib'
It may be enough to only require the response headers utility (not tested):
require 'rack/contrib/response_headers'
Then you can use this utility to tap into the headers, for example:
use Rack::ResponseHeaders do |headers| # tap into headers
unless headers['cache-control'] # if header not set,
headers['cache-control'] = "public, max-age=1800" # set it to ...
end
end
Let me know whether this is working for you.
I am in an environment where I don't have access to install any gems. I only have standard ruby (version:1.8.7) installation.
I am trying something like this:
require 'rss/1.0'
require 'rss/2.0'
require 'open-uri'
source = "http://www.example.com/feed.atom" # url or local file
content = "" # raw content of rss feed will be loaded here
open(source) do |s| content = s.read end
rss = RSS::Parser.parse(content, false)
When I am parsing the content, I am getting nil. So I am wondering if in-built RSS module supports parsing an atom feed.
If you look under RSS::Maker what it can parse.
As an alternative, consider trying the nokogiri gem.