Use Fog with Ruby to generate a Pre-signed URL to PUT a file in Amazon S3 - ruby

I am using the Fog gem to generate presigned urls. I can do this successfully to get read access to the file. Here's what I do:
fog_s3 = Fog::Storage.new({
:provider => 'AWS',
:aws_access_key_id => key,
:aws_secret_access_key => secret
})
object_path = 'foo.wav'
expiry = Date.new(2014,2,1).to_time.to_i
url = fog_s3.directories.new(:key => bucket).files.new(:key => object_path).url(expiry,path_style: true)
But this doesn't work when I try to upload the file. Is there a way to specify the http verb so it would be a PUT and not a GET?
EDIT I see a method: put_object_url which might help. I don't know how access it.
Thanks
EDIT based upon your suggestion:
It helped - it got me a PUT - not GET. However, I'm still having issues. I added content type:
headers = { "Content-Type" => "audio/wav" }
options = { path_style: true }
object_path = 'foo.wav'
expiry = Date.new(2014,2,1).to_time.to_i
url = fog_s3.put_object_url(bucket,object_path, expiry, headers, options)
but the url does not contain Content-Type in it. When done from Javascript in HTML I get the Content-Type in the url and that seems to work. Is this an issue with Fog? or is my header incorrect?

I think put_object_url is indeed what you want. If you follow the url method back to where it is defined, you can see it uses a similar method underlying it called get_object_url here (https://github.com/fog/fog/blob/dc7c5e285a1a252031d3d1570cbf2289f7137ed0/lib/fog/aws/models/storage/files.rb#L83). You should be able to do something similar and can do so by calling this method from the fog_s3 object you already created above. It should end up just looking like this:
headers = {}
options = { path_style: true }
url = fog_s3.put_object_url(bucket, object_path, expires, headers, options)
Note that unlike get_object_url there is an extra headers option snuck in there (which you can use to do stuff like set Content-Type I believe).
Hope that sorts it for you, but just let me know if you have further questions. Thanks!
Addendum
Hmm, seems there may be a bug related to this after all (I'm wondering now how much this portion of the code has been exercised). I think you should be able to work around it though (but I'm not certain). I suspect you can just duplicate the value in the options as a query param also. Could you try something like this?
headers = query = { 'Content-Type' => 'audio/wav' }
options = { path_style: true, query: query }
url = fog_s3.put_object_url(bucket, object_path, expires, headers, options)
Hopefully that fills in the blanks for you (and if so we can think some more about fixing that behavior within fog if it makes sense to do so). Thanks!

Instead of using the *put_object_url* might I suggest that you try using the bucket.files.create action which take a Fog file Hash attributes and return a Fog::Storage::AWS::File.
I prefer to break it down in a bit more steps, here is an example:
fog_s3 = Fog::Storage.new({
:provider => 'AWS',
:aws_access_key_id => key,
:aws_secret_access_key => secret
})
# Define the filename
ext = :wav
filename = "foo.#{ext.to_s}"
# Path to your audio file?
path ="/"
# Define your expiry in the amount of seconds
expiry = 1.day.to_i
#Initialize the bucket to store too
fog_bucket = connection.directories.get(bucket)
file = {
:key => "#{filename}",
:body => IO.read("#{path}#{filename}"),
:content_type => Mime::Type.lookup_by_extension(ext),
:cache_control => "public, max-age=#{expiry}",
:expires => CGI.rfc1123_date(Time.now + expiry),
:public => true
}
# Returns a Fog::Storage::AWS::File
file = fog_bucket.files.create( file )
# Now to retrieve the public_url
url = file.public_url
Note: For subdir's checkout the :prefix option for a AWS bucket.
Fog File Documentation:
Optional attributes... bottom of the page, :) http://rubydoc.info/gems/fog/Fog/Storage/AWS/File
Hopefully the example will help explain the steps in creating a fog file... Cheers! :)

Related

Setting ACL for a Pre Signed Object URL with Fog

I'm generating a fog pre-signed URL for AWS using the following snippet:
bucket = "..."
object = "demo.jpg"
expires = Integer(Time.now + 4.hours)
headers = {}
options = { path_style: true }
fog.put_object_url(bucket, object, expires, headers, options)
This works great - except that the uploaded objects aren't accessible to the public. How can a public-read access control list (ACL) be applied to the upload path?
You have to list these extra parameters (eg. x-amz-acl, Content-Type) under the "query" key of the options hash.
So your example would be.
bucket = "..."
object = "demo.jpg"
expires = Integer(Time.now + 4.hours)
headers = {}
query = {"x-amz-acl" => "public-read"}
options = { path_style: true, query: query }
fog.put_object_url(bucket, object, expires, headers, options)
You have probably solved this by now but incase anyone else is stuck on this, as the lack of surrounding documentation does not make it very straight forward to implement.

How can I upload files to Redmine via ActiveResource / REST API?

I am trying to batch-upload images to Redmine and link them each to a certain wiki pages.
The docs (Rest_api, Using the REST API with Ruby) mention some aspects, but the examples fail in various ways. I also tried to derive ideas from the source - without success.
Can anyone provide a short example that shows how to upload and link an image from within Ruby?
This is a bit tricky as both attachments and wiki APIs are relatively new, but I have done something similar in the past. Here is a minimal working example using rest-client:
require 'rest_client'
require 'json'
key = '5daf2e447336bad7ed3993a6ebde8310ffa263bf'
upload_url = "http://localhost:3000/uploads.json?key=#{key}"
wiki_url = "http://localhost:3000/projects/some_project/wiki/some_wiki.json?key=#{key}"
img = File.new('/some/image.png')
# First we upload the image to get attachment token
response = RestClient.post(upload_url, img, {
:multipart => true,
:content_type => 'application/octet-stream'
})
token = JSON.parse(response)['upload']['token']
# Redmine will throw validation errors if you do not
# send a wiki content when attaching the image. So
# we just get the current content and send that
wiki_text = JSON.parse(RestClient.get(wiki_url))['wiki_page']['text']
response = RestClient.put(wiki_url, {
:attachments => {
:attachment1 => { # the hash key gets thrown away - name doesn't matter
:token => token,
:filename => 'image.png',
:description => 'Awesome!' # optional
}
},
:wiki_page => {
:text => wiki_text # original wiki text
}
})

HTTParty options parameter not functioning properly

I'm setting up an application which can make LastFM API Requests.
These are simple get requests and I'm using the HTTParty gem.
My function is as follows:
def get_albums
self.class.base_uri "http://ws.audioscrobbler.com/2.0/"
options = {
:user => "Gerard1992",
:method => "user.gettopalbums",
:api_key => Constants::LASTFM_API_KEY,
:format => "json"
}
puts options.to_query
self.class.get "/?#{options.to_query}", {} #options don't work
end
This piece of code that's shown above works. The get request returns a set of JSON. My problem is that this /?#{options.to_query} doesn't look that neat. And neither does the actual (now empty {}) options parameter. How do I get the HTTParty options parameter to work like it should?
This is what I've tried, but both cases failed:
self.class.get "/", options
self.class.get "/", options => options
I appreciate the help.
The correct option for query parameters in HTTParty is :query, so what you want is:
self.class.get "/", query: options
You can see all the available parameters in the docs.
Send :verify => false in options hash

Rails pagination in API using Her, Faraday

I've been trying to figure this out all day, and it's driving me crazy.
I have two rails apps, ServerApp and ClientApp. ClientApp gets data from ServerApp through an API, using the Her gem. Everything was great until I needed pagination information.
This is the method I am using to get the orders (this uses kamainari for pagination and ransack for search):
# ServerApp
def search
#search = Order.includes(:documents, :client).order('id desc').search(params[:q])
#orders = #search.result(distinct: true).page(params[:page]).per(params[:per])
respond_with #orders.as_json(include: :documents)
end
It returns an array of hashes in json, which Her uses as a collection of orders. That works fine.
# Response
[
{
"client_id": 239,
"created_at": "2013-05-15T15:37:03-07:00",
"id": 2422,
"ordered_at": "2013-05-15T15:37:03-07:00",
"origin": "online",
"updated_at": "2013-05-15T15:37:03-07:00",
"documents": [
{ ... }
]
},
...
]
But I needed pagination information. It looked like I needed to send it as metadata with my json. So I change my response to this:
respond_to do |format|
format.json do
render json: { orders: #orders.as_json(include: :documents), metadata: 'sent' }
end
end
This does indeed send over metadata, so in my ClientApp I can write #orders.metadata and get 'sent'. But now my orders are nested in an array inside of 'orders', so I need to use #orders.orders, and then it treats it like an array instead of a Her collection.
After doing some reading, it seemed sending pagination info through headers was the way a lot of other people did this (I was able to get the headers set up in an after_filter using this guide). But I am even more lost on how to get those response headers in my ClientApp - I believe I need a Faraday Middleware but I just am having no luck getting this to work.
If anyone knows how I can just get this done, I would be very grateful. I can't take another day of banging my head against the wall on this, but I feel like I am just one vital piece of info away from solving this!
I encountered the same issue and solved it by adding my own middleware and rewriting the "parse" and "on_complete" methods without that much hassle and avoiding the use of global variables.
Here's the code:
class CustomParserMiddleware < Her::Middleware::DefaultParseJSON
def parse(env)
json = parse_json(env[:body])
pagination = parse_json(env[:response_headers][:pagination_key]) || {}
errors = json.delete(:errors) || {}
metadata = json.delete(:metadata) || {}
{
:data => json,
:errors => errors,
:metadata => {
:pagination => pagination,
:additional_metadata => metadata
},
end
def on_complete(env)
env[:body] = case env[:status]
when 204
parse('{}')
else
parse(env)
end
end
end
then, you can access the pagination as follows:
model = Model.all
model.metadata[:pagination]
I finally got this working. The trick was to use a global variable in the faraday on_complete - I tried to find a better solution but this was the best I could do. Once again, I got the header code from here. Here's the full guide to how to get pagination working with Her:
First, on my server side, I have the Kaminari gem, and I pass page and per as params to the server from the client. (This is also using ransack for searching)
def search
#search = Order.order('id desc').search(params[:q])
#orders = #search.result(distinct: true).page(params[:page]).per(params[:per])
respond_with #orders.as_json(include: :items)
end
My client makes the request like so:
#orders = Order.search(q: { client_id_eq: #current_user.id }, page: params[:page], per: 3)`
Back on the server, I have this in my ApiController (app controller for api):
protected
def self.set_pagination_headers(name, options = {})
after_filter(options) do |controller|
results = instance_variable_get("##{name}")
headers["X-Pagination"] = {
total_count: results.total_count,
offset_value: results.offset_value
}.to_json
end
end
In the server orders_controller.rb, I set the pagination headers for the search method:
class OrdersController < ApiController
set_pagination_headers :orders, only: [:search]
...
end
Now to receive the headers we need a Faraday middleware in Her on the client.
# config/initializers/her.rb
Her::API.setup url: Constants.api.url do |c|
c.use TokenAuthentication
c.use HeaderParser # <= This is my middleware for headers
c.use Faraday::Request::UrlEncoded
c.use Her::Middleware::DefaultParseJSON
c.use Faraday::Adapter::NetHttp
c.use Faraday::Response::RaiseError
end
# lib/header_parser.rb
# don't forget to load this file in application.rb with something like:
# config.autoload_paths += Dir[File.join(Rails.root, "lib", "*.rb")].each { |l| require l }
class HeaderParser < Faraday::Response::Middleware
def on_complete(env)
unless env[:response_headers]['x-pagination'].nil?
# Set the global var for pagination
$pagination = JSON.parse(env[:response_headers]['x-pagination'], symbolize_names: true)
end
end
end
Now back in your client controller, you have a global variable of hash called $pagination; mine looks like this:
$pagintation = { total_count: 0, offset_value: 0 }`
Finally, I added Kaminari gem to my client app to paginate the array and get those easy pagination links:
#orders = Kaminari.paginate_array(#orders, total_count: $pagination[:total_count]).page(params[:page]).per(params[:per_page])`
I hope this can help someone else, and if anyone knows a better way to do this, let me know!
You can pass header options to Faraday when setting up the connection, see the docs at http://rubydoc.info/gems/faraday/0.8.7/Faraday/Connection:initialize
Sometimes it helps to do a curl request first, esp. use -vv option for verbose output where you will see all headers. (Maybe you can attach some log outputs from the Server too)
You can use e.g. clogger (http://clogger.rubyforge.org/) do monitor header information on the Rails server side

Specify token options for OmniAuth OAuth2 based strategy

I'm creating custom strategy for Nimble.com API. As they're using OAuth, it's pretty simple.
require 'omniauth-oauth2'
module OmniAuth
module Strategies
class Nimble < OmniAuth::Strategies::OAuth2
option :name, "nimble"
option :client_options, {
:site => "https://api.nimble.com",
:authorize_url => '/oauth/authorize',
:token_url => '/oauth/token'
}
# option :access_token_options, {
# :mode => :query,
# :param_name => :access_token
# }
option :provider_ignores_state, true
uid { raw_info['email'] }
info do
{
'uid' => raw_info['email'],
'name' => raw_info['name'],
'email' => raw_info['email']
}
end
extra do
{ 'raw_info' => raw_info }
end
def raw_info
access_token.options[:mode] = :query
access_token.options[:param_name] = :access_token
#raw_info ||= access_token.get('/api/users/myself/', {:parse => :json}).parsed
end
end
end
end
For passing tokens, they need to use access_token parameter in URL. When I specify options in raw_info function directly, as in sample — it's OK.
When I'm trying to specify this options in access_token_options hash (like in commented section) — parameters aren't passing to token. I'm not very good in Ruby, so I didn't figure out from libraries sources — how correctly pass parameters to access_token in OmniAuth OAuth2 descendants.
I'd like to make it "right way", so access_token initialised with correct options, plese someone point me the right way.
Thank you!
I've explored several existing strategies (GitHub, 4SQ), and looks like it's normal practice to directly modify access token options.
So I'll stay with it :)

Resources