Anyone got a working example of using ruby to post to a presigned URL on s3
I have used aws-sdk and right_aws both.
Here is the code to do this.
require 'rubygems'
require 'aws-sdk'
require 'right_aws'
require 'net/http'
require 'uri'
require 'rack'
access_key_id = 'AAAAAAAAAAAAAAAAA'
secret_access_key = 'ASDFASDFAS4646ASDFSAFASDFASDFSADF'
s3 = AWS::S3.new( :access_key_id => access_key_id, :secret_access_key => secret_access_key)
right_s3 = RightAws::S3Interface.new(access_key_id, secret_access_key, {:multi_thread => true, :logger => nil} )
bucket_name = 'your-bucket-name'
key = "your-file-name.ext"
right_url = right_s3.put_link(bucket_name, key)
right_scan_command = "curl -I --upload-file #{key} '#{right_url.to_s}'"
system(right_scan_command)
bucket = s3.buckets[bucket_name]
form = bucket.presigned_post(:key => key)
uri = URI(form.url.to_s + '/' + key)
uri.query = Rack::Utils.build_query(form.fields)
scan_command = "curl -I --upload-file #{key} '#{uri.to_s}'"
system(scan_command)
Can you provide more information on how a "presigned URL" works? Is it like this:
AWS::S3::S3Object.url_for(self.full_filename,
self.bucket_name, {
:use_ssl => true,
:expires_in => ttl_seconds
})
I use this code to send authenticated clients the URL to their S3 file. I believe this is the "presigned URL" that you're asking about. I haven't used this code for a PUT, so I'm not exactly sure if it's right for you, but it might get you close.
I know this is an older question, but I was wondering the same thing and found an elegant solution in the AWS S3 Documentation.
require 'net/http'
file = "somefile.ext"
url = URI.parse(presigned_url)
Net::HTTP.start(url.host) do |http|
http.send_request("PUT", url.request_uri, File.read(file), {"content-type" => "",})
end
This worked great for my Device Farm uploads.
Does anything on the s3 library page cover what you need? There are loads of examples there.
There are some generic REST libraries for Ruby; Google for "ruby rest client". See also HTTParty.
I've managed to sort it out. Turns out the HTTP:Net in Ruby is has some short comings. Lot of Monkeypatch later I got it working.. More details when I have time. thank
Related
I am trying to get the bandwidth data for all the softlayer servers under my account.
Thanks to account_servers.rb I am able to get the server id for all the servers. Now I would like to get the Bandwidth used by the servers for a particular time frame. The data that I am interested is
http://sldn.softlayer.com/reference/datatypes/SoftLayer_Metric_Tracking_Object_Bandwidth_Summary
.
I tried to get information using softlayer_client.service_named("Metric_Tracking_Object_Bandwidth_Summary"). Unfortunately I am not able to get the details.
I did find a java code, but I am interested in ruby code. Can someone please guide me to get the server bandwith summary?
Getting bandWidth data in SL
Please, try the following Ruby examples:
require 'rubygems'
require 'softlayer_api'
server_id = 11498369
# Your SoftLayer API username.
SL_API_USERNAME = 'set me'
# Your SoftLayer API key.
SL_API_KEY = 'set me'
softlayer_client = SoftLayer::Client.new(:username => SL_API_USERNAME,
:api_key => SL_API_KEY)
vsi_service = softlayer_client.service_named('SoftLayer_Virtual_Guest')
metric_tracking_object_id = vsi_service.object_with_id(server_id).getMetricTrackingObjectId
metric_service = softlayer_client.service_named('SoftLayer_Metric_Tracking_Object')
service_ref = metric_service.object_with_id(metric_tracking_object_id)
begin
object_template = [{
'keyName' => 'PUBLICOUT',
'summaryType' => 'sum'
}]
result = service_ref.getSummaryData('2016-03-29T00:00:00','2016-03-30T00:00:00',object_template,600)
puts result.inspect
rescue => e
puts 'Error when executing the script...'
$stdout.print(e.inspect)
end
References:
SoftLayer_Metric_Tracking_Object::getSummaryData
SoftLayer_Virtual_Guest::getMetricTrackingObjectId
Second example using SoftLayer_Virtual_Gues::getBandwidthDataByDate:
require 'rubygems'
require 'softlayer_api'
require 'pp'
require 'date'
# Set the server id that you wish to get Bandwidth information.
server_id = 11498369
softlayer_client = SoftLayer::Client.new(:username => 'set me',
:api_key => 'set me')
server = SoftLayer::VirtualServer.server_with_id(server_id, :client => softlayer_client)
get_bandwidth_data_by_date = server.service.getBandwidthDataByDate('2016-03-29T00:00:00','2016-03-30T00:00:00','public')
pp('getBandwidthDataByDate: ', get_bandwidth_data_by_date)
References:
SoftLayer_Virtual_Guest::getBandwidthDataByDate
Disclamer: I created my own Ruby SoftLayer client, you can check it at http://github.com/zertico/softlayer specially for situations like this one where you want to access some specific data (and I'm not SoftLayer staff ;) )
If you'd like to give it a try the code that solves your problem is
ps: I'm considering you are manipulating a SoftLayer_Hardware_Server, right?
hardware = Softlayer::Hardware::Server.find(123)
hardware.get_current_bandwidth_summary
mask = 'mask[currentBandwidthSummary]'
hardware = Softlayer::Hardware::Server.mask(mask).find(123)
hardware.current_bandwidth_summary
You will access a ruby object like this one:
=> #<Softlayer::Metric::Tracking::Object::Bandwidth::Summary:0x007ff74b683540
#allocation_amount="1234",
#allocation_id=111111,
#amount_out="12.34567",
#average_daily_usage="1.23",
#currently_over_allocation_flag=0,
#id=1111111,
#outbound_bandwidth_amount="123.45678",
#projected_bandwidth_usage="123.45",
#projected_over_allocation_flag=0>
Hope it can helps you, comment if you have any doubt about the client usage
Want to scrape a bunch of tweets via the Twitter API, as an output I get cURL command, something like that
curl --get 'https://api.twitter.com/1.1/search/tweets.json' --data 'q=football' --header 'Authorization: OAuth oauth_consumer_key="**hidden**", oauth_nonce="**hidden**", oauth_signature="**hidden**", oauth_signature_method="HMAC-SHA1", oauth_timestamp="**hidden**", oauth_token="**hidden**", oauth_version="1.0"' --verbose
My question, is there a way to use this command into a Ruby script to scrape the tweets ?
Using the Twitter gem available here http://rdoc.info/gems/twitter with the following code you can get all the tweets from a ruby script.
require 'twitter'
client = Twitter::REST::Client.new do |config|
config.consumer_key ="hidden"
config.consumer_secret ="hidden"
config.access_token ="hidden"
config.access_token_secret ="hidden"
end
client.search("football").collect do |tweet|
puts tweet.text
end
you can wrap it in backticks and get the output like from any unix(like) command
script.rb
cmd=`echo 'hello world'`
puts cmd
ouptput: hello world
It is better to use existing API as #Hunter McMillen had suggested, but if you want to perform http-requests yourself, you can use net/http lib. Example below:
require 'net/http'
uri = URI('http://example.com/index.html')
params = { :limit => 10, :page => 3 }
uri.query = URI.encode_www_form(params)
res = Net::HTTP.get_response(uri)
puts res.body if res.is_a?(Net::HTTPSuccess)
Here is the info on how to set headers.
I'm trying to build a test that will allow me to exercise FilePicker.io security. The code is run as:
ruby test.rb [file handle]
and the result is the query string that I can append to a FilePicker URL. I'm pretty sure my policy is getting read properly, but my signature isn't. Can someone tell me what I'm doing wrong? Here's the code:
require 'rubygems'
require 'base64'
require 'cgi'
require 'openssl'
require 'json'
handle = ARGV[0]
expiry = Time::now.to_i + 3600
policy = {:handle=>handle, :expiry=>expiry, :call=>["pick","read", "stat"]}.to_json
puts policy
puts "\n"
secret = 'SECRET'
encoded_policy = CGI.escape(Base64.encode64(policy))
signature = OpenSSL::HMAC.hexdigest('sha256', secret, encoded_policy)
puts "?signature=#{signature}&policy=#{encoded_policy}"
The trick is to use Base64.urlsafe_encode64 instead of CGI.escape:
require 'rubygems'
require 'base64'
require 'cgi'
require 'openssl'
require 'json'
handle = ARGV[0]
expiry = Time::now.to_i + 3600
policy = {:handle=>handle, :expiry=>expiry}.to_json
puts policy
puts "\n"
secret = 'SECRET'
encoded_policy = Base64.urlsafe_encode64(policy)
signature = OpenSSL::HMAC.hexdigest('sha256', secret, encoded_policy)
puts "?signature=#{signature}&policy=#{encoded_policy}"
When tested with the sample values for expiry, handle, and secret in the Filepicker.io docs it returns same values as the python example.
I resolved this in my Ruby 1.8 environment by removing the CGI.escape and gsubbing out the newline:
Base64.encode64(policy).gsub("\n","")
elevenarms's answer is the best for Ruby 1.9 users, but you have to do something a bit kludgy like the above for Ruby 1.8. I'll accept his answer nonetheless, since most of us are or shortly will be in 1.9 these days.
I'd like to use the Readability API through the Readit gem; however, I've been having some trouble trying to get an access token through XAuth. Here's the code that I have:
require 'highline/import'
require 'yaml'
require 'oauth'
require 'readit'
config = YAML.load_file("config/readability.yaml")
uname = ask ("Username: ")
passwd = ask ("Password: ") {|q| q.echo = false}
consumer = OAuth::Consumer.new(config["-consumer_key"], config["-consumer_secret"], :site => "https://www.readability.com/api/rest/v1/oauth/access_token/")
access_token = consumer.get_access_token(nil, {}, {:x_auth_mode => 'client_auth', :x_auth_username => uname, :x_auth_password => passwd})
However, when I try to run this, I get the following:
/Users/mike/.rvm/gems/ruby-1.9.3-p125/gems/oauth-0.4.5/lib/oauth/consumer.rb:219:in `token_request': 404 NOT FOUND (OAuth::Unauthorized)
from /Users/mike/.rvm/gems/ruby-1.9.3-p125/gems/oauth-0.4.5/lib/oauth/consumer.rb:109:in `get_access_token'
from instab.rb:11:in `<main>'
Can someone explain to me what I am doing wrong?
You should write as follows:
consumer = ::OAuth::Consumer.new(Readit::Config.consumer_key,Readit::Config.consumer_secret,:site=>"https://www.readability.com/", :access_token_path => "/api/rest/v1/oauth/access_token/")
I need to send a POST request as an XML string but I get odd results. The code:
require 'rest_client'
response = RestClient.post "http://127.0.0.1:2000", "<tag1>text</tag1>", :content_type => "text/xml"
I expect to receive "<tag1>text</tag1>" as the parameter on the request server. Instead, I get "tag1"=>"text". It converts the XML to a hash. Why is that? Any way around this?
Try this:
response = RestClient.post "http://127.0.0.1:2000",
"<tag1>text</tag1>",
{:accept => :xml, :content_type => :xml}
I think you just needed to specify the ":accept" to let it know you wanted to receive it in the XML format. Assuming it's your own server, you can debug on the server and see the request format used is probably html.
Hope that helps.
Instead of using RestClient, use Ruby's built-in Open::URI for GET requests or something like Net::HTTP or the incredibly powerful Typhoeus:
uri = URI('http://www.example.com/search.cgi')
res = Net::HTTP.post_form(uri, 'q' => 'ruby', 'max' => '50')
In Typhoeus, you'd use:
res = Typhoeus::Request.post(
'http://localhost:3000/posts',
:params => {
:title => 'test post',
:content => 'this is my test'
}
)
Your resulting page, if it's in XML will be easy to parse using Nokogiri:
doc = Nokogiri::XML(res.body)
At that point you'll have a fully parsed DOM, ready to be searched, using Nokogiri's search methods, such as search and at, or any of their related methods.