How to use Typhoeus::Request object using https - ruby

I'm trying to make an https request using the Typhoeus::Request object and i don't get it working.
The code i'm running is something like this:
url = "https://some.server.com/"
req_opts = {
:method => :get,
:headers => {
"Content-Type"=>"application/json",
"Accept"=>"application/json"
},
:params=>{},
:params_encoding=>nil,
:timeout=>0,
:ssl_verifypeer=>true,
:ssl_verifyhost=>2,
:sslcert=>nil,
:sslkey=>nil,
:verbose=>true
}
request = Typhoeus::Request.new(url, req_opts)
response = request.run
The response i'm getting is this:
HTTP/1.1 302 Found
Location: https://some.server.com:443/
Date: Sat, 27 Apr 2019 02:25:05 GMT
Content-Length: 5
Content-Type: text/plain; charset=utf-8
Why is this happening?

Well it's hard to know because your example is not a reachable url. But 2 things I see is that you are not passing an ssl cert or key. But also 302 indicates a redirect. You can try to follow redirection but your first problem is probably you don't need to set SSL options, why are you?
See if you try the following options:
req_opts = {
:method => :get,
:headers => {
"Content-Type"=>"application/json",
"Accept"=>"application/json"
},
:params=>{},
:params_encoding=>nil,
:timeout=>0,
:followlocation => true,
:ssl_verifypeer=>false,
:ssl_verifyhost=>0,
:verbose=>true
}
See the following sections for more info
https://github.com/typhoeus/typhoeus#following-redirections
https://github.com/typhoeus/typhoeus#ssl

Related

Faraday Gem does not send fileData

I am doing an API automation in Ruby using the Faraday Gem to automate a file upload in my API.
I have the follow problem: I need to upload the file as form-data, but it is not working.
Here's the latest syntax that I'm using:
conn = Faraday.new($api['upload']) do |f|
f.request :multipart
f.adapter Faraday.default_adapter
end
formdata = { :file => Faraday::UploadIO.new('./arquivo/pequeno.pdf', 'file/pdf') }
headers = {'Content-Type' => 'multipart/form-data', 'Authorization' => 'Bearer ' + #token, 'uuidUser' => #uuid}
conn.post('/upload', formdata, headers)
Also, I cannot get the response body or code. I got the error that method body or code or status does not exist.
Do you have any idea what I may be doing wrong?
I put the the logger to see further what is happening, and this is the result:
W, [2019-11-27T11:15:15.385754 #4208] WARN -- : HTTP 500
D, [2019-11-27T11:15:15.386412 #4208] DEBUG -- : "x-content-type-options: nosniff\nx-xss-protection: 1; mode=block\ncache-control: no-cache, no-store, max-age=0, must-revalidate\npragma: no-cache\nexpires: 0\nx-frame-options: DENY\ncontent-type: application/json;charset=UTF-8\ntransfer-encoding: chunked\ndate: Wed, 27 Nov 2019 14:15:14 GMT\nconnection: close\n\n{\"exception\":\"org.springframework.web.multipart.MultipartException\",\"status\":500,\"error\":[\"Failed to parse multipart servlet request; nested exception is java.io.IOException: org.apache.tomcat.util.http.fileupload.FileUploadException: the request was rejected because no multipart boundary was found\"]}"
I solved the problem.
It was on the type of file. Instead of using 'file/pdf' I changed to 'application/pdf' and worked.
For each type of file there is a different way of usage:
pdf:
application/pdf
docx:
application/vnd.openxmlformats-officedocument.wordprocessingml.document

How to Wrap guzzle json post body in array

I am having difficulty with an API and guzzle. The API requires a json content type, and basic auth. The api example request is
POST /endpoint/v1/create?id=1
[
{
"Name": "Test Room"
}
]
My Code:
use GuzzleHttp\Client;
use GuzzleHttp\Psr7\Request;
use GuzzleHttp\Middleware;
use GuzzleHttp\Exception\RequestException;
use GuzzleHttp\Psr7;
$client = new Client(['base_uri' => env('base_uri')]);
$headers = [
'Authorization' => 'Basic',
'Accept' => 'application/json',
'Content-Type' => 'application/json',
];
$uri='/endpoint/v1/create?id=' . env('id');
$payload = ['Name' => 'Test Name'];
$response = $this->client->request('POST', $uri, ['auth' => ['username', 'password'], 'headers' => $headers, 'json' => $payload]);
All seems good to me. I've used guzzle this way in the past. However, The server responds with a "No data was submitted" Message.
The request:
POST /endpoint/v1/create?id=1 HTTP/1.1
User-Agent: GuzzleHttp/6.3.3 curl/7.58.0 PHP/7.2.5-1+ubuntu18.04.1+deb.sury.org+1
Authorization: Basic {Auth basic string goes here}
Host: {host goes here}
Accept: application/json
Content-Type: application/json
{"Name":"Test Name"}
The Response:
HTTP/1.1 400 Bad Request
Cache-Control: no-cache
Pragma: no-cache
Content-Type: application/json; charset=utf-8
Expires: -1
WWW-Authenticate: Basic
Date: Thu, 21 Mar 2019 01:04:21 GMT
Content-Length: 36
{"Message":"No data was submitted."}
EDIT:
I'm able to successfully complete this request in postman. The API responds to [{"Name":"Test Name"}] but not {"Name":"Test Name"}, does anyone know how to replicate that with guzzle?
I was able to solve this problem by wrapping the payload in an array like so:
$payload = [['Name' => 'Test Name']];

Ruby Mechanize, fill dynamic Form / Send JSON (Airbnb calendar)

My goal
I try to update my airbnb calendar using Ruby. For example, here is a URL of a calendar : https://www.airbnb.com/manage-listing/ROOM_ID/calendar
The issue
If you already use Airbnb, to update your calendar, you have to click on the start date then the end date and after that, a form pop-up.
So, when I use Mechanize to get the page content, this form is not loaded and doesn't appears (even the calendar is load dynamically, not able to simulate click too), impossible to use basic Mechanize form filling...
What I did so far
I tried to use the developer tools from Chrome to check the Network. When I update my calendar using Chrome, there is one JSON PUT at https://www.airbnb.com/api/v2/calendars/ROOM_ID/START_DATE/END_DATE?_format=host_calendar&t=1427377357561&key=d306zoyjsyarp7ifhu67rjxn52tv0t20 with some JSON data such as days, availability, price...
My first solution was to tried to reproduce this JSON call with this code :
data = { "event_name" => "calendar",
"event_data" => { "page_uri" => "/manage-listing/ROOM_ID/calendar",
"controller" => "rooms",
"action" => "manage_listing",
"hosting_id" => ROOM_ID,
"start_date" => "2015-03-26",
"end_date" => "2015-03-29",
"available" => true,
"native_price" => 111,
"native_currency" => "EUR"
}
}
page = agent.post 'https://www.airbnb.com/api/v2/calendars/ROOM_ID/2015-03-26/2015-03-29?_format=host_calendar&t=1427374574309&key=d306zoyjsyarp7ifhu67rjxn52tv0t20', data.to_json, {'Content-Type' => 'application/json'}
But I get a 404 response :
Mechanize::ResponseCodeError (404 => Net::HTTPNotFound for https://www.airbnb.com/api/v2/calendars/ROOM_ID/2015-03-26/2015-03-29?_format=host_calendar&t=1427374574309&key=d306zoyjsyarp7ifhu67rjxn52tv0t20 -- unhandled response)
Do you have any suggestions to either send the form even if it is not on the page content, or POST the request with JSON ?
Thanks for your help
Here is the complete JSON call from Chrome :
General
Remote Address:xx.xx.xx.xx:xx
Request URL:https://www.airbnb.com/api/v2/calendars/ROOM_ID/2015-03-26/2015-03-29?_format=host_calendar&t=1427379998507&key=d306zoyjsyarp7ifhu67rjxn52tv0t20&currency=EUR&locale=fr-CA
Request Method:PUT
Status Code:200 OK
Response Headers
cache-control:max-age=0, private, must-revalidate
connection:keep-alive
content-encoding:gzip
content-length:236
content-type:application/json; charset=utf-8
date:Thu, 26 Mar 2015 14:26:46 GMT
etag:W/"10845765865e36a6ccb1541bbda1c2a7"
server:nginx/1.7.7
status:200 OK
status:200 OK
strict-transport-security:max-age=10886400; includeSubdomains
vary:Accept-Encoding
version:HTTP/1.1
x-frame-options:SAMEORIGIN
x-hi-human:The Production Infrastructure team added this header. Come work with us! Email kevin.rice+hiring#airbnb.com
x-ua-compatible:IE=Edge,chrome=1
x-xss-protection:1; mode=block
Request Headers
:host:www.airbnb.com
:method:PUT
:path:/api/v2/calendars/ROOM_ID/2015-03-26/2015-03-29?_format=host_calendar&t=1427379998507&key=d306zoyjsyarp7ifhu67rjxn52tv0t20&currency=EUR&locale=fr-CA
:scheme:https
:version:HTTP/1.1
accept:application/json, text/javascript, */*; q=0.01
accept-encoding:gzip, deflate, sdch
accept-language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4
content-length:59
content-type:application/json
cookie:__ssid=4166c81a-49bd-4826-ac44-08307c5700ca; _csrf_token=V4%24.airbnb.ca%24CL1nNdfYkF0%24ulPyJJJWr1h6CvuBMf32YcXtnZssDud3_CqBQoqXOU0%3D; li=1; roles=0; _airbed_session_id=dfa72c17e6d014f9fd0b9705d097e5d8; flags=4027711488; EPISODES=s=1427377914349&r=https%3A%2F%2Ffr.airbnb.ca%2Fmanage-listing%2F5780104%2Fcalendar; _ga=GA1.2.1981489078.1427272843; fbs=not_authorized; _pt=1--WyJjZmYxZmE4N2RhOTU4NGNhYzhhN2M5YTIyNzkyMDliMDI0YTk1YWEzIl0%3D--2890e7d8df5181677516659fbdc4761e6de82a61; bev=1427272835_bw8KI59ELTQAsMt3; _user_attributes=%7B%22curr%22%3A%22EUR%22%2C%22guest_exchange%22%3A0.9134%2C%22id%22%3A29905162%2C%22hash_user_id%22%3A%22cff1fa87da9584cac8a7c9a2279209b024a95aa3%22%2C%22eid%22%3A%22FBPqvskr4MN1Rnpqf-oY-lG7-VNdCJVSYwUMUtm6YyOXzEpbRvmU9FWTxKNdf0UA%22%2C%22num_msg%22%3A0%2C%22num_h%22%3A1%2C%22name%22%3A%22St%C3%A9phane%22%2C%22is_admin%22%3Afalse%2C%22can_access_photography%22%3Afalse%7D
origin:https://www.airbnb.com
referer:https://www.airbnb.com/manage-listing/ROOM_ID/calendar
user-agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36
x-csrf-token:V4$.airbnb.ca$CL1nNdfYkF0$ulPyJJJWr1h6CvuBMf32YcXtnZssDud3_CqBQoqXOU0=
x-requested-with:XMLHttpRequest
Query String Parameters
_format:host_calendar
t:1427379998507
key:d306zoyjsyarp7ifhu67rjxn52tv0t20
currency:EUR
locale:fr-CA
Request Payload
{availability: "available", daily_price: "999", notes: ""}
availability: "available"
daily_price: "999"
notes: ""
I succeeded to update the calendar of my room I used a JSON PUT request. Here is what I did.
The data looks like :
data = { "availability" => availability,
"daily_price" => price,
"notes" => note
}.to_json
Retrieve the cookie :
cookie_csrf_token = ''
cookie_airbed_session_id = ''
agent.cookie_jar.each do |value|
if value.to_s.include? "_csrf_token"
cookie_csrf_token = value.to_s
elsif value.to_s.include? "_airbed_session_id"
cookie_airbed_session_id = value.to_s
end
end
The headers :
headers = { 'X-CSRF-Token' => URI.unescape(cookie_csrf_token.scan(/=(.*)/).join(",")),
'Content-Type' => 'application/json',
'Cookie' => "#{cookie_csrf_token}; #{cookie_airbed_session_id}"
}
The only cookies you need is csrf_token and airbed_session_id which are related to each other. My mistake was to use the csrf_token from the login page... You can find these cookies in the cookie_jar variable from your Mechanize agent.
After that you will need to construct your URL. The URL has a particular parameter which is called "key". You can retrieve it in a meta tag (id='_bootstrap-layout-init') from your calendar page. Do to that I used Nokogiri combined with some regex :
param_t = Time.now.to_i
noko.xpath("//meta[#id='_bootstrap-layout-init']/#content").each do |attr|
param_key = attr.value[/key":"(.*?)"/, 1]
end
Now you are good to go to update your calendar :
url = "https://www.airbnb.com"
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
# Send the PUT request to update the calendar
res = http.start { |req|
req.send_request('PUT', "/api/v2/calendars/#{room_id}/#{start_date}/#{end_date}?_format=host_calendar&t=#{param_t}&key=#{param_key}", data, headers)
}

Proxy error when using OAuth2 gem

Hi I’m trying to use oauth 2 to connect my app to a local API.
The authorization request succeed and return me a code on my callback address to make the access token request.
Here is my callback code :
def client
OAuth2::Client.new(KEY, SECRET, site: SITE, authorize_url: '/oauth2/auth', token_url: 'http://localhost:8080/oauth2/token')
end
get '/auth/callback' do
access_token = client.auth_code.get_token(params[:code], "redirect_uri" => redirect_uri)
access_token = session[:access_token] = access_token.token
#access_token = access_token
end
In get_token function, I made a pry to check my connection configuration (which is correct):
1.9.3 (#<OAuth2::Client:0x007fb255183e18>):0 > connection
=> #<Faraday::Connection:0x007fb2551817f8
#parallel_manager=nil,
#headers={"User-Agent"=>"Faraday v0.9.0"},
#params={}, #options=#<Faraday::RequestOptions (empty)>,
#ssl=#<Faraday::SSLOptions (empty)>,
#default_parallel_manager=nil,
#builder=#<Faraday::RackBuilder:0x007fb255188760
#handlers=[Faraday::Request::UrlEncoded, Faraday::Adapter::NetHttp, Faraday::Response::Logger]>,
#url_prefix=#<URI::HTTP:0x007fb255187b08 URL:http://localhost:3000/>,
#proxy=#<Faraday::ProxyOptions uri=#<URI::HTTP:0x007fb255186b68 URL:http://proxy_url:port>, user="me", password=« MyPassword">>
1.9.3 (#<OAuth2::Client:0x007fb255183e18>):0 > url
=> "http://localhost:8080/oauth2/token »
1.9.3 (#<OAuth2::Client:0x007f9bdb94cce8>):0 > verb
=> :post
1.9.3 (#<OAuth2::Client:0x007f9bdb94cce8>):0 > opts
=> {
:body => {
"client_id" => "Test",
"client_secret" => "test_secret",
"code" => "d3c4661b11ea5e2752852db72e1ce100f2f1804d",
"grant_type" => "authorization_code",
"redirect_uri" => "http://localhost:4567/auth/callback"
},
:headers => {
"Content-Type" => "application/x-www-form-urlencoded"
},
:parse => nil,
:raise_errors => true
}
Debug informations from request :
I, [2014-08-20T12:01:19.450941 #12792] INFO -- : post http://localhost:8080/oauth2/token
D, [2014-08-20T12:01:19.451040 #12792] DEBUG -- request: User-Agent: "Faraday v0.9.0"
Content-Type: "application/x-www-form-urlencoded"
I, [2014-08-20T12:01:19.451203 #12792] INFO -- Status: 401
D, [2014-08-20T12:01:19.451258 #12792] DEBUG -- response: cache-control: "no-cache"
pragma: "no-cache"
www-authenticate: "NTLM, BASIC realm=\"OUR_DOMAIN_NAME\""
content-type: "text/html; charset=utf-8"
proxy-connection: "close"
set-cookie: "BCSI-CS-925062b0249dae29=2; Path=/"
connection: "close"
content-length: "706"
x-rbt-optimized-by: "riv-olo (RiOS 8.5.1) IK"
OAuth2::Error - Nice page from my proxy telling me that I’m not authentified on the domain
When I make a POST request with POSTMAN REST Client using my authorization code, everything is fine and the API return me an access_token.
I don’t understand why the request is passing by the proxy because all my apps and API are on my localhost.
Could someone tell me why this error append using my local app ?
Thanks
As my postman request worked and not those of my app, I used Wireshark to see the difference between them.
In Oauth, my proxy parameters were used by default. So my request went on the proxy IP.
And on Postman, requests went directly on localhost.
So I changed my client definition to force empty proxy parameters :
PROXY_URI = ''
PROXY_USER = ''
PROXY_PASSWORD = ''
def client
OAuth2::Client.new(KEY, SECRET, site: SITE, authorize_url: '/oauth2/auth', token_url: 'http://localhost:8080/oauth2/token', connection_opts: {
:proxy => {:uri => PROXY_URI,
:user => PROXY_USER,
:password => PROXY_PASSWORD}})
end
Now request are send to localhost as expected.
It's ok for local tests, but now I'll need to make conditional assignment of my proxy variables.
Why the GET method were sent to localhost and not POST method ? It's still a mystery !
It sounds like you're trying to hit localhost via your proxy, which won't work. If you're using OS X:
Go to System Preferences > Network.
Select the connection you're using on the left (eg. "Wi-Fi" or "Ethernet").
Click "Advanced..." at the bottom right.
Go to the "Proxies" tab.
In the box at the bottom ("Bypass proxy settings"), add 127.0.0.1, localhost.
Click "OK", then "Apply".
Once you've done that, OS X shouldn't route requests to localhost via your proxy, and Postman should work fine.

OAuth gem not signing requests

I'm using the ruby gem for OAuth (http://oauth.rubyforge.org/) and I can't get it to create the authorization header for the provider I'm attempting to hit.
Here is my code:
consumer = OAuth::Consumer.new(auth[:consumer_key], auth[:consumer_secret], {
:site => 'http://api.rdio.com',
:scheme => :header
})
access_token = OAuth::AccessToken.new(consumer)
ap access_token.post('/1', :method => 'search', :query => 'Robert', :types => 'User')
When the requests happens, The header is not present in the call.
#<Net::HTTP::Post:0x7fbf149e91e0
#body_data = nil,
#header = {
"accept" => [
[0] "*/*"
],
"user-agent" => [
[0] "Ruby"
],
"content-length" => [
[0] "0"
],
"content-type" => [
[0] "application/x-www-form-urlencoded"
]
},
The header I'm referring to is the one that looks like this:
OAuth oauth_nonce=\"225579211881198842005988698334675835446\", oauth_signature_method=\"HMAC-SHA1\", oauth_token=\"token_411a7f\", oauth_timestamp=\"1199645624\", oauth_consumer_key=\"consumer_key_86cad9\", oauth_signature=\"1oO2izFav1GP4kEH2EskwXkCRFg%3D\", oauth_version=\"1.0\"
Looks like you are trying to do 2-legged oauth. See if this code works for you.
Edit: Updated Code Sample
gem 'oauth'
require 'oauth'
require 'net/http'
consumer = OAuth::Consumer.new('ENTER_KEY', 'ENTER_SECRET', {
:site => 'http://api.rdio.com',
:scheme => :header
})
resp = consumer.request(:post, '/1/search', nil, {}, 'method=search&query=Robert&types=User', { 'Content-Type' => 'application/x-www-form-urlencoded' })
puts resp.code + "\r\n"
puts resp.body
Edit: Added captured http stream
POST /1/search HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: */*
User-Agent: OAuth gem v0.4.5
Content-Length: 37
Authorization: OAuth oauth_consumer_key="REDACTED_KEY", oauth_nonce="dwp8m2TGPHQNx3A7imLi7OkAULL7c0IWbTKefPXCsAY", oauth_signature="LxDZn6UNFLY%2FaXItu6MPK5a11js%3D", oauth_signature_method="HMAC-SHA1", oauth_timestamp="1330193449", oauth_version="1.0"
Connection: close
Host: api.rdio.com
method=search&query=Robert&types=UserHTTP/1.1 200 OK
X-Mashery-Responder: mashery-web1.LAX
Content-Type: application/json
Vary: Accept-Encoding
Vary: Accept-Language, Cookie
Content-Language: en
Cache-Control: no-cache
X-Version: 11.1
Accept-Ranges: bytes
Date: Sat, 25 Feb 2012 18:10:50 GMT
Server: Mashery Proxy
Content-Length: 2763
Connection: close
{"status": "ok", "result": {"person_count": 9603, "track_count": 93409, "number_results": 200, "playlist_count": 205, "results": ***TRUNCATED RESULTS FOR BREVITY***

Resources