Faraday middleware know when something was redirected - ruby

I'm using the following code to make a request and follow redirects:
require 'faraday'
require 'faraday_middleware'
conn = Faraday.new() do |f|
f.use FaradayMiddleware::FollowRedirects, limit: 5
f.adapter Faraday.default_adapter
end
resp = conn.get('http://www.example.com/redirect')
resp.status
This code outputs 200 because it followed the redirect, which is great. But is there anyway to know if a redirect existed or not? something like resp.redirected which is set to true if a redirect was followed or false if no redirect was followed?
I didn't see anything obvious in the FollowRedirects code.
Will I need to write my own custom middleware if I want to know this? Does anyone know of middleware out there that might do this already?

I found a solution. You can pass a callback to FaradayMiddleware::FollowRedirects. The callback should live in a hash that the FollowRedirects takes a second parameter. Since we have to use the use function for middleware you can pass the hash as a second parameter to that function.
redirects_opts = {}
# Callback function for FaradayMiddleware::FollowRedirects
# will only be called if redirected to another url
redirects_opts[:callback] = proc do |old_response, new_response|
# you can pull the new redirected URL with this line of code.
# since you have access to the new url you can make a variable or
# instance vairable to keep track of the current URL
puts 'new url', new_response.url
end
#base_client = Faraday.new(url: url, ssl: { verify: true, verify_mode: 0 }) do |c|
c.request :multipart
c.request :url_encoded
c.response :json, content_type: /\bjson$/
c.use FaradayMiddleware::FollowRedirects, redirects_opts //<- pass hash here
c.adapter Faraday.default_adapter
end

Actually, I think I just found the answer based on the post here: https://stackoverflow.com/a/20818142/4701287
I need to compare the original url i passed in with the resulting url. Extending my example from above:
original_url = 'http://www.example.com/redirect'
resp = conn.get(original_url)
was_redirected = (original_url == resp.to_hash[:url].to_s)

Related

Ruby Http gem (http client) inside Sinatra POST

I am trying to relay a GET request so when the user does a POST request to the server then the server does another GET request to some other URL and returns the result.
The issue is that when I use puts to print the result I see the correct result that I am expecting but the last line (I believe in ruby the last line of the function automatically returns) does not respond to the POST request (it returns empty response). Coming from JavaScript I believe it is doing an asynchronous GET call and POST response is not waiting until GET client is finished.
Any help would be appreciated.
require 'rubygems'
require 'sinatra'
require "http"
my_app_root = File.expand_path( File.dirname(__FILE__) + '/..' )
set :port, 80
set :bind, '0.0.0.0'
set :public_dir, my_app_root + '/public'
post "/weather" do
puts HTTP.get('https://www.metaweather.com/api/location/search/?query=milwaukee') # prints correct result
HTTP.get('https://www.metaweather.com/api/location/search/?query=milwaukee') # response of POST method is empty string!
end
link to http gem
Changes to
post "/weather" do
puts HTTP.get('https://www.metaweather.com/api/location/search/?query=milwaukee') # prints correct result
HTTP.get('https://www.metaweather.com/api/location/search/?query=milwaukee').to_s
end
The HTTP.get method returns a HTTP::Response object rather than a String object.
From the source code only specific types of object can be returned.
res = [res] if Integer === res or String === res
if Array === res and Integer === res.first
res = res.dup
status(res.shift)
body(res.pop)
headers(*res)
elsif res.respond_to? :each
body res
end
nil # avoid double setting the same response tuple twice

How do I follow URL redirection?

I have a URL and I need to retrieve the URL it redirects to (the number of redirections is arbitrary).
One real example I'm working on is:
https://www.google.com/url?q=http://m.zynga.com/about/privacy-center/privacy-policy&sa=D&usg=AFQjCNESJyXBeZenALhKWb52N1vHouAd5Q
which will eventually redirect to:
http://company.zynga.com/privacy/policy
which is the URL I'm interested in.
I tried with open-uri as follows:
privacy_url = "https://www.google.com/url?q=http://m.zynga.com/about/privacy-center/privacy-policy&sa=D&usg=AFQjCNESJyXBeZenALhKWb52N1vHouAd5Q"
final_url = nil
open(privacy_url) do |h|
puts "Redirecting to #{h.base_uri}"
final_url = h.base_uri
end
but I keep getting the original URL back, meaning that final_url is equal to privacy_url.
Is there any way to follow this kind of redirection and programmatically access the resulting URL?
I finally made it, using the Mechanize gem. They key is to enable the follow_meta_refresh options, which is disabled by default.
Here's how
require 'mechanize'
browser = Mechanize.new
browser.follow_meta_refresh = true
start_url = "https://www.google.com/url?q=http://m.zynga.com/about/privacy-center/privacy-policy&sa=D&usg=AFQjCNESJyXBeZenALhKWb52N1vHouAd5Q"
final_url = nil
browser.get(start_url) do |page|
final_url = page.uri.to_s
end
puts final_url # => http://company.zynga.com/privacy/policy

Using Net::HTTP, how can you store the url that you queried?

require 'net/http'
require 'uri'
#url = 'http://foobar.com'
#query_string = {foo1: 'bar1', foo2: 'bar2'}
#post_data = Net::HTTP.post_form(URI.parse(#url), #query_string)
#request = # ? How can I get the request URL + query?
#response = #post_data.body
Does anyone know how you can get the actual URL that you queried, not just the response?
i.e. I want to store this in a variable to record what was sent:
http://foobar.com?foo1=bar1&foo2=bar2
It's not getting it from the response itself (I don't think there is a way to do this with net/http), but you can get the variable you want by doing this:
#request = URI.parse(#url).tap{|x|x.query=URI.encode_www_form(#query_string)}.to_s
# => "http://foobar.com?foo1=bar1&foo2=bar2"
I believe you want to look into Ruby's URI Module:
http://www.ruby-doc.org/stdlib-2.0/libdoc/uri/rdoc/URI.html
Not sure, but you could probably get away with something like:
query_string = URI.encode_www_form(#query_string).to_s
record_of_uri = "#{#url}/?#{query_string}"
Noting that a post request doesn't send it's params in the url, to compose #request as described use:
#request = URI.escape(#url + '?' + URI.encode_www_form(#query_string))

How Do I search Twitter for a word with Ruby?

I have written code in Ruby that will display the timeline for a specific user. I would like to write code to be able to just search twitter to just find every user that has mentioned a word. My code is currently:
require 'rubygems'
require 'oauth'
require 'json'
# Now you will fetch /1.1/statuses/user_timeline.json,
# returns a list of public Tweets from the specified
# account.
baseurl = "https://api.twitter.com"
path = "/1.1/statuses/user_timeline.json"
query = URI.encode_www_form(
"q" => "Obama"
)
address = URI("#{baseurl}#{path}?#{query}")
request = Net::HTTP::Get.new address.request_uri
# Print data about a list of Tweets
def print_timeline(tweets)
tweets.each do |tweet|
require 'date'
d = DateTime.parse(tweet['created_at'])
puts " #{tweet['text'].delete ","} , #{d.strftime('%d.%m.%y')} , #{tweet['user']['name']}, #{tweet['id']}"
end
end
# Set up HTTP.
http = Net::HTTP.new address.host, address.port
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
# If you entered your credentials in the first
# exercise, no need to enter them again here. The
# ||= operator will only assign these values if
# they are not already set.
consumer_key = OAuth::Consumer.new(
"")
access_token = OAuth::Token.new(
"")
# Issue the request.
request.oauth! http, consumer_key, access_token
http.start
response = http.request request
# Parse and print the Tweet if the response code was 200
tweets = nil
puts "Text,Date,Name,id"
if response.code == '200' then
tweets = JSON.parse(response.body)
print_timeline(tweets)
end
nil
How would I possibly change this code to search all of twitter for a specific word?
The easiest approach would be to use 'Twitter' gem. Refer to this Link for more information and the result type of the search results. Once you have all the correct authorization attribute in place (oAuth-Token,oAuth-secret, etc) you should be able to search as
Twitter.search('Obama')
or
Twitter.search('Obama', options = {})
Let us know, if that worked for you or not.
p.s. - Please mark the post as answered if it helped you. Else put a comment back with what is missing.
The Twitter API suggests the URI your should be using for global search is https://api.twitter.com/1.1/search/tweets.json and this means:
Your base_url component would be https://api.twitter.com
Your path component would be /1.1/search/tweets.json
Your query component would be the text you are searching for.
The query part takes a lot of values depending upon the API spec. Refer to the specification and you can change it as per your requirement.
Tip: Try to use irb (I'd recommend pry) REPL which makes it a lot easier to explore APIs. Also, checkout the Faraday gem which can be easier to use than the default HTTP library in Ruby IMO.

Retrieving full request string using Ruby curl

I intend to send a request like the following:
c = Curl::Easy.http_post("https://example.com", json_string
) do |curl|
curl.headers['Accept'] = 'application/json'
curl.headers['Content-Type'] = 'application/json'
curl.headers['Api-Version'] = '2.2'
end
I want to log the exact http request that is being made. Is there a way to get the actual request that was made (base path, query parameters, headers and body)?
The on_debug handler has helped me before. In your example you could try:
curl.on_debug do |type, data|
puts type, data
end
You can reach the solution in differents manner:
Inside your block you can put:
curl.verbose = true # that prints a detailed output of the connection
Or outside the block:
c.url # return the url with queries
c.total_time # retrieve the total time for the prev transfer (name resolving, TCP,...)
c.header_str # return the response header
c.headers # return your call header
c.body_str # return the body of the response
Remember to call c.perform (if not yet performed) before call these methods.
Many more option can be found here: http://curb.rubyforge.org/classes/Curl/Easy.html#M000001

Resources