Hello stack overflow people.
Does someone know of a code example to make the ruby httpclient do an async post? It has a method but it looks like it just gives you a connection back you have to keep checking, which I assume would still be blocking. I did not see a way to "fire and forget" or just pass a method that it could call later in a separate thread while the rest of my code kept running.
thanks,
craig
This sounds like you're programming in evented style. Maybe you are even using eventmachine? You don't say so, but in the case you do, this project: https://github.com/eventmachine/em-http-request will let you do something close:
EventMachine.run {
http = EventMachine::HttpRequest.new('http://127.0.0.1/').get :query => {'keyname' => 'value'}
http.callback {
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
}
}
Related
I would like to test an API client in rspec.
I'm currently mocking Typhoeus - but I wondered if there is a more end-to-end way of doing this. Essentially, what I would like is something like:
it "makes a connection to the server" do
MockServer.new do |server|
subject.url = server.url
subject.run!
expect(server.last_request.params).to eq({some: "params"})
expect(server.last_request.headers).to include({"X-whatty-what" => "yepyep"})
end
end
Maybe I could do this with Sinatra, or even rack directly... Anybody done something like this before?
Stub the requests using Webmock.
It is recommended to stub every request you expect to make separately so you know exactly what is requested, but you can also use a syntax similar to what you asked for.
it "makes a connection to the server" do
stub_request(:any, "www.example.com")
subject.url = "www.example.com"
subject.run!
expect(
a_request(:post, "www.example.com").with(
body: { some: "params" },
headers: { "X-whatty-what" => "yepyep" }
)
).to have_been_made
end
The webmock docs have plenty of good examples, take a look.
I am using Typhoeus and would like to make a single request without blocking for the response. Later on, I might check the response, or I might not. The point is I don't want the code execution to wait for the response.
Is there a way to do this built-in to Typhoeus?
Otherwise I guess I have to use threads and do it myself?
You could try using a thread:
response = nil
request_thread = Thread.new {
# Set up the request object here
response = request.response
}
From there you can check response == nil to see if the request has been made yet, and you can call request_thread.join to block until the thread is done executing.
I would suggest looking into the 'unirest' gem for Ruby.
As far as I am aware, Typhoeus blocks on the 'hydra.run' call
With Unirest, it does not block on the get / post / put / etc call, but continues to run. If you want, you can store the 'object' in a hash or an array with an identifier to retrieve later, like so:
identifier_requests['id'] = Unirest.post(url,headers: headers, parameters: param, auth: auth)
Then to block, or retrieve responses, use one of the calls on the response object:
response_code = (identifier_requests['id']).code
response.body
http://unirest.io/ruby.html
Typhoeus has non-blocking calls built-in. From their docs:
request = Typhoeus::Request.new("www.example.com", followlocation: true)
request.on_complete do |response|
if response.success?
# hell yeah
elsif response.timed_out?
# aw hell no
log("got a time out")
elsif response.code == 0
# Could not get an http response, something's wrong.
log(response.return_message)
else
# Received a non-successful http response.
log("HTTP request failed: " + response.code.to_s)
end
end
request.run
This is from their docs at https://github.com/typhoeus/typhoeus
I have been tasked with creating a Ruby API that retrieves youtube URL's. However, I am not sure of the proper way to create an 'API'... I did the following code below as a Sinatra server that serves up JSON, but what exactly would be the definition of an API and would this qualify as one? If this is not an API, how can I make in an API? Thanks in advance.
require 'open-uri'
require 'json'
require 'sinatra'
# get user input
puts "Please enter a search (seperate words by commas):"
search_input = gets.chomp
puts
puts "Performing search on YOUTUBE ... go to '/videos' API endpoint to see the results and use the output"
puts
# define query parameters
api_key = 'my_key_here'
search_url = 'https://www.googleapis.com/youtube/v3/search'
params = {
part: 'snippet',
q: search_input,
type: 'video',
videoCaption: 'closedCaption',
key: api_key
}
# use search_url and query parameters to construct a url, then open and parse the result
uri = URI.parse(search_url)
uri.query = URI.encode_www_form(params)
result = JSON.parse(open(uri).read)
# class to define attributes of each video and format into eventual json
class Video
attr_accessor :title, :description, :url
def initialize
#title = nil
#description = nil
#url = nil
end
def to_hash
{
'title' => #title,
'description' => #description,
'url' => #url
}
end
def to_json
self.to_hash.to_json
end
end
# create an array with top 3 search results
results_array = []
result["items"].take(3).each do |video|
#video = Video.new
#video.title = video["snippet"]["title"]
#video.description = video["snippet"]["description"]
#video.url = video["snippet"]["thumbnails"]["default"]["url"]
results_array << #video.to_json.gsub!(/\"/, '\'')
end
# define the API endpoint
get '/videos' do
results_array.to_json
end
An "API = Application Program Interface" is, simply, something that another program can reliably use to get a job done, without having to busy its little head about exactly how the job is done.
Perhaps the simplest thing to do now, if possible, is to go back to the person who "tasked" you with this task, and to ask him/her, "well, what do you have in mind?" The best API that you can design, in this case, will be the one that is most convenient for the people (who are writing the programs which ...) will actually have to use it. "Don't guess. Ask!"
A very common strategy for an API, in a language like Ruby, is to define a class which represents "this application's connection to this service." Anyone who wants to use the API does so by calling some function which will return a new instance of this class. Thereafter, the program uses this object to issue and handle requests.
The requests, also, are objects. To issue a request, you first ask the API-connection object to give you a new request-object. You then fill-out the request with whatever particulars, then tell the request object to "go!" At some point in the future, and by some appropriate means (such as a callback ...) the request-object informs you that it succeeded or that it failed.
"A whole lot of voodoo-magic might have taken place," between the request object and the connection object which spawned it, but the client does not have to care. And that, most of all, is the objective of any API. "It Just Works.™"
I think they want you to create a third-party library. Imagine you are schizophrenic for a while.
Joe wants to build a Sinatra application to list some YouTube videos, but he is lazy and he does not want to do the dirty work, he just wants to drop something in, give it some credentials, ask for urls and use them, finito.
Joe asks Bob to implement it for him and he gives him his requirements: "Bob, I need YouTube library. I need it to do:"
# Please note that I don't know how YouTube API works, just guessing.
client = YouTube.new(api_key: 'hola')
video_urls = client.videos # => ['https://...', 'https://...', ...]
And Bob says "OK." end spends a day in his interactive console.
So first, you should figure out how you are going to use your not-yet-existing lib, if you can – sometimes you just don't know yet.
Next, build that library based on the requirements, then drop it in your Sinatra app and you're done. Does that help?
I'm looking for an alternate Ruby HTTP library that makes multiple HTTP calls simultaneously and performs better than the core Net::HTTP library.
You are probably looking for Typhoeus.
Typhoeus runs HTTP requests in parallel while cleanly encapsulating handling logic
https://github.com/typhoeus/typhoeus
Why do you need a networking library handle parallelism? That is exactly what threads are for.
require "open-uri"
fetcher = lambda do |uri|
puts "Started fetching #{uri}"
puts open(uri).read
puts "Stopped fetching #{uri}"
end
thread1 = Thread.new("http://localhost:9292", &fetcher)
thread2 = Thread.new("http://localhost:9293", &fetcher)
thread1.join
thread2.join
Also, I don't understand what do you mean by "performs better". Core libraries are usually good enough to be in the core. Do you have any problems with Net::HTTP?
You can use the Parallel gem, it should work with any Ruby HTTP library.
Not sure if it performs better then Typhoeus, BUT you could use Eventmacheine + em-http-request. There is an example for sending multiple requests.
require 'eventmachine'
require 'em-http'
EventMachine.run {
multi = EventMachine::MultiRequest.new
reqs = [
'http://google.com/',
'http://google.ca:81/'
]
reqs.each_with_index do |url, idx|
http = EventMachine::HttpRequest.new(url, :connect_timeout => 1)
req = http.get
multi.add idx, req
end
multi.callback do
p multi.responses[:callback].size
p multi.responses[:errback].size
EventMachine.stop
end
}
https://github.com/igrigorik/em-http-request
I'm new to EM and am following this example:
EventMachine.run {
http = EventMachine::HttpRequest.new('http://google.com/').get :query => {'keyname' => 'value'}
http.errback { p 'Uh oh'; EM.stop }
http.callback {
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
}
}
I want to do something similar.
I want to fetch "JavaScript Object Notation" (JSON) files from several different web servers, in parallel.
I cannot find the way how to store all these JSON files in a common variable, so that I can do some calculations about them afterwards, something like in every request I store the JSON in a global array.
You want the requests to be in parallel and to process them after all have been completed?
You can use EventMachine::MultiRequest from em-http-request. The wiki has documentation on issuing parallel requests, see "Synchronizing with Multi interface".
You should add our code to multi.callback and you will receive an array of requests.