Is there a ruby library, with which I can request the web server to return only the header response and no content? This will help me speed up a script in which all I care is the response code.
I am now using this
Net::HTTP.get_response(URI(url))
but the server generates all the assert files and so on, which I do not want.
You might use rest-client gem and in particular head method.
http://www.rubydoc.info/github/rest-client/rest-client/RestClient#head-class_method
This can be done using 'Net::HTTP::Head' in the net-http library. other libraries also support it, just remember to look for the HEAD method instead of the Get method
There is a library in Python that I love called "Requests". Requests is a HTTP client build on urllib3. "requests doc".
I am looking for something similar in Ruby. Basically what I need is:
Upload files support (multipart/form-data).
Easy get/post.
Cookies can be passed from a response object to a request object (build manually login script).
Stable and Flexible.
Sessions support (to not have to handle cookies manually if we don't have too).
I've looked at Typhoeus, but the code example in the home page doesn't work; they have moved code along and the get method is not longer directly accessible like that, so it's not starting well. Curb seems nice and I like cURL, there is also rest-client, which seems popular, and em-http seems pretty fast according to benchmark. There is a also Patron and curb-fu, which I haven't have the time to try. And, of course, Net:HTTP. But, it doesn't seem to have a mainstream solution that everyone points to.
I think a lot of people have been in my situation and I wonder what they have choosen and why?
The author of the comparison is the author of httpclient, but from the looks of it the comparison is fair.
For a more narrative style with some explanation of the matrix, see http://www.slideshare.net/HiroshiNakamura/rubyhttp-clients-comparison from the same author.
The comparison comes out partly in favor of httpclient, which I can also recommend. Simple, featureful, compatible with all Ruby platforms and performant. Better cookie support than anything else out there, but the presentation mentions that cookies may leak from one (malevolent) site to another if you use the same client object. Don't know if this is still true.
There is https://github.com/cyx/requests, which is exactly what the question is asking for, a port of the requests lib from python.
The built-in OpenURI is the first place to look. It's simple and handles the basics nicely.
Typhoeus, which I've used several times for parallel processes, works nicely. Documentation and the codebase are available at Github.
irb(main):009:0> response = Typhoeus::Request.get("www.example.com")
=> #<Typhoeus::Response:0x007ffbcc067cf8 #code=302, #curl_return_code=0, #curl_error_message="No error", #status_message=nil, #http_version=nil, #headers="HTTP/1.0 302 Found\r\nLocation: http://www.iana.org/domains/example/\r\nServer: BigIP\r\nConnection: close\r\nContent-Length: 0\r\n\r\n", #body="", #time=0.035584, #requested_url=nil, #requested_http_method=nil, #start_time=nil, #start_transfer_time=0.035529, #app_connect_time=2.8e-05, #pretransfer_time=0.000429, #connect_time=2.8e-05, #name_lookup_time=2.8e-05, #request=:method => :get,
:url => www.example.com, #effective_url="HTTP://www.example.com", #primary_ip="192.0.43.10", #redirect_count=0, #mock=false>
irb(main):010:0> puts response.headers
HTTP/1.0 302 Found
Location: http://www.iana.org/domains/example/
Server: BigIP
Connection: close
Content-Length: 0
I use Net::HTTP occasionally too, but OpenURI and Typhoeus, with Hydra, have proven to be easy to use and integrate with my code.
I've eventually found this HTTPClient :
https://github.com/nahi/httpclient
I've started using it, it matches the features I wanted, and more over it's pretty fast according to some benchmark. It also support some advanced things like streaming or chunked response. It's shame though it's not famous in the ruby community. :)
Have you looked at the HTTParty gem?
If you need cookies and form handling, mechanize is the only way to go.
I'm sorry to hear, that Typhoeus didn't work out for you. The reason is, that the README shows howto work with Typhoeus v0.5.0.rc which can be installed with
gem install typhoeus --pre
or
gem "typhoeus", git: "git://github.com/typhoeus/typhoeus.git"
.
There is no session support for Typhoeus but other than that it could be a good fit. At least its stable as hell since it is build on top of libcurl.
File sending example:
Typhoeus.post("www.example.com/file", body: { file: File.open("testfile.txt","r") })
There is unfortunately no shortcut to deal with cookies, you have to set them manually:
Typhoeus.get("www.example.com/needs_cookie", headers: { Cookie: "PRIVATE" })
TLDR: I would choose Typhoeus for its speed and libcurl if you're willing to set things up yourself. Otherwise I would look into Faraday and use it with the Typhoeus adapter.
Edit: I've added installation instructions to the README.
Edit: 0.5 is released.
This question seems to be lacking recent answers. So am filling in the void.
Coming from python myself, and having loved requests library for what it does easily, I recently discovered a very nice Ruby equivalent in rest_client
It supports all the features mentioned in the question, and seems to be very nice from usability perspective - what requests library aimed to achieve.
I wrote my own FaceBook library that uses actual Curl requests, not libcurl.
Is there a way to test it? I'm asking this because most solutions involve using something like fakeweb which as far as I can tell will not work here.
The existing code can be found on my github page.
One approach would be to use a different host/port in test mode (eg localhost:12345)
Then in your test run a sinatra or webrick servlet on that port that you configure to respond to the requests your code should be making
You could mock Request.dispatcher with an expected behavior, pretty much like Fakeweb would do.
There are a few examples on this file, specially https://github.com/chrisk/fakeweb/blob/master/lib/fake_web/ext/net_http.rb#L44.
When running your tests/specs, monkey-patch the run method of your Request class to hook into the Marston VCR library. See the existing library_hooks subdir for examples and ideas on how to do this -- the fakeweb implementation is a good place to start.
VCR works well with live services like Facebook's because it captures interactions "as is", and VCRs can be easily re-recorded when the services change.
I'm running into problems with your library, however. You need to require the cgi and json libraries; it also looks like it requires a Rails environment (it's failing to find with_indifferent_access on Hash).
I've been using Ruby for quite some time now, however unlike PHP, as far as I know there is not a standard http/Curl (fetching, processing forms) like library that is easy and powerful like PHP's libCuRL binding.
While Net::HTTP is part of the Ruby standard library, I always find that API hard to remember and program with.
Can anyone give suggestions on which http/curl library I should use over Net::HTTP?
Take a look at HTTParty or REST Client.
I would recommend using the Typhoeus gem. It's got a pretty clean API and allows you to make concurrent requests.
I'll second Ryan's recommendation for Typhoeus, and recommend HTTPClient also. Both are very full featured and handle parallel
requests easily.
For simple requests it's hard to beat Open-URI for its simplicity:
require 'open-uri'
html = open('http://www.example.com').read
If you're parsing a page it works great with Nokogiri:
require 'open-uri'
require 'nokogiri'
doc = Nokogiri::HTML(open('http://www.example.com'))
I wrote a wrapper for the Net:HTTP lib recently, its very very simplistic. I wanted something with a simple API that was easy to use and remember, it's been working well for me:
https://github.com/ctcherry/plain_http
I've been looking at this for a couple of days now and haven't
found a solution. Is there a way to upload a file using OAuth-Ruby?
I am working with a REST system that protects their resource with oauth. I am building a test tool using ruby and oauth-ruby to make it easier to upload test data to the system. But I can't get around to upload files to the resources.
When I send a normal request, everything works but adding a file as a
parameter makes the signature invalid.
Example:
#access_token.post("http://.../imageresource", {:name=>"awesome cat"}, {'Content-Type' => 'multipart/form-data'})
works but gives me:
<error>
<message>images/POST: Request has no file data</message>
</error>
I am not sure how to add a file to the post.
Any thoughts on this?
Thanks,
I know this is old but I'm looking to do this too, this looks like it could do the trick.
Actually there's a question ruby-how-to-post-a-file-via-http-as-multipart-form-data that has an example.
This is either impossible to do with the oauth gem or exceedingly difficult. Either way, I don't know of any way to do it using that gem.
It can be done trivially with my signet gem as long as you have a handy way to construct a valid multipart request body. The construction of such a request body is out-of-scope of an OAuth gem, but should be pretty easy to do with most HTTP clients. The httpadapter gem can then translate the request into a form that signet can sign. Let me know if your preferred HTTP client isn't supported by httpadapter and I'll get that resolved immediately.
See the second example on the fetch_protected_resource method to get an idea for how this might be done.