I'm using Curb to get various URLs, and if the response is 200, I get what I need. However, if the response is a redirect, Curb doesn't seem to follow the redirects, even though I ask it to - e.g:
easy = Curl::Easy.new
easy.follow_location = true
easy.max_redirects = 3
easy.url = "http://stats.berr.gov.uk/ed/vat/VATStatsTables2a2d2007.xls"
easy.perform
=> Curl::Err::GotNothingError: Curl::Err::GotNothingError
from /Users/stuart/.rvm/gems/ruby-2.0.0-p0#datakitten/gems/curb-0.8.4/lib/curl/easy.rb:60:in `perform'
However, if I do curl -L http://stats.berr.gov.uk/ed/vat/VATStatsTables2a2d2007.xls on the command line, I get the expected response. What am I doing wrong?
It sounds as this server returns an empty reply[1] if you do not provide an user agent.
To solve you problem just set one:
...
easy.useragent = "curb"
easy.perform
[1]: curl -A '' -L http://stats.berr.gov.uk/... gives (52) Empty reply from server.
Related
I am writing a simple server in Ruby in order to understand the Socket module. Here is my code:
require 'socket'
s = TCPServer.new(3939)
loop do
c = s.accept
STDERR.puts c.gets
c.close
end
I simply want to print the request to the server console before closing the socket. Why does it print the request thrice, instead of just once?
If I curl that code
$ curl localhost:3939
I get an empty reply
curl: (52) Empty reply from server
and a single GET request
GET / HTTP/1.1
Is there a way to make sure that AB gets proper responses from server? For example:
To force it to output the response of a single request to STDOUT OR
To ask it to check that some text fragment is included into the response body
I want to make sure that authentication worked properly and i am measuring response time of the target page, not the login form.
Currently I just replace ab -n 100 -c 1 -C "$MY_COOKIE" $MY_REQUEST with curl -b "$MY_COOKIE" $MY_REQUEST | lynx -stdin .
If it's not possible, is there an alternative more comprehensive tool that can do that?
You can use the -v option as listed in the man doc:
-v verbosity
Set verbosity level - 4 and above prints information on headers, 3 and above prints response codes (404, 200, etc.), 2 and above prints warnings and info.
https://httpd.apache.org/docs/2.4/programs/ab.html
So it would be:
ab -n 100 -c 1 -C "$MY_COOKIE" -v 4 $MY_REQUEST
This will spit out the response headers and HTML content. The 3 value will be enough to check for a redirect header.
I didn't try piping it to Lynx but grep worked fine.
Apache Benchmark is good for a cursory glance at your system but is not very sophisticated. I am currently attempting to tune a web service and am finding that AB does not measure complete response time when considering the transfer of the body. Also as you mention you can not verify what is returned.
My current recommendation is Apache JMeter. http://jmeter.apache.org/
I am having much better success with it. You may find the Response Assertion useful for your situation. http://jmeter.apache.org/usermanual/component_reference.html#Response_Assertion
I have a ruby script, that posts data to a URL:
require 'httparty'
data = {}
data[:client_id] = '123123'
data[:key] = '123321'
url = "http://someserver.com/endpoint/"
response = HTTParty.post(url, :body => data)
Now i am using Charles for sniffing the HTTP traffic. This works great from the browser, but not from the terminal, where I run my script:
$ ruby MyScript.rb
How can I tell ruby or my Terminal.app to use the Charles proxy at http://localhost:88888
Update Another solution would be to see the request before it is being sent. So that I would not necessarily need the proxy.
Setting the proxy as timmah suggested should work.
Anyway 88888 is not a valid port! I think you want to use 8888 (Charles proxy default port).
So the right commands would be:
export http_proxy=localhost:8888
ruby MyScript.rb
If your script was to use https:// you would also/instead need to specify a HTTPS proxy like so:
export https_proxy=localhost:8888
How can I get a server response time with ruby?
Im using Net::HTTP.get_response(URI.parse(url)) to the response code and response time of the URL. Actually my code is:
start_time = Time.now
#req = Net::HTTP.get_response(URI.parse(url))
end_time = (Time.now - start_time) * 1000
puts ("%.2f" % end_time + "ms")
Is working perfectly, but I'm getting too high response times, e.g.: Twitter.com (630.52ms). If I try to ping twitter.com, I'm getting 70/120ms of response.
Is this the best way to calculate the response time of this server?
What you implemented does not show the server response time, it shows:
the time spent in ruby to send the request
the network time of the request
the server's response time
the network time of the response
the time spent in ruby to process the response
If you need to see only the time that the server took to process the request you need to do it with another way. You can use the HTTP response headers. The date header in particular can help you with that.
Ruby timings are included in your version of code as #xlembouras said.
I would suggest to use curl for what you are looking for:
response_time_total = `curl -w \"%{time_total}\" google.com -o /dev/null -s`
You can try it out in terminal:
curl -w \"%{time_total}\n\" google.com -o /dev/null -s
There are different metrics you can get like time_namelookup, time_connect, time_starttransfer, time_redirect, etc. Example:
response_times = `curl -w \"%{time_connect}:%{time_starttransfer}:%{time_total}\" google.com -o /dev/null -s`
time_connect, time_starttransfer, time_total = response_times.split(':')
All available metrics and detailed information can be found at the cURL manpage
Refs
How do I measure request and response times at once using cURL?
Follow the instruction in Paypal Developer 'make your first call':
curl https://api.sandbox.paypal.com/v1/oauth2/token \
-H "Accept: application/json" \
-H "Accept-Language: en_US" \
-u "EOJ2S-Z6OoN_le_KS1d75wsZ6y0SFdVsY9183IvxFyZp:EClusMEUk8e9ihI7ZdVLF5cZ6y0SFdVsY9183IvxFyZp" \
-d "grant_type=client_credentials"
It is working and get the expected result like the instruction states, but I prefer to use ruby curb gem:
require 'curl'
paypal_result = Curl::Easy.http_post("https://api.sandbox.paypal.com/v1/oauth2/token", "grant_type=client_credentials") do |http|
http.headers['Accept'] = "application/json"
http.headers['Accept-Language'] = "en_US"
http.username = "EOJ2S-Z6OoN_le_KS1d75wsZ6y0SFdVsY9183IvxFyZp:EClusMEUk8e9ihI7ZdVLF5cZ6y0SFdVsY9183IvxFyZp"
end
puts paypal_result.body_str
However I got the following:
{"error":"invalid_client","error_description":"The basic auth authorization header cannot be decoded"}
It is an error for sure, but what's wrong with my curb syntax? Any idea?
I can't tell you exactly what's wrong, but I can tell you how to see what's happening. Try putting it into verbose mode, so you can see what data and headers are actually being sent:
curl = Curl::Easy.new
# Make it verbose, prints to stderr
curl.verbose = true
paypal_result = curl.http_post("https://api.sandbox.paypal.com/v1/oauth2/token", "grant_type=client_credentials") do |http|
# etc.
end
You can compare with the CLI version by using the -v flag.
Hope that helps you discover the difference.