I have a ruby code that triggers php script over https.
Use case: The php script usually finishes in 5 minutes so I have set up time out for https request after 10 minutes. I need a timer that would trigger code after let's say 7 minutes after the https request started.
I was thinking of using thread that I created just before I initiate https request. I am not sure if this the correct way to approach this. Maybe there is not need to use threads at all. I am using ruby 1.8.7 (2010-08-16 patchlevel 302) [i386-mingw32]. Also I don't now if I can 'kill' the thread on successful finish of https request.
uri = URI.parse(url)
start = Time.new
http_read_timeout=60*10
connection = Net::HTTP.new(uri.host, 443)
connection.use_ssl = true
begin
response = connection.start() do |http|
http.open_timeout = 50
http.read_timeout = http_read_timeout
http.request_get(uri.request_uri)
# here I need to place a code that is triggered
# in case of custom timeout is reached
end
rescue Timeout::Error
# "Connection failed
time_out_message ="security time out - after #{http_read_timeout} sec"
return time_out_message
end
puts "finished"
The basic structure could be like this:
seconds_timer = MyDelay
counter = 0
test_thread = Thread.new do
run_http_php_test
end
while test_thread.alive?
counter += 1
if counter > seconds_timer
handle_custom_timeout_somehow
# if you want to halt run_http_php_test:
test_thread.kill if test_thread.alive?
# otherwise:
break
end
sleep 1
end
# the below doesn't apply if you kill the run_http_php_test thread
test_thread.join if test_thread.alive?
...but of course you could change that sleep 1 to whatever polling interval you like. Polling is nicer than just forcing your original thread to sleep, because the code will finish faster if run_http_php_test is done before you hit your custom timeout value.
Most or all of your code above can be in the run_http_php_test method, or inserted directly...whichever you'd prefer.
ruby 1.9.3 implements timeout module that has a timeout function. you can see it here. if you scroll down you can click show source and see the definition for timeout method. you can copy it if you dont want to upgrade to ruby 1.9.3 (I recommend upgrade since 1.8.7 is very slow compared to 1.9.3)
Related
This code sends two HTTP requests to the www.example.com website:
require 'socket'
#host = 'www.example.com'
#port = 80
#path = "/"
# Build HTTP request
def request(close=false)
"GET #{#path} HTTP/1.1\r\nHost: #{#host}#{"\r\nConnection: close" if close}\r\n\r\n"
end
# Build socket
socket = TCPSocket.open(#host,#port) # Connect to server
# Send request twice via socket
2.times {socket.print request}
Here are various methods I have found for reading the response:
# Method 1: close_write and read
socket.close_write # Without this line, the next line hangs
response = socket.read
puts response.length
# Method 2: send another http request with 'Connection: close' header, then use 'read'
socket.print request(true) # Without this line, the next line hangs
response = socket.read
puts response.length
# Method 3: recv
# puts socket.eof? # Method fails when I add this line
r1, r2 = socket.recv(1000000), socket.recv(1000000)
puts r1.length, r2.length
# Method 4: IO.select and read_nonblock
puts socket.eof?
# IO.select([socket]) # The code still works without this IO.select...
r1 = socket.read_nonblock(9999999)
IO.select([socket]) # ...but not without this one
r2 = socket.read_nonblock(9999999)
puts r1.length, r2.length
puts socket.eof? # Hangs for ages before returning 'true'
Questions:
What exactly is 'socket.close_write' line doing in Method 1 and why is it necessary for the method to work?
In Method 2, is the 'Connection: close' header somehow achieving the same result as the 'socket.close_write' line in Method 1? If not, what is it doing, and why is it necessary for the rest of the method to work?
Why, in Method 3, does the addition of the commented line, 'puts socket.eof?', cause the rest of the code to hang?
In Method 3, how and why does the recv call stop at the end of an HTTP response (as opposed to picking up the next response as well)?
Why is it that the second IO.select in Method 4 is necessary, but the first isn't?
What does IO.select actually do?
Why does the last line, 'puts socket.eof?', in Method 4 hang for ages before returning true?
Is there a general way of checking how many responses a socket is currently expecting, and reading that number of responses from the socket, without closing the socket for writing?
Finally, if not possible in an answer here, is there a good resource somewhere where I can get some clarity on all of the above, and general clarity on reading from TCP sockets (or network sockets in general)?
Thanks.
socket.close_write is not necessary. I mean you can get what socket returns by socket.read, but you are gonna to wait some time. The reason why is because you are trying to read the whole stream in one with socket.read. It takes time. You can find out what socket returns by doing:
socket.each_line do |line|
puts line
end
BTW, Nagle's algorithm makes it slow too.
Whatclose_write does is let the client side half-close the socket.When the server side notice this, it will also close its side. Then you can finish reading very quickly.
Or you can use IO::select.As the documentation said:
It monitors given arrays of IO objects, waits one or more of IO objects ready for reading, are ready for writing, and have pending exceptions respectively, and returns an array that contains arrays of those IO objects. It will return nil if optional timeout value is given and no IO object is ready in timeout seconds.
ready = IO.select([socket], nil, nil, 10)
if ready
# do something
else
# raise timeout
end
Here we pass the first parameter, which is the object ready for reading and the last is the timeout you want set. This means if the reading is not ready in 10 seconds it will return nil then raise timeout error.
I'm expecting this piece of code to run for no longer than 5 seconds:
require 'httpi'
require 'timeout'
puts Time.new
begin
request,response=nil,nil
Timeout::timeout(5){
request=HTTPI::Request.new(url: "http://example.com")
response=HTTPI.get(request)
}
rescue
puts "except: #{$!}"
ensure
puts Time.new
end
But this is the output I'm getting:
2016-11-04 09:44:55 -0400
D, [2016-11-04T09:44:55.916557 #2476] DEBUG -- : HTTPI GET request to example.com (net_http)
except: execution expired
2016-11-04 09:45:16 -0400
I'm assuming NET's default HTTP timeout is 20 seconds, so Timeout::timeout is just allowing the code to run however long it wants. Why?
As you can see here, here and here, the Ruby's Timeout module is famous for having some problems.
You should consider not using this module, unless it is extremely necessary.
Instead, you can use the read_timeout and/or open_timeout options provided by the HTTPI's API.
request = HTTPI::Request.new(url: "http://example.com")
request.open_timeout = 5 # seconds
request.read_timeout = 5 # seconds
response = HTTPI.get(request)
There is a some heavy page, that after visiting it Selenium doesn't respond to Capybara for a minute, so whatever do I call, throws Net::ReadTimeout.
I could edit it globally somehow like:
http_client = Selenium::WebDriver::Remote::Http::Default.new
http_client.timeout = 120
Capybara::Selenium::Driver.new(app,
http_client: http_client,
But in the case of some repetitive timeouts my tests would last for too long, so I do not want to increase timeout globally.
I want to increase it for a single test somehow like:
before do
#timeout = page.driver.bridge.http.timeout
page.driver.bridge.http.timeout = 120
end
after do
page.driver.bridge.http.timeout = #timeout
end
But in /lib/selenium/webdriver/common/driver.rb the bridge method is private, while only browser and capabilities are exposed to public.
So what is the correct way to edit this timeout attribute globally?
UPD: Even if I find how to set this attribute, seems like the before/after approach doesn't work, because #http ||= ( saves the default timeout value in the first before in the chain of setUps, that precede mine.
Capybara has a default_wait_time that can be changed in the middle of tests:
using_wait_time 120 do
foo(bar)
end
This is how I broke private method, attribute without getter, and patched timeout for a single command:
http = page.driver.browser.send(:bridge).http.instance_variable_get(:#http)
old_timeout = http.read_timeout
begin
http.read_timeout = 120
find("anything") # here we had timeout
ensure
http.read_timeout = old_http_timeout
end
I'm using Celluloid::IO to do DNS query and below is my code:
require 'celluloid/io'
class MyResolver
include Celluloid::IO
def initialize
#resolver = DNSResolver.new
end
def resolve(domain)
ips = #resolver.resolve domain
#sleep 1
return {domain: domain, ip: ips}
end
end
pool = MyResolver.pool(size: 5)
domains = [
'www.google.com',
## many other record
]
futures = domains.map {|d| pool.future.resolve(d)}
futures.each do |future|
puts "#{future.value}"
end
This code works and finished in few seconds. But when I add the line sleep 1(just for learning purpose), after printing some results, the process blocked forever, which is very strange.
Thanks for any help.
sleep is an overridden keyword in Celluloid, so if you want sleep from Ruby itself, use Kernel.sleep. But that being said, as of 0.17.0-dependent branch of Celluoid::IO this error you describe does not exist ( anymore? ).
I used your reproducible failing case to test the new celluloid-pool gem being released in version 0.17.0 of Celluloid, and it is working no problem with sleep 1 as is.
Something is wrong with DNSResolver at least in that case, but you can use "thread-aware DNS resolver" Resolv from standard ruby library - no any blocks with ~25k domains array. Don't forget to catch exceptions from Resolv.
I have a function that gets response over http. It runs some tests. Lately it started to happen that the test never finishes. So I introduced a time out. Then I found out that if I stop the database server the test script finishes with a db error that is in fact very good lead why the test didn't finish as expected. So to get the error could help to save me time. Because I wouldn't have to reproduce the whole test again manually.
Q1: Is there any way to let the connection time out but then get the response after the database server is restarted? Note that I cannot send the http request again as it would start the same text again.
Q2: I think that a solution would be to introduce timer while "waiting" for http response. But I don't know how to do that. Any idea?
My function is like
def execute_db2_script(url)
db2_database = 'RATIONAL'
http_read_timeout=$http_read_timeout
uri = URI.parse(url)
start = Time.new
connection = Net::HTTP.new(uri.host, 443)
connection.use_ssl = true
begin
response = connection.start() do |http|
http.open_timeout = 50
http.read_timeout = http_read_timeout
http.request_get(uri.request_uri)
end
rescue Timeout::Error
time_out_message ="security time out - after #{$http_read_timeout} sec"
return time_out_message
end
return response.body.gsub("\n","<BR>")
end
You can use retry keyword
def execute_db2_script(url)
...
begin
...
rescue Timeout::Error
time_out_message ="security time out - after #{$http_read_timeout} sec"
if "the server is going to restart then"
retry # this will restart begin-rescue-end block again
else
return time_out_message
end
end
response.body.gsub("\n","<BR>")
end