Ruby: begin, sleep, retry: where to put incrementer - ruby

I have a method 'rate_limited_follow' that takes my Twitter useraccount and follows all the users in an array 'users'. Twitter's got strict rate limits, so the method deals with that contingency by sleeping for 15 minutes and then retrying again. (I didn't write this method, rather got it from the Twitter ruby gem api). You'll notice that it checks to see if the number of attempts are less than the MAX_ATTEMPTS.
My users array has about 400 users that I'm trying to follow. It's adding 15 users at a time (when the rate limits seems to kick in), then sleeping for 15 minutes. Since I set the MAX_ATTEMPTS constant to 3 (just to test it), I expected it to stop trying once it had added 45 users (3 times 15) but it's gone past that, continuing to add 15 users around every fifteen minutes, so it seems as if num_attempts is somehow remaining below 3, even though it's gone through this cycle more than 3 times. Is there something I don't understand about the code? Once 'sleep' is finished and it hits 'retry', where does it start again? Is there some reason num_attempts isn't incrementing?
Calling the method in the loop
>> users.each do |i|
?> rate_limited_follow(myuseraccount, i)
>> end
Method definition with constant
MAX_ATTEMPTS = 3
def rate_limited_follow (account, user)
num_attempts = 0
begin
num_attempts += 1
account.twitter.follow(user)
rescue Twitter::Error::TooManyRequests => error
if num_attempts <= MAX_ATTEMPTS
sleep(15*60) # minutes * 60 seconds
retry
else
raise
end
end
end

Each call to rate_limited_follow resets your number of attempts - or, to rephrase, you are keeping track of attempts per user rather than attempts over your entire array of users.
Hoist num_attempt's initialization out of rate_limited_follow, so that it isn't being reset by each call, and you'll have the behavior that you're looking for.

Related

I have a problem about sleep for loop on ruby

guys i write a chat bot for twitch in hobby. i want to spam a word in chat in a time like that write to chat"hello" and wait 10 sec then write again
i did that but in my code have a answer system like that when anyone write to chat !hey my bot answering hello to him. and when i tried in same system that not working
until #socket.eof? do
message = #socket.gets
puts message
if message.match(/PRIVMSG ##{#channel} :(.*)$/)
a = Time.now
b = Time.now + 5
while a < b do
write_to_chat("!prime")
a += 4
sleep(20)
end
end
if message.match(/PRIVMSG ##{#channel} :(.*)$/)
content = $~[1]
username = message.match(/#(.*).tmi.twitch.tv/)[1]
if content.include? 'theoSea'
write_to_chat(" theoAse theoAse theoAse")
end
i solved it with scheduler
i wanted a code for write every 5 min i tried lots of method but sleep command
and i found this link "https://github.com/jmettraux/rufus-scheduler"
all in link u can use this gem.
thx for everything guys.

metriks log to file not working

I wrote a basic program to test the ruby metriks gem
require 'metriks'
require 'metriks/reporter/logger'
#registry = Metriks::Registry.new
#logger = Logger.new('/tmp/metrics.log')
#reporter = Metriks::Reporter::Logger.new(:logger => #logger)
#reporter.start
#registry.meter('tasks').mark
print "Hello"
#registry.meter('tasks').mark
#reporter.stop
After i execute the program, there is nothing in the log other than it got created.
$ cat /tmp/metrics.log
# Logfile created on 2015-06-15 14:23:40 -0700 by logger.rb/44203
You should either pass in your own registry while instantiating Metriks::Reporter::Logger or use the deafult registry (Metrics::Resgitry.default) if you are using a logger to log metrics.
Also the default log write interval is 60 seconds, your code completes before that so even if everything is setup okay it won't get recorded. So, since you want to use your own registry, this should work for you (I'm adding a little sleep since I'm gonna use an interval of 1 second) :
require 'metriks'
require 'metriks/reporter/logger'
#registry = Metriks::Registry.new
#logger = Logger.new('/tmp/metrics.log')
#reporter = Metriks::Reporter::Logger.new(:logger => #logger,
:registry => #registry
:interval => 1)
#reporter.start
#registry.meter('tasks').mark
print "Hello"
#registry.meter('tasks').mark
# Just giving it a little time so the metrics will be recorded.
sleep 2
#reporter.stop
But I don't really think short intervals are good.
UPDATE : Also I think #reporter.write will help you write down the logs instantly regardless of the time interval. So you don't have to use sleep (better).

making 'some_element'.present? in watir wait for less than 5 secs

How do we make some_element.present? or some_element.visible? wait for less than 5 secs.? Because I think some_element.present? alone will wait for default value of 30 secs before timing out.
Thanks
The Element#present? (and Element#visible? and Element#exists?) method does not wait at all. You can see this by checking the time before and after attempting to locate an element that is not present:
puts Time.now
#=> 2014-07-31 22:14:08 -0400
puts browser.element(id: 'does_not_exist').present?
#=> false
puts Time.now
#=> 2014-07-31 22:14:08 -0400
As you can see, the time before and after checking the prescence of the element is a negligible amount.
It should be noted that the above was executed against a tiny page. For a very large page, which would require more inspection, the method could take longer to execute. However, that would be an issue of execution time rather than being Watir is actually waiting.
I believe you are asking how to shorten the length of time before timeout, by default its set to 30 seconds, see below on how to customize that time.
According to http://watirwebdriver.com/waiting/
Explicit waits
There are four built in methods that you can use to make your waiting experience more pleasant (and remove those evil sleep statements from your code)
Watir::Wait.until { ... }: where you can wait for a block to be true
object.when_present.set: where you can do something when it’s present
object.wait_until_present:; where you just wait until something is present
object.wait_while_present:; where you just wait until something disappears
The default timeout for all these methods is 30 seconds, but your can pass an argument to any of these to increase (or decrease) it as needed.
and http://rdoc.info/gems/watir-webdriver/Watir/EventuallyPresent
- (Object) wait_until_present(timeout = nil)
Waits until the element is present.
Examples:
browser.button(:id => 'foo').wait_until_present
Parameters:
timeout (Fixnum) (defaults to: nil) β€” seconds to wait before timing out
- (Object) wait_while_present(timeout = nil)
Waits while the element is present.
Examples:
browser.button(:id => 'foo').wait_while_present
Parameters:
timeout (Integer) (defaults to: nil) β€” seconds to wait before timing out
- (Object) when_present(timeout = nil)
Waits until the element is present.
Examples:
browser.button(:id => 'foo').when_present.click
browser.div(:id => 'bar').when_present { |div| ... }
browser.p(:id => 'baz').when_present(60).text
Parameters:
timeout (Fixnum) (defaults to: nil) β€” seconds to wait before timing out

Rate Exceeding in workflow_execution polling

I am currently trying to modify a plugin for posting metrics to new-relic via AWS. I have successfully managed to make the plugin post metrics from swf to new relic (not originally in plugin), but have encountered a problem if the program runs for too long.
When the program runs for a bout 10 minutes I get the following error:
Error occurred in poll cycle: Rate exceeded
I believe this is coming from my polling swf for the workflow executions
domain.workflow_executions.each do |execution|
starttime = execution.started_at
endtime = execution.closed_at
isOpen = execution.open?
status = execution.status
if endtime != nil
running_workflow_runtime_total += (endtime - starttime)
number_of_completed_executions += 1
end
if status.to_s == "open"
openCount = openCount + 1
elsif status.to_s == "completed"
completedCount = completedCount + 1
elsif status.to_s == "failed"
failedCount = failedCount + 1
elsif status.to_s == "timed_out"
timed_outCount = timed_outCount + 1
end
end
This is called in a polling cycle every 60 seconds
Is there a way to set the polling rate? Or another way to get the workflow executions?
Thanks, here's a link to the ruby sdk for swf => link
The issue is likely that you are creating a large number of workflow executions and each iteration through the loop in workflow_executions is causing a lookup, which eventually is exceeding your rate limit.
This could also be getting a bit expensive, so be careful.
It's not clear what you're really trying to do, so I can't tell you how to fix it unless you post all your code (or the parts around calls to SWF).
You can see here:
https://github.com/aws/aws-sdk-ruby/blob/05d15cd1b6037e98f2db45f8c2597014ee376a59/lib/aws/simple_workflow/workflow_execution_collection.rb
That a call is made to SWF for each workflow in the collection.

Execution expired doing Twitter gem queries with `sleep`

I am calling Twitter's GET friends/ids and GET users/lookup resources. They work great until I take rate limits into account. Here's an example of the working code:
def followers_ids(screen_name)
cursor = "-1"
followers_ids = []
while cursor != 0 do
followers = Twitter.follower_ids(screen_name, :cursor => cursor)
cursor = followers.next_cursor
followers_ids << followers.ids
end
followers_ids.flatten!
followers_ids
end
When I throw a sleep into the while loop like so:
while cursor != 0 do
followers = Twitter.follower_ids(screen_name, :cursor => cursor)
cursor = followers.next_cursor
followers_ids << followers.ids
sleep(60)
end
I get different sets of errors at different times.
C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:799in 'connect': execution expired (Twitter::Error::ClientError)
from C:/RailsInstaller/Ruby1.9.3/lib/ruby/1.9.1/net/http.rb:799:in 'block in connect'
..
from C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/faraday-0.8.7/lib/faraday/adapter/net_http.rb:73:in 'perform_request'
from C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/faraday-0.8.7/lib/faraday/adapter/net_http.rb:38:in 'call'
and
C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/twitter-4.6.2/lib/twitter/response/raise_error.rb:21:in 'on_complete': Twitter is down or being upgraded (Twitter::Error:BadGateway)
C:/RailsInstaller/Ruby1.9.3/lib/ruby/gems/1.9.1/gems/faraday-0.8.7/lib/faraday/response.rb:9:in 'block in call'
..
Anyone have any idea what could be going on? I had read somewhere that, when sleeping, your program will ignore all messages. Could this be part of it? Is there a better way I can be doing this?

Resources