Using tweetstream and timeout - ruby

Ruby version: 2.0.0-p0
Mac - Mountain Lion
Following is my code (tw_stream_track.rb):
require 'tweetstream'
TweetStream.configure do |config|
config.consumer_key = 'xxxxxxxxxxxxxxx'
config.consumer_secret = 'xxxxxxxxxxxxxxx'
config.oauth_token = 'xxxxxxxxxxxxxxx-xxxxxxxxxxxxxxx'
config.oauth_token_secret = 'xxxxxxxxxxxxxxx'
config.auth_method = :oauth
end
client = TweetStream::Client.new
client.on_error do |message|
puts message
end
client.track('apple', 'microsoft', 'samsung') do |status|
puts "#{status.text}"
end
When I run it from a terminal:
ruby tw_stream_track.rb
I get the following:
/Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/tweetstream-2.5.0/lib/tweetstream/client.rb:436:in `block in connect': Failed to reconnect after 11 tries. (TweetStream::ReconnectError)
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/em-twitter-0.2.2/lib/em-twitter/connection.rb:296:in `call'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/em-twitter-0.2.2/lib/em-twitter/connection.rb:296:in `invoke_callback'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/em-twitter-0.2.2/lib/em-twitter/connection.rb:268:in `rescue in schedule_reconnect'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/em-twitter-0.2.2/lib/em-twitter/connection.rb:263:in `schedule_reconnect'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/em-twitter-0.2.2/lib/em-twitter/connection.rb:93:in `unbind'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/eventmachine-1.0.3/lib/eventmachine.rb:1440:in `event_callback'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/tweetstream-2.5.0/lib/tweetstream/client.rb:388:in `start'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/tweetstream-2.5.0/lib/tweetstream/client.rb:131:in `filter'
from /Users/mine/.rvm/gems/ruby-2.0.0-p0/gems/tweetstream-2.5.0/lib/tweetstream/client.rb:98:in `track'
This is my first time using tweetstream, and from the documentation, it seems as if this is what I'm supposed to be doing. What am I missing?

I heard TweetStream had some problems with ruby 2.0.0, did you try with ruby 1.9.3 ?
UPDATE
I just had a try and your code is working fine with ruby 1.9.3 (installed with rvm).
The only pb that I have is that the stream seems to stop retrieving new messages after some time.

In my case this error appeared because I was using same keys in 2 rails app instances(on staging and production servers) + 1 local(development) instance.
The 3rd instance was unable to connect to tweets stream. It is cause because of the twitter limitations
So I resolved this by creating dedicated twitter applications(with separate keys) for each Rails application instance.

Related

event machine with em http gem cannot load middleware oauth

I am trying to get the stock price streaming to work using TradeKing API
https://developers.tradeking.com/documentation/ruby-streaming
or the copied and pasted codes below
require 'em-http'
require 'em-http/middleware/oauth'
credentials = {
:consumer_key => "<CONSUMER_KEY>",
:consumer_secret => "<CONSUMER_SECRET>",
:access_token => "<ACCESS_TOKEN>",
:access_token_secret => "<ACCESS_TOKEN_SECRET>"
}
EM.run do
conn = EventMachine::HttpRequest.new('https://stream.tradeking.com/v1/market/quotes.json?symbols=F')
conn.use EventMachine::Middleware::OAuth, credentials
http = conn.get
http.stream { |chunk| puts chunk }
http.errback do
EM.stop
end
trap("INT") { http.close; EM.stop }
trap("TERM") { http.close; EM.stop }
end
After getting key, secret, and token, I built a simple Ruby app to play with the codes but I get an error that says it cannot load
require 'em-http/middleware/oauth'
If I disable this, the code EventMachine::Middleware::OAuth will not work.
Here is the error message:
c:/tools/ruby215/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/
kernel_require.rb:54:in `require': cannot load such file --
simple_oauth (LoadError)
from c:/tools/ruby215/lib/ruby/site_ruby/2.1.0/rubyg
ems/core_ext/kernel_require.rb:54:in `require'
from c:/tools/ruby215/lib/ruby/gems/2.1.0/gems/em-ht
tp-request-1.1.2/lib/em-http/middleware/oauth.rb:1:in `<top
(required)>'
from c:/tools/ruby215/lib/ruby/site_ruby/2.1.0/rubyg
ems/core_ext/kernel_require.rb:54:in `require'
from c:/tools/ruby215/lib/ruby/site_ruby/2.1.0/rubyg
ems/core_ext/kernel_require.rb:54:in `require'
from app.rb:2:in `<main>'
I am new to event machine and em-http gem. I looked at their documentation but couldn't find information about this error. Can someone help me figure out why the file cannot load?
OK, so I solved this myself. I haven't worked with vanilla Ruby in a while so I got a bit rusty with reading the error messages. The error says that it cannot load the file and points to line 1 of the the oauth.rb file which is in the em-http-request gem. I looked into the file and the first line says require simple_oauth which I haven't installed it yet. I would think installing em-http-request gem would install simple_oauth as a dependency, but I guess not (well the em-http-request hasn't been updated for several years).
The fix to this problem is to install simple_oauth gem.
gem install simple_oauth
and run those codes again and it should work.
https://rubygems.org/gems/simple_oauth/versions/0.3.1
https://github.com/laserlemon/simple_oauth
I hope this helps anyone who is having the same problem (since TradeKing API doc isn't as clear).

Using ruby and aws-sdk to pull ec2 tags

Folks,
I am trying to pull in the EC2 tags via ruby aws-sdk and iam roles. The following bit of code produces error below. What am I doing wrong?
require 'rubygems'
require 'aws-sdk'
require 'fileutils'
ec2 = AWS::EC2.new()
instance_id = `wget -q -O - http://169.254.169.254/latest/meta-data/instance-id`
outdir = '/opt/facts/tags/'
unless File.directory?(outdir)
FileUtils.mkdir_p(outdir)
end
ec2= AWS::EC2.new()
instance = ec2.instances[instance_id]
tags = instance.tags
tags.each do |num|
File.open(outdir+num[0], "w") do |f|
f.write(num[1])
end
end
Following output:
/usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/http/connection_pool.rb:127:in `session_for': undefined method `continue_timeout=' for #<Net::HTTP ec2.us-east-1.amazonaws.com:443 open=false> (NoMethodError)
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/http/net_http_handler.rb:52:in `handle'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:236:in `make_sync_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:262:in `retry_server_errors'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:229:in `make_sync_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:489:in `client_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:371:in `log_client_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:457:in `client_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:353:in `return_or_raise'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/core/client.rb:456:in `client_request'
from (eval):3:in `describe_tags'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/ec2/filtered_collection.rb:44:in `send'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/ec2/filtered_collection.rb:44:in `filtered_request'
from /usr/lib/ruby/gems/1.8/gems/aws-sdk-1.9.0/lib/aws/ec2/resource_tag_collection.rb:156:in `each'
from ./puppetclient.rb:22
That seems to be a bug that was already reported today: undefined method `continue_timeout=', you could try pointing to the main branch of the git repository since there's already a patch for it.
Edit:
You can already point the latest version in rubygems since they already deployed the fix on version 1.9.1:
https://rubygems.org/gems/aws-sdk
There was a bug announced today in AWS-SDK-ruby https://github.com/aws/aws-sdk-ruby/issues/232
You can fix it by updating your gemfile
gem 'aws-sdk', "1.9.1"
And then running "bundle"

padrino && websockets

i'm looking for a way to open and use websockets from within a Padrino application. i know Padrino works with a single thread but i'm looking for a way to open websockets and share variables between its "onopen" "onclose" "onmessage" methods and Padrino controllers.
any idea how it's done ?
links i looked into:
Examples of Eventmachine usage with Padrino and Sinatra (only Sinatra worked for me)
em-websocket on GitHub
UPDATE 1:
this is my main.rb:
require 'rubygems' # <-- Added this require
require 'em-websocket'
require 'padrino-core'
require 'thin'
require File.expand_path("../config/boot.rb", __FILE__)
SOCKETS = []
EventMachine.run do # <-- Changed EM to EventMachine
# class App < Sinatra::Base
# get '/' do
# SOCKETS.each {|s| s.send "fooooo"}
# return "foo"
# end
# end
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8080) do |ws| # <-- Added |ws|
# Websocket code here
ws.onopen {
ws.send "connected!!!!"
SOCKETS << ws
}
ws.onmessage { |msg|
puts "got message #{msg}"
ws.send "ECHO: #{msg}"
}
ws.onclose {
ws.send "WebSocket closed"
SOCKETS.delete ws
}
end
# You could also use Rainbows! instead of Thin.
# Any EM based Rack handler should do.
#App.run!({:port => 3000}) # <-- Changed this line from Thin.start to App.run!
Thin::Server.start Padrino.application, '0.0.0.0', 3000
end
i'm getting this exception:
/home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/thin-1.2.11/lib/thin/daemonizing.rb:2:in `require': no such file to load -- daemons (LoadError)
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/thin-1.2.11/lib/thin/daemonizing.rb:2:in `<top (required)>'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/thin-1.2.11/lib/thin/server.rb:50:in `<class:Server>'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/thin-1.2.11/lib/thin/server.rb:48:in `<module:Thin>'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/thin-1.2.11/lib/thin/server.rb:1:in `<top (required)>'
from main.rb:39:in `block in <main>'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `call'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run_machine'
from /home/cstore/.rvm/gems/ruby-1.9.2-p290#runtime/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in `run'
from main.rb:9:in `<main>'
UPDATE 2:
Resolved thanks to Nathan !
I just added 'daemons' to Gemfile and reloaded my application.
Maybe you need to install daemons:
Edit your Gemfile:
# Adding this
gem 'daemons'
Install missing gems:
$ bundle install
What in particular from this example: https://github.com/igrigorik/em-websocket and Any success with Sinatra working together with EventMachine WebSockets? didn't work with Padrino but did with Sinatra? Can you explain the errors you got and why those examples failed (stacktraces)? Maybe we can help investigate.
I ran across this post and it helped me a bit, but I wanted to offer an alternative solution to anyone else who might stumble upon it. I chose to just directly modify the config.ru and mount a websocket-rack application.
Here's my config.ru where WSApp is a subclass of Rack::WebSocket::Application and is placed in the lib/ directory (therefore being automatically loaded by Padrino):
#!/usr/bin/env rackup
# encoding: utf-8
# This file can be used to start Padrino,
# just execute it from the command line.
require File.expand_path("../config/boot.rb", __FILE__)
# Setup routes
map '/' do
run Padrino.application
end
map '/ws' do
run WSApp.new
end
Since this is the top hit in Google right now, I'd like to link it to padrino-websockets, a clean DSL for writing websockets applications in Padrino.

ruby-openid: #socket not set while performing discovery

I'm having a bit of truble with omniauth/openid.
When trying to authenticate, I found this in my logs:
OpenID::FetchingError: Error fetching https://www.google.com/accounts/o8/.well-known/host-meta?hd=profiles.google.com%2Fmy_username: undefined method `io' for nil:NilClass
The important thing there is undefined method io' for nil:NilClass which comes from openid/fetchers.rb, in the following snippet:
module Net
class HTTP
def post_connection_check(hostname)
check_common_name = true
cert = #socket.io.peer_cert
cert.extensions.each { |ext|
next if ext.oid != "subjectAltName"
ext.value.split(/,\s+/).each{ |general_name|
if /\ADNS:(.*)/ =~ general_name
check_common_name = false
...
That error is generated by #socket.io.peer_cert, #socket is not defined.
Have any of you encountered this before? Not quite sure what the cause is.
Versions I'm running:
ruby 1.9.3dev (2010-08-17 trunk 29020) [x86_64-darwin10.4.0]
ruby-openid (2.1.8)
ruby-openid-apps-discovery (1.2.0)
omniauth 0.2.0
We had this same problem and it was a direct result of Net::HTTP#connect never being invoked. Turns out we had the fakeweb gem scoped into the environment that was throwing the error (development, in our case).
Narrowing fakeweb's scope allows for normal processing of #connect and #socket is once again happy.
group :test do
gem 'fakeweb'
end
We came across the same / very similar problem with both fakeweb and webmock (when using the VCR gem). Switching from fakeweb to typhoeus seemed to have solved this problem for us.

cucumber, capybara & selenium works randomly

Setup with cucumber, capybara and selenium but some scenarios works only randomly.
Running
ruby 1.8.6 on rvm
rails 2.3.8
selenium pops open firefox 3.6
I have tried to add this with no luck:
with_scope(selector) do
click_button(button)
selenium.wait_for_page_to_load
end
The error output is sometimes:
> Given I am logged in and have created newsletter and subscribers # features/step_definitions/newsletter_send_steps.rb:108
end of file reached (EOFError)
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:133:in `sysread'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:133:in `rbuf_fill'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/timeout.rb:62:in `timeout'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/timeout.rb:93:in `timeout'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:132:in `rbuf_fill'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:116:in `readuntil'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/protocol.rb:126:in `readline'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:2020:in `read_status_line'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:2009:in `read_new'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1050:in `request_without_fakeweb'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1037:in `request_without_fakeweb'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:543:in `start'
/Users/christianhager/.rvm/rubies/ruby-1.8.6-p399/lib/ruby/1.8/net/http.rb:1035:in `request_without_fakeweb'
./features/step_definitions/web_steps.rb:24:in `__instance_exec2'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:23:in `/^(?:|I )press "([^\"]*)"(?: within "([^\"]*)")?$/'
features/enhanced/newsletter_send1.feature:7:in `Given I am logged in and have created newsletter and subscribers'
And othertimes:
> no button with value or id or text 'create_user_button' found (Capybara::ElementNotFound)
./features/step_definitions/web_steps.rb:24:in `__instance_exec2'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:9:in `with_scope'
./features/step_definitions/web_steps.rb:23:in `/^(?:|I )press "([^\"]*)"(?: within "([^\"]*)")?$/'
features/enhanced/newsletter_send1.feature:7:in `Given I am logged in and have created newsletter and subscribers'
And sometimes it just works....
This is how my env.rb looks like
ENV["RAILS_ENV"] ||= "cucumber"
require File.expand_path(File.dirname(__FILE__) + '/../../config/environment')
require 'cucumber/formatter/unicode' # Remove this line if you don't want Cucumber Unicode support
require 'cucumber/rails/world'
require 'cucumber/rails/active_record'
require 'cucumber/web/tableish'
require 'capybara/rails'
require 'capybara/cucumber'
require 'capybara/session'
require 'cucumber/rails/capybara_javascript_emulation'
require "selenium-webdriver"
Capybara.default_driver = :selenium
Capybara.default_wait_time = 5
Capybara.ignore_hidden_elements = false
Capybara.default_selector = :css
ActionController::Base.allow_rescue = false
require 'database_cleaner'
DatabaseCleaner.strategy = :truncation
Before do
Capybara.reset_sessions!
DatabaseCleaner.clean
end
Cucumber::Rails::World.use_transactional_fixtures = false
Cucumber-steps:
Given I am on the signup page
And I fill in "user_login" with "jeppsipeppsi#arcticelvis.com" within "body"
And I fill in "user_password" with "secret" within "body"
And I fill in "user_password_confirmation" with "secret" within "body"
And I check "terms_of_use" within "body"
And I press "create_user_button" within "body"
Any insight would be great :)
It's HTTP mocking, if you remove fakeweb or webmock from your environment (entirely), it should all work again.
The last comment by Adam Greene DOES WORK regarding setting up Curb with:
Selenium::WebDriver.for :firefox, :http_client => Selenium::WebDriver::Remote::Http::Curb
Read the thread on the Capybara group.
The problem we're having is playing back recorded http traffic using fakeweb or webmock since web driver is now Curb. So if you're goal was to fake out traffic over Capybara, you'll get browser testing to work again but you won't be able to play the traffic back over the same browser. (We're using VCR to record.)
Adding Curb support is listed as a "ticket" on the Fakeweb's Github Issues site.
I bumped into this in a Rails 2.3 app with cucumber/capybara/akephalos/fakeweb recently, but ultimately got to resolve this by completely killing all gems in my bundle (they where kept in .bundle/ and reinstalling.

Resources