I'm working with the Twitter API using the code below and I'm hitting the API rate limit inside the loop.
My question is: how am I hitting the rate limit? I believe that I'm only making one API call when I'm assigning myFollowers and in the loop I'm just looking through the location value of each object in myFollwers which should just be server side processing.
require 'twitter'
client = Twitter::REST::Client.new do |config|
config.consumer_key = "xx"
config.consumer_secret = "xx"
config.access_token = "xx"
config.access_token_secret = "xx"
end
myFollowers = client.followers("mytwittername")
myFollowers.each do |follower|
if follower.location.include? 'New'
print Name: follower.name
puts Location: follower.location
end
end
I believe that by default, client.followers is not going load up the entire user entity for each follower, so when you iterate over the followers you are generating a new request and thus quickly hitting the rate limit.
Try setting :include_user_entities on the client.followers call to true and see what happens. See link to code/documentation below.
http://www.rubydoc.info/gems/twitter/Twitter/REST/FriendsAndFollowers#followers-instance_method
If that doesn't work, stop your script after your client.followers call and let us know what the myFollowers collection looks like.
Related
I have set up a dashboard using dashing with a number of (mostly) existing widgets. That worked so far - see production dashboard here (work in progress).
Now I would like to have an Instagram widget that displays the n lastest images taken by username.
I have found a widget that will display images by long and lat and also was able to get my tokens configured, so I can talk to the Instagram API.
Here's the code of my current widget originally from #mjamieson's gist on github.
require 'instagram'
require 'rest-client'
require 'json'
# Instagram Client ID from http://instagram.com/developer
Instagram.configure do |config|
config.client_id = ENV['INSTAGRAM_CLIENT_ID']
config.client_secret = ENV['INSTAGRAM_CLIENT_SECRET']
end
# Latitude, Longitude for location
instadash_location_lat = '45.429522'
instadash_location_long = '-75.689613'
SCHEDULER.every '10m', :first_in => 0 do |job|
photos = Instagram.media_search(instadash_location_lat,instadash_location_long)
if photos
photos.map do |photo|
{ photo: "#{photo.images.low_resolution.url}" }
end
end
send_event('instadash', photos: photos)
end
I got this to work, but would like to modify the given API call to only display images taken by me / a user of my choice. Unfortunately I don't understand ruby or json enough to figure out what the Instagram API documentation wants me to do.
I found the following url
https://api.instagram.com/v1/users/{user-id}/media/recent/?access_token={acces-token}
and tried it (with my credentials filled in). It returned json data correctly including my images (among other data).
How can I modify the given code to display images by username instead of location?
Any help is greatly appreciated.
You'll need an access_token to get content from some user. Take a look at sample application on gem page.
It seems you need something like this:
# here we take access token from session, assuming you already got it
# sometime before and stored it there for future use
client = Instagram.client(:access_token => session[:access_token])
photos = client.user_recent_media
And this example how to get this access_token using OAuth2 browser authorization and sinatra app:
require "sinatra"
require "instagram"
enable :sessions
CALLBACK_URL = "http://localhost:4567/oauth/callback"
Instagram.configure do |config|
config.client_id = "YOUR_CLIENT_ID"
config.client_secret = "YOUR_CLIENT_SECRET"
# For secured endpoints only
#config.client_ips = '<Comma separated list of IPs>'
end
get "/" do
'Connect with Instagram'
end
get "/oauth/connect" do
redirect Instagram.authorize_url(:redirect_uri => CALLBACK_URL)
end
get "/oauth/callback" do
response = Instagram.get_access_token(params[:code], :redirect_uri => CALLBACK_URL)
session[:access_token] = response.access_token
redirect "/nav"
end
Solution
require 'sinatra'
require 'instagram'
# Instagram Client ID from http://instagram.com/developer
Instagram.configure do |config|
config.client_id = ENV['INSTAGRAM_CLIENT_ID']
config.client_secret = ENV['INSTAGRAM_CLIENT_SECRET']
config.access_token = ENV['INSTAGRAM_ACCESS_TOKEN']
end
user_id = ENV['INSTAGRAM_USER_ID']
SCHEDULER.every '2m', :first_in => 0 do |job|
photos = Instagram.user_recent_media("#{user_id}")
if photos
photos.map! do |photo|
{ photo: "#{photo.images.low_resolution.url}" }
end
end
send_event('instadash', photos: photos)
end
Explaination
1.) In addition to the client_id and client_secret I had defined before, I just needed to add my access_token to the Instagram.configure section.
2.) The SCHEDULER was correctly working, but needed to call Instagram.user_recent_media("#{user_id}") instead of Instagram.media_search(instadash_location_lat,instadash_location_long)
3.) To do that I had to set a second missing variable for user_id
Now the call gets recent media filtered by user ID and outputs it into the dashing widget.
Thanks for the participation and hints! That pointed me into the right direction of the documentation and helped me to figure it out myself.
I have been tasked with creating a Ruby API that retrieves youtube URL's. However, I am not sure of the proper way to create an 'API'... I did the following code below as a Sinatra server that serves up JSON, but what exactly would be the definition of an API and would this qualify as one? If this is not an API, how can I make in an API? Thanks in advance.
require 'open-uri'
require 'json'
require 'sinatra'
# get user input
puts "Please enter a search (seperate words by commas):"
search_input = gets.chomp
puts
puts "Performing search on YOUTUBE ... go to '/videos' API endpoint to see the results and use the output"
puts
# define query parameters
api_key = 'my_key_here'
search_url = 'https://www.googleapis.com/youtube/v3/search'
params = {
part: 'snippet',
q: search_input,
type: 'video',
videoCaption: 'closedCaption',
key: api_key
}
# use search_url and query parameters to construct a url, then open and parse the result
uri = URI.parse(search_url)
uri.query = URI.encode_www_form(params)
result = JSON.parse(open(uri).read)
# class to define attributes of each video and format into eventual json
class Video
attr_accessor :title, :description, :url
def initialize
#title = nil
#description = nil
#url = nil
end
def to_hash
{
'title' => #title,
'description' => #description,
'url' => #url
}
end
def to_json
self.to_hash.to_json
end
end
# create an array with top 3 search results
results_array = []
result["items"].take(3).each do |video|
#video = Video.new
#video.title = video["snippet"]["title"]
#video.description = video["snippet"]["description"]
#video.url = video["snippet"]["thumbnails"]["default"]["url"]
results_array << #video.to_json.gsub!(/\"/, '\'')
end
# define the API endpoint
get '/videos' do
results_array.to_json
end
An "API = Application Program Interface" is, simply, something that another program can reliably use to get a job done, without having to busy its little head about exactly how the job is done.
Perhaps the simplest thing to do now, if possible, is to go back to the person who "tasked" you with this task, and to ask him/her, "well, what do you have in mind?" The best API that you can design, in this case, will be the one that is most convenient for the people (who are writing the programs which ...) will actually have to use it. "Don't guess. Ask!"
A very common strategy for an API, in a language like Ruby, is to define a class which represents "this application's connection to this service." Anyone who wants to use the API does so by calling some function which will return a new instance of this class. Thereafter, the program uses this object to issue and handle requests.
The requests, also, are objects. To issue a request, you first ask the API-connection object to give you a new request-object. You then fill-out the request with whatever particulars, then tell the request object to "go!" At some point in the future, and by some appropriate means (such as a callback ...) the request-object informs you that it succeeded or that it failed.
"A whole lot of voodoo-magic might have taken place," between the request object and the connection object which spawned it, but the client does not have to care. And that, most of all, is the objective of any API. "It Just Works.™"
I think they want you to create a third-party library. Imagine you are schizophrenic for a while.
Joe wants to build a Sinatra application to list some YouTube videos, but he is lazy and he does not want to do the dirty work, he just wants to drop something in, give it some credentials, ask for urls and use them, finito.
Joe asks Bob to implement it for him and he gives him his requirements: "Bob, I need YouTube library. I need it to do:"
# Please note that I don't know how YouTube API works, just guessing.
client = YouTube.new(api_key: 'hola')
video_urls = client.videos # => ['https://...', 'https://...', ...]
And Bob says "OK." end spends a day in his interactive console.
So first, you should figure out how you are going to use your not-yet-existing lib, if you can – sometimes you just don't know yet.
Next, build that library based on the requirements, then drop it in your Sinatra app and you're done. Does that help?
In the following code I specify that I want to make 1 request to the twitter api, but when I puts the output to terminal I continue to receive many responses until I kill the program. How can I limit it to only the specified amount of requests? I am new to ruby so I may be missing something obviously here.
Docs to the twitter gem api
require 'twitter'
...
client = Twitter::REST::Client.new do |config|
config.consumer_key = credentials[:consumer_key]
config.consumer_secret = credentials[:consumer_secret]
config.access_token = credentials[:access_token]
config.access_token_secret = credentials[:access_token_secret]
end
search_results = client.search "China", {:count => 1, :lang => 'en', :result_type => 'recent'}
search_results.each do |value|
puts value.user.screen_name
end
As docs state,
:count (Integer) — The number of tweets to return per page, up to a
maximum of 100.
So, that is kinda reverse for what you're trying to achieve: the more count is, the less http requests you gonna make (as each of them would contain more tweets). Note that you can not set count to an amount more than 100, so the only way to restrict amount of requests in case there is a huge amount of tweets would be
search_results.take(requests_amount)
# => made requests_amount requests, returns requests_amount pages
# each of page should contain up to count tweets, with max of 100
so I'm trying to set up a script to grab tweets so I can use them in my app. Currently I'm using Ruby 2.0.0 and the 1.1 Twitter API REST. Currently I'm not worried about storing the tweets in a database or anything I'm just simply trying to get the correct tweets into terminal. Here is my current code.
require 'twitter'
client = Twitter::REST::Client.new do |config|
config.consumer_key = "XXXXXXXXXXXXXXXXXX"
config.consumer_secret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
config.access_token = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
config.access_token_secret = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
end
topics = ["#BreakingBad", "#Gameofthones"]
client.filter(:track => topics.join(",")) do |tweet|
puts tweet.text
endtag = options[:search]
My problem is that when I run the script I get flooded with just a mass of tweets in all different languages. Any help would be great. Thanks.
I am having trouble with twitter_user.rb, which is supposed to get the number of tweets, followers, and following of a given Twitter username.
I assume that I am supposed to replace TWITTER_USERNAME in line 9 with the Twitter username that I am interested in. I did that and started dashing but I got:
scheduler caught exception:
undefined method '[]' for nil:NilClass
/.../jobs/twitter_user.rb:19:in 'block in <top (required)>'
It looks like the problem is with line 19 which is:
tweets = /profile["']>[\n\t\s]*<strong>([\d.,]+)/.match(response.body)[1].delete('.,').to_i
Can anybody tell me what is going on and how to fix it?
Your assumption is incorrect. The program is looking for an environment variable called TWITTER_USERNAME that is set to the relevant user name. If that variable doesn't exist then the code uses foobugs instead.
If you would rather modify the code than set up an environment variable, then change
twitter_username = ENV['TWITTER_USERNAME'] || 'foobugs'
to
twitter_username = 'myusername'
This is untested code, but it's a general idea how it should have been written. If you clone the source on the original page you can adjust it for your own purposes (i.e. fix it):
require 'nokogiri'
doc = Nokogiri::XML(content)
tweets = doc.at('profile strong').text.delete('.,').to_i
following = doc.at('following strong').text.delete('.,').to_i
followers = doc.at('followers strong').text.delete('.,').to_i
The above three lines can be reduced to something like:
tweets, following, followers = %w[profile following followers].map{ |tag|
doc.at("#{ tag } strong").text.delete(',.').to_i
}
Again, without a usable sample of the XML/HTML I can't do much more, but as a practice we (programmers) shouldn't use regular expressions to try to parse XML or HTML. It's much to easy to break a pattern with either of those types of files.
I managed to solve the same issue for myself by using the twitter API instead to pull out the relevant information. It seems the web page had changed too much for the scraping to work and it could also stop working again at no notice as various people have already said...
This is the solution I used.
#### Get your twitter keys & secrets:
#### https://dev.twitter.com/docs/auth/tokens-devtwittercom
Twitter.configure do |config|
config.consumer_key = 'YOUR_CONSUMER_KEY'
config.consumer_secret = 'YOUR_CONSUMER_SECRET'
config.oauth_token = 'YOUR_OAUTH_TOKEN'
config.oauth_token_secret = 'YOUR_OAUTH_SECRET'
end
twitter_username = 'foobugs'
MAX_USER_ATTEMPTS = 10
user_attempts = 0
SCHEDULER.every '10m', :first_in => 0 do |job|
begin
tw_user = Twitter.user("#{twitter_username}")
if tw_user
tweets = tw_user.statuses_count
followers = tw_user.followers_count
following = tw_user.friends_count
send_event('twitter_user_tweets', current: tweets)
send_event('twitter_user_followers', current: followers)
send_event('twitter_user_following', current: following)
end
rescue Twitter::Error => e
user_attempts = user_attempts +1
puts "Twitter error #{e}"
puts "\e[33mFor the twitter_user widget to work, you need to put in your twitter API keys in the jobs/twitter_user.rb file.\e[0m"
sleep 5
retry if(user_attempts < MAX_USER_ATTEMPTS)
end
end
I have resolved by substituting this line:
followers = /<strong>([\d.]+)<\/strong> Follower/.match(response.body)[0].delete('.,').to_i
with these two:
followers_count_metadata = /followers_count":[\d]+/.match(response.body)
followers = /[\d]+/.match(followers_count_metadata.to_s).to_s