Timeout error when using SoftLayer Ruby API - ruby

I am trying to get simple billing information using the script below. The script fails with a timeout error. Can someone help me figure out the problem?
require 'rubygems'
require 'softlayer_api'
require 'pp'
billing = SoftLayer::Service.new("SoftLayer_Account",:username => "USER", :api_key => "KEY", :timeout => 99999)
object_mask = "mask[orderItem[order[userRecord[username]]], invoiceItem[totalRecurringAmount]]"
user_bill= billing.object_mask(object_mask).getNextInvoiceTopLevelBillingItems
pp user_bill

If the API is responding normally for other calls this is most likely caused by the amount of data being requested. The SLAPI will often time out when more information than it can handle is requested.
You can avoid this by using result limits or specifying only the specific properties you want.
When you reference a relational property by default the entire set of local properties is returned. Even when passing through one property to another. The above call would return the entire set of billing items with their associated local properties, all of the local properties for the order(which is being pulled in for each order item redundantly), and the entire invoice item with totalRecurringAmount.
By specifying id at each level you can reduce the amount of data returned:
mask[
orderItem[
id,
order[
id,
userRecord[username]
]
],
invoiceItem[
id,
totalRecurringAmount
]
]
However, at some number of devices/products the call will start to become overloaded again and it will be necessary to paginate the results and process them in batch.
http://sldn.softlayer.com/blog/phil/How-Solve-Error-fetching-http-headers

Probably your account has many billing items and it causes the exception, take a look the following article:
http://sldn.softlayer.com/blog/phil/How-Solve-Error-fetching-http-headers
I can recommend to use resultLimits, in order to avoid the exception, take a look the following Ruby script:
# Get Next Invoice Top Level Billing Items
#
# This script retrieves the billing items that will be on an account's next invoice
#
# Important manual pages:
# http://sldn.softlayer.com/reference/services/SoftLayer_Account/getNextInvoiceTopLevelBillingItems
# http://sldn.softlayer.com/reference/datatypes/SoftLayer_Billing_Item
#
# #license <http://sldn.softlayer.com/article/License>
# #author SoftLayer Technologies, Inc. <sldn#softlayer.com>
require 'rubygems'
require 'softlayer_api'
require 'pp'
# Helper function to fetch through all results from SoftLayer api
# using small page sizes and sleeping before every new page fetch.
def fetch_all(service, method)
records = []; offset = 0; limit = 1000
loop do
results = service.result_limit(offset, limit).send(method)
records += results
break if results.size < limit
offset += limit
sleep 3
end
records
end
# Creating the account service.
billing = SoftLayer::Service.new("SoftLayer_Account", :username => "set me", :api_key => "set me", :timeout => 200)
# object mask
object_mask = "mask[orderItem[order[userRecord[username]]], invoiceItem[totalRecurringAmount]]"
begin
# Calling helper function to get all the result for getNextInvoiceTopLevelBillingItems method
user_bill = fetch_all(billing.object_mask(object_mask), :getNextInvoiceTopLevelBillingItems)
# Print
pp user_bill
rescue StandardError => exception
puts "Error: #{exception}"
end
I hope it helps

Related

How can I retrieve a full list of companies (advertisers) from the Google Ad Manager API?

I'm building a rails 5 API that interacts with Google Ad Manager API and I need a way to retrieve the list of companies (also called advertisers) so I can use them in order to create new orders and eventually, line items.
I haven't been able to find this in the Ad Manager documentation, not for ruby and not for other languages either.
I don't have any code yet since I don't know what service to use.
I'd expect an array of advertisers: [{:name => 'Company 1', :id => '1234'}, {:name => 'Company 2', :id => '4321'}]
Any help would be really appreciated. I haven't been able to find this in the Ad Manager documentation, not for ruby and not for other languages either.
Thanks in advance!
I have found the answer. I was wrong, it is part of the Ruby examples:
def get_advertisers(ad_manager)
company_service = ad_manager.service(:CompanyService, API_VERSION)
# Create a statement to select companies.
statement = ad_manager.new_statement_builder do |sb|
sb.where = 'type = :type'
sb.with_bind_variable('type', 'ADVERTISER')
end
# Retrieve a small amount of companies at a time, paging
# through until all companies have been retrieved.
page = {:total_result_set_size => 0}
begin
# Get the companies by statement.
page = company_service.get_companies_by_statement(
statement.to_statement()
)
# Print out some information for each company.
unless page[:results].nil?
page[:results].each_with_index do |company, index|
puts '%d) Company with ID %d, name "%s", and type "%s" was found.' %
[index + statement.offset, company[:id], company[:name],
company[:type]]
end
end
# Increase the statement offset by the page size to get the next page.
statement.offset += statement.limit
end while statement.offset < page[:total_result_set_size]
puts 'Total number of companies: %d' % page[:total_result_set_size]
end

Bandwidth summary per server

I am trying to get the bandwidth data for all the softlayer servers under my account.
Thanks to account_servers.rb I am able to get the server id for all the servers. Now I would like to get the Bandwidth used by the servers for a particular time frame. The data that I am interested is
http://sldn.softlayer.com/reference/datatypes/SoftLayer_Metric_Tracking_Object_Bandwidth_Summary
.
I tried to get information using softlayer_client.service_named("Metric_Tracking_Object_Bandwidth_Summary"). Unfortunately I am not able to get the details.
I did find a java code, but I am interested in ruby code. Can someone please guide me to get the server bandwith summary?
Getting bandWidth data in SL
Please, try the following Ruby examples:
require 'rubygems'
require 'softlayer_api'
server_id = 11498369
# Your SoftLayer API username.
SL_API_USERNAME = 'set me'
# Your SoftLayer API key.
SL_API_KEY = 'set me'
softlayer_client = SoftLayer::Client.new(:username => SL_API_USERNAME,
:api_key => SL_API_KEY)
vsi_service = softlayer_client.service_named('SoftLayer_Virtual_Guest')
metric_tracking_object_id = vsi_service.object_with_id(server_id).getMetricTrackingObjectId
metric_service = softlayer_client.service_named('SoftLayer_Metric_Tracking_Object')
service_ref = metric_service.object_with_id(metric_tracking_object_id)
begin
object_template = [{
'keyName' => 'PUBLICOUT',
'summaryType' => 'sum'
}]
result = service_ref.getSummaryData('2016-03-29T00:00:00','2016-03-30T00:00:00',object_template,600)
puts result.inspect
rescue => e
puts 'Error when executing the script...'
$stdout.print(e.inspect)
end
References:
SoftLayer_Metric_Tracking_Object::getSummaryData
SoftLayer_Virtual_Guest::getMetricTrackingObjectId
Second example using SoftLayer_Virtual_Gues::getBandwidthDataByDate:
require 'rubygems'
require 'softlayer_api'
require 'pp'
require 'date'
# Set the server id that you wish to get Bandwidth information.
server_id = 11498369
softlayer_client = SoftLayer::Client.new(:username => 'set me',
:api_key => 'set me')
server = SoftLayer::VirtualServer.server_with_id(server_id, :client => softlayer_client)
get_bandwidth_data_by_date = server.service.getBandwidthDataByDate('2016-03-29T00:00:00','2016-03-30T00:00:00','public')
pp('getBandwidthDataByDate: ', get_bandwidth_data_by_date)
References:
SoftLayer_Virtual_Guest::getBandwidthDataByDate
Disclamer: I created my own Ruby SoftLayer client, you can check it at http://github.com/zertico/softlayer specially for situations like this one where you want to access some specific data (and I'm not SoftLayer staff ;) )
If you'd like to give it a try the code that solves your problem is
ps: I'm considering you are manipulating a SoftLayer_Hardware_Server, right?
hardware = Softlayer::Hardware::Server.find(123)
hardware.get_current_bandwidth_summary
mask = 'mask[currentBandwidthSummary]'
hardware = Softlayer::Hardware::Server.mask(mask).find(123)
hardware.current_bandwidth_summary
You will access a ruby object like this one:
=> #<Softlayer::Metric::Tracking::Object::Bandwidth::Summary:0x007ff74b683540
#allocation_amount="1234",
#allocation_id=111111,
#amount_out="12.34567",
#average_daily_usage="1.23",
#currently_over_allocation_flag=0,
#id=1111111,
#outbound_bandwidth_amount="123.45678",
#projected_bandwidth_usage="123.45",
#projected_over_allocation_flag=0>
Hope it can helps you, comment if you have any doubt about the client usage

How to reuse the query result for faster export to csv and xls without using global or session variable

I have a functionality that initially shows the results in HTML (a report) and then
can be exported to CSV and XLS
The idea is to reuse the results, of the query used to render the HTML, to export the same records without re-running the query again
The closest implementation is this: Storing the result in the global variable $last_consult
I have the following INDEX method in a Ruby controller
def index
begin
respond_to do |format|
format.html {
#filters = {}
#filters['email_enterprise'] = session[:enterprise_email] ;
# Add the selected filters
if (params[:f_passenger].to_s != '')
#filters['id_passenger'] = params[:f_passenger] ;
end
if (session[:role] == 2)
#filters['cost_center'] = session[:cc_name]
end
# Apply the filters and assign them to $last_consult that is used
$last_consult = InfoVoucher.where(#filters)
#info_vouchers = $last_consult.paginate(:page => params[:page], :per_page => 10)
estimate_destinations (#info_vouchers)
#cost_centers = fill_with_cost_centers(#info_vouchers)
}
format.csv {
send_data InfoVoucher.export
}
format.xls {
send_data InfoVoucher.export(col_sep: "\t")
}
The .export method is defined like this
class InfoVoucher < ActiveRecord::Base
include ActiveModel::Serializers::JSON
default_scope { order('voucher_created_at DESC') }
def attributes
instance_values
end
#Exporta a CSV o XLS
def self.export(options = {})
column_names = ["...","...","...","...","...",...]
exported_col_names = ["Solicitud", "Inicio", "Final", "Duracion", "Pasajero", "Unidades", "Recargo", "Propina", "Costo", "Centro de costo", "Origen", "Destino", "Proyecto", "Conductor", "Placas"]
CSV.generate(options) do |csv|
csv << exported_col_names
$last_consult.each do |row_export|
csv << row_export.attributes['attributes'].values_at(*column_names)
end
end
end
end
But this approach only works as long as there is no concurrent users between viewing the report and exporting it which in this case is unaceptable
I try to use a session variable to store the query result but since the result of the query can be quite it fails with this error
ActionDispatch::Cookies::CookieOverflow: ActionDispatch::Cookies::CookieOverflow
I have read about flash but don't consider it a good choice for this
Can you please point me in the right direction in how to persist the query results ,currently store in $last_consult, and make it avaible for the CSV and XLS export without using a global or session variable
Rails 4 has a bunch of cache solutions:
SQL query caching: caches the query result set for the duration of the request.
Memory caching: Limited to 32 mb. An example use is small sets, such as a list of object ids that were time-consuming to generate, e.g. the result of a complex select.
File caching: Great for huge results. Probably not what you want for your particular DB query, unless your results are huge and also you're using a RAM disk or SSD.
Memcache and dalli: an excellent fast distributed cache that's independent of your app. For your question, memcache can be a very good solution for apps that return the same results or reports to multiple users.
Terracotta Ehcache: this is enterprise and JRuby. I haven't personally used it. Looks like it good be good if you're building a serious workhorse app.
When you use any of these, you don't store the information in a global variable, nor a controller variable. Instead, you store the information by creating a unique cache key.
If your information is specific to a particular user, such as the user's most recent query, then a decent choice for the unique cache key is "#{current_user.id}-last-consult".
If your information is generic across users, such as a report that depends on your filters and not on a particular user, then a decent choice for the unique cache key is #filters.hash.
If your information is specific to a particular user, and also the the specific filters, the a decent choice for the unique cache is is "#{current_user.id}-#{#filters.hash}". This is a powerful generic way to cache user-specific information.
I have had excellent success with the Redis cache gem, which can work separately from Rails 4 caching. https://github.com/redis-store/redis-rails
I found this great article about most of the caching strategies that you mention
http://hawkins.io/2012/07/advanced_caching_part_1-caching_strategies/
After reading joelparkerhenderson answer I read this this great article about most of the caching strategies he mentioned
http://hawkins.io/2012/07/advanced_caching_part_1-caching_strategies/
I decided to use Dalli gem that depends on memcached 1.4+
And in order to configure and use Dalli I read
https://www.digitalocean.com/community/tutorials/how-to-use-memcached-with-ruby-on-rails-on-ubuntu-12-04-lts and
https://github.com/mperham/dalli/wiki/Caching-with-Rails
And this is how it ended up being implemented:
Installation/Configuration
sudo apt-get install memcached
installation can be verified running command
memcached -h
Then install the Dalli gem and configure it
gem install dalli
Add this lines to the Gemfile
# To cache query results or any other long-running-task results
gem 'dalli'
Set this lines in your config/environments/production.rb file
# Configure the cache to use the dalli gem and expire the contents after 1 hour and enable compression
config.perform_caching = true
config.cache_store = :dalli_store, 'localhost:11211', {:expires_in => 1.hour, :compress => true }
Code
In the controller I created an new method called query_info_vouchers which runs the query and stores the result in the cache by calling the Rails.cache.write method
In the index method I call the fetch to see if any cached data is available and this is only done for the CSV and XLS export format
def index
begin
add_breadcrumb "Historial de carreras", info_vouchers_path
respond_to do |format|
format.html {
query_info_vouchers
}
format.csv {
#last_consult = Rails.cache.fetch ("#{session[:passenger_key]}_last_consult") do
query_info_vouchers
end
send_data InfoVoucher.export(#last_consult)
}
format.xls {
#last_consult = Rails.cache.fetch ("#{session[:passenger_key]}_last_consult") do
query_info_vouchers
end
send_data InfoVoucher.export(#last_consult,col_sep: "\t")
}
end
rescue Exception => e
Airbrake.notify(e)
redirect_to manager_login_company_path, flash: {notice: GlobalConstants::ERROR_MESSAGES[:no_internet_conection]}
end
end
def query_info_vouchers
# Por defecto se filtran las carreras que son de la empresa
#filters = {}
#filters['email_enterprise'] = session[:enterprise_email] ;
# Add the selected filters
if (params[:f_passenger].to_s != '')
#filters['id_passenger'] = params[:f_passenger] ;
end
if (session[:role] == 2)
#filters['cost_center'] = session[:cc_name]
end
# Apply the filters and store them in the MemoryCache to make them available when exporting
#last_consult = InfoVoucher.where(#filters)
Rails.cache.write "#{session[:passenger_key]}_last_consult", #last_consult
#info_vouchers = #last_consult.paginate(:page => params[:page], :per_page => 10)
estimate_destinations (#info_vouchers)
#cost_centers = fill_with_cost_centers(#last_consult)
end
An in the model .export method
def self.export(data, options = {})
column_names = ["..","..","..","..","..","..",]
exported_col_names = ["Solicitud", "Inicio", "Final", "Duracion", "Pasajero", "Unidades", "Recargo", "Propina", "Costo", "Centro de costo", "Origen", "Destino", "Proyecto", "Conductor", "Placas"]
CSV.generate(options) do |csv|
csv << exported_col_names
data.each do |row_export|
csv << row_export.attributes['attributes'].values_at(*column_names)
end
end
end

ODBC on Mac Lion with Ruby Sequel

I'm having issues getting Sequel to connect to a MS SQL Database.
I have installed the following:
UnixODBC
FreeTDS
I have configured both software packages, and the following commands allow me to connect to my database without a problem:
isql
tsql
osql
However, when I try it from Ruby code using the Sequel.odbc command, I receive the following error:
ODBC::Error: IM003 (0) [iODBC][Driver Manager]Specified driver could not be loaded.
This is as far as I can get. I used to receive another error first, but managed to solve that by redoing the configuration part. Guess I missed something there.
EDIT
This is the code for my base talker class. It basically loads a YAML file like rails does, holding the database settings and establishes a connection to the database.
This seems to be working, trying it manually returns me a DB object from sequel:
module Talkers
require 'yaml'
require 'sequel'
class BaseTalker
# This function will load the desired settings as a hash from the database.yml file
# in the config folder. The data will be returned as a hash following the standard
# YAML structure.
def self.load_config(name)
cfg = YAML::load(File.open(File.join(ENV['API_ROOT'], 'config', 'database.yml')))
cfg.key?(name) ? cfg[name] : nil
end
# This function will establish a connection with the Florensia User database and return
# the Sequel database object, so that queries can be executed against the database.
def self.connect_to_user_db
settings = self.load_config("florensia_user_#{ENV['RACK_ENV']}")
Sequel.odbc settings['dsn'], :db_type => settings['adapter'], :user => settings['user'], :password => settings['password']
end
end
end
The class below inherits from the talker and performs certain actions for a User. It contains the DB logic specific to the game. When I call this logic, I receive the errors:
module Talkers
require 'yaml'
require 'sequel'
class LoginDbTalker < BaseTalker
#
# Bans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly set:
# - id : The identifier of the account.
# - gm_name : The name of the GM setting the ban.
# - punish_code : The punishment code being applied on the account.
# - days : The duration of the ban in days, starting from today.
#
# The function will return true if the ban has been properly set; otherwise the function
# will return false.
def self.ban_user(options = {})
return false if options.empty?
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :apply_ban)
ps.call(
:punishcode => options[:punish_code],
:punishstory => "Banned by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => (options[:days].to_i == -1) ? (Date.today + (30 * 265)) : (Date.today + options[:days].to_i))
true
rescue Exception => e
puts "Exception caught in ban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}, punish_code=#{options[:punish_code]}, days=#{options[:days]}"
false
end
#
# Unbans the specified User from the game. The function requires the following information
# to be supplied in order for the ban to be properly lifted:
# - id : The identifier of the account.
# - gm_name : The name of the GM removing the ban.
#
# The function will return true if the ban has been properly lifted; otherwise the function
# will return false.
def self.unban_user(options = {})
db = self.connect_to_user_db
ds = db[:tbl_User].filter(:id => options[:id])
ps = ds.prepare(:update, :lift_ban)
ps.call(
:punishcode => '0',
:punishstory => "Ban removed by #{options[:gm_name]}",
:punishdate => Date.today,
:punishfreedate => Date.today
)
true
rescue Exception => e
puts "Exception caught in unban_user: #{e.to_s}"
puts "Provided variables: id=#{options[:id]}, gm_name=#{options[:gm_name]}"
false
end
#
# Kicks the specified User from the game, regardless on the server or character he currently is on.
# This requires a direct connection to the game servers so a specialized command can be sent that
# instruct the server to close the connection with the offending player.
# The function returns true if the kick succeeded; otherwise false.
def self.kick_player(id)
false
end
end
end
Calling any of the ban/unban functions results in the error message.
EDIT2
I've added the folder /Library/ODBC and linked all config files to there for iODBC. This removes the error I had before and now brings me this error:
ODBC::Error: 01000 (20002) [FreeTDS][SQL Server]Adaptive Server connection failed
So it seems I made some progress again
I recommend you use the tinytds adapter instead of the odbc adapter if you are connecting to Microsoft SQL Server from OS X. I know there are people who have got Sequel/ODBC running on non-Windows boxes, but I only have experience with Sequel/ODBC on Windows.

Using Open-URI to fetch XML and the best practice in case of problems with a remote url not returning/timing out?

Current code works as long as there is no remote error:
def get_name_from_remote_url
cstr = "http://someurl.com"
getresult = open(cstr, "UserAgent" => "Ruby-OpenURI").read
doc = Nokogiri::XML(getresult)
my_data = doc.xpath("/session/name").text
# => 'Fred' or 'Sam' etc
return my_data
end
But, what if the remote URL times out or returns nothing? How I detect that and return nil, for example?
And, does Open-URI give a way to define how long to wait before giving up? This method is called while a user is waiting for a response, so how do we set a max timeoput time before we give up and tell the user "sorry the remote server we tried to access is not available right now"?
Open-URI is convenient, but that ease of use means they're removing the access to a lot of the configuration details the other HTTP clients like Net::HTTP allow.
It depends on what version of Ruby you're using. For 1.8.7 you can use the Timeout module. From the docs:
require 'timeout'
begin
status = Timeout::timeout(5) {
getresult = open(cstr, "UserAgent" => "Ruby-OpenURI").read
}
rescue Timeout::Error => e
puts e.to_s
end
Then check the length of getresult to see if you got any content:
if (getresult.empty?)
puts "got nothing from url"
end
If you are using Ruby 1.9.2 you can add a :read_timeout => 10 option to the open() method.
Also, your code could be tightened up and made a bit more flexible. This will let you pass in a URL or default to the currently used URL. Also read Nokogiri's NodeSet docs to understand the difference between xpath, /, css and at, %, at_css, at_xpath:
def get_name_from_remote_url(cstr = 'http://someurl.com')
doc = Nokogiri::XML(open(cstr, 'UserAgent' => 'Ruby-OpenURI'))
# xpath returns a nodeset which has to be iterated over
# my_data = doc.xpath('/session/name').text # => 'Fred' or 'Sam' etc
# at returns a single node
doc.at('/session/name').text
end

Resources