I'm using the google API ruby client and I want to implement some more complex analytics queries such as suggested in this document
https://developers.google.com/analytics/devguides/reporting/core/v3/common-queries
This document suggests that metrics can be supplied as a comma delimited string of multiple metrics but the API client only accepts an expression.
How can I query on multiple metrics in a single query? The ruby client appears only to accept an expression which generally consists of a single metric such as sessions or pageviews like this:
metric = Google::Apis::AnalyticsreportingV4::Metric.new(expression: 'ga:sessions')
If I remove "expression" and enter a list of metrics I just get an error.
Invalid value 'ga:sessions;ga:pageviews' for metric parameter.
Here is my solution, together with a generic method for reporting Google Analytics data:
This answer should be read in conjunction with https://developers.google.com/drive/v3/web/quickstart/ruby
analytics = Google::Apis::AnalyticsreportingV4::AnalyticsReportingService.new
analytics.client_options.application_name = APPLICATION_NAME
analytics.authorization = authorize
def get_analytics_data( analytics,
view_id,
start_date: (Date.today + 1 - Date.today.wday) - 6,
end_date: (Date.today + 1 - Date.today.wday),
metrics: ['ga:sessions'],
dimensions: [],
order_bys: [],
segments: nil, # array of hashes
filter: nil,
page_size: nil )
get_reports_request_object = Google::Apis::AnalyticsreportingV4::GetReportsRequest.new
report_request_object = Google::Apis::AnalyticsreportingV4::ReportRequest.new
report_request_object.view_id = view_id
analytics_date_range_object = Google::Apis::AnalyticsreportingV4::DateRange.new
analytics_date_range_object.start_date = start_date
analytics_date_range_object.end_date = end_date
report_request_object.date_ranges = [analytics_date_range_object]
# report_request_metrics = []
report_request_object.metrics = []
metrics.each { |metric|
analytics_metric_object = Google::Apis::AnalyticsreportingV4::Metric.new
analytics_metric_object.expression = metric
report_request_object.metrics.push(analytics_metric_object) }
# report_request_object.metrics = report_request_metrics
unless dimensions.empty?
report_request_object.dimensions = []
dimensions.each { |dimension|
analytics_dimension_object = Google::Apis::AnalyticsreportingV4::Dimension.new
analytics_dimension_object.name = dimension
report_request_object.dimensions.push(analytics_dimension_object) }
end
unless segments.nil?
report_request_object.segments = []
analytics_segment_object = Google::Apis::AnalyticsreportingV4::Segment.new
analytics_dynamic_segment_object = Google::Apis::AnalyticsreportingV4::DynamicSegment.new
analytics_segment_definition_object = Google::Apis::AnalyticsreportingV4::SegmentDefinition.new
analytics_segment_filter_object = Google::Apis::AnalyticsreportingV4::SegmentFilter.new
analytics_simple_segment_object = Google::Apis::AnalyticsreportingV4::SimpleSegment.new
analytics_or_filters_for_segment_object = Google::Apis::AnalyticsreportingV4::OrFiltersForSegment.new
analytics_segment_filter_clause_object = Google::Apis::AnalyticsreportingV4::SegmentFilterClause.new
analytics_segment_metric_filter_object = Google::Apis::AnalyticsreportingV4::SegmentMetricFilter.new
analytics_dimension_object = Google::Apis::AnalyticsreportingV4::Dimension.new
analytics_dimension_object.name = 'ga:segment'
report_request_object.dimensions.push(analytics_dimension_object)
analytics_or_filters_for_segment_object.segment_filter_clauses = []
analytics_simple_segment_object.or_filters_for_segment = []
analytics_segment_definition_object.segment_filters = []
segments.each { |segment|
analytics_segment_metric_filter_object.metric_name = segment[:metric_name]
analytics_segment_metric_filter_object.comparison_value = segment[:comparison_value]
analytics_segment_metric_filter_object.operator = segment[:operator]
analytics_segment_filter_clause_object.metric_filter = analytics_segment_metric_filter_object
analytics_or_filters_for_segment_object.segment_filter_clauses.push(analytics_segment_filter_clause_object)
analytics_simple_segment_object.or_filters_for_segment.push(analytics_or_filters_for_segment_object)
analytics_segment_filter_object.simple_segment = analytics_simple_segment_object
analytics_segment_definition_object.segment_filters.push(analytics_segment_filter_object)
analytics_dynamic_segment_object.name = segment[:name]
analytics_dynamic_segment_object.session_segment = analytics_segment_definition_object
analytics_segment_object.dynamic_segment = analytics_dynamic_segment_object
report_request_object.segments.push(analytics_segment_object) }
end
unless order_bys.empty?
report_request_object.order_bys = []
order_bys.each { |orderby|
analytics_orderby_object = Google::Apis::AnalyticsreportingV4::OrderBy.new
analytics_orderby_object.field_name = orderby
analytics_orderby_object.sort_order = 'DESCENDING'
report_request_object.order_bys.push(analytics_orderby_object)}
end
unless filter.nil?
report_request_object.filters_expression = filter
end
unless page_size.nil?
report_request_object.page_size = page_size
end
get_reports_request_object.report_requests = [report_request_object]
response = analytics.batch_get_reports(get_reports_request_object)
end
If using dimensions, you can report data like this:
response = get_analytics_data(analytics, VIEW_ID, metrics: ['ga:pageviews'], dimensions: ['ga:pagePath'], order_bys: ['ga:pageviews'], page_size: 25)
response.reports.first.data.rows.each do |row|
puts row.dimensions
puts row.metrics.first.values.first.to_i
puts
end
Related
I am developing an application that allows to place orders with python and QtDesigner. I can't manage to place two commands in a row. The first command passes without any problem but when I want to place another command without closing the application, this error is displayed: "self.ui.treeWidgetcommand.topLevelItem(self.Line ).setText(0, str(Id))
AttributeError: 'NoneType' object has no attribute 'setText'".
def AddCommande(self):
QtWidgets.QTreeWidgetItem(self.ui.treeWidgetcommande)
Libelle = self.ui.comboBoxproduit.currentText()
Qte = int(self.ui.lineEditQteproduit.text())
Info = self.stock.GetProductName(Libelle)[0]
Id = str(int(Info[0]))
Pu = Info[1]
Total = int(Qte)*int(Pu)
data=(Libelle,Qte,Id,Pu,Total)
#print(data)
self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setText(0, str(Id))
self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setText(1, str(Libelle))
self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setText(2, str(Qte))
self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setText(3, str(Pu))
self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setText(4, str(Total))
self.Ligne +=1
def ValiderCommande(self):
Client = self.ui.comboBoxclient.currentText()
IdClient = self.stock.GetClientIdByName(Client.split(" ")[0])
PrixTotal = 0
UniqueId = random.random()
Date = date.today()
Data = (IdClient,PrixTotal,Date,UniqueId)
if self.stock.AddCommande(Data) == 0:
for i in range(self.Ligne):
IdCommande = self.stock.GetClientIdByUniqueId(UniqueId)
Libelle = self.ui.treeWidgetcommande.topLevelItem(i).text(1)
IdProduit = self.ui.treeWidgetcommande.topLevelItem(i).text(0)
Pu = self.ui.treeWidgetcommande.topLevelItem(i).text(3)
Qte = self.ui.treeWidgetcommande.topLevelItem(i).text(2)
Total = int(self.ui.treeWidgetcommande.topLevelItem(i).text(4))
InfoData = (IdCommande, Libelle, Qte, Pu, Total)
data = (Qte,IdProduit)
if self.stock.AjoutInfoCommande(InfoData) == 0:
PrixTotal += Total
self.stock.UpdateQteStock(data)
if self.stock.UpdateCommande(PrixTotal,IdCommande) == 0:
self.ui.treeWidgetcommande.clear()
#self.ui.treeWidgetcommande.topLevelItem(self.Ligne).setHidden(True)
self.ui.lineEditQteproduit.setText(" ")
`
I would like after placing an order, reset my treeWidget array and be able to place other orders.
I have a document that is using tabs to fill a document. The document is signed before and after completion of a task. Is it possible to modify tabs on an envelope - then re-generate an DocuSign_eSign::RecipientViewRequest (still having the initial signature / fields)?
Thus far I've been able to generate two DocuSign_eSign::RecipientViewRequest, but cannot figure out how to change the tabs in between signing:
PRE_SIGNER = 'pre_signer'
POST_SIGNER = 'post_signer'
PRIVATE_KEY = CREDENTIALS['private_key']
PUBLIC_KEY = CREDENTIALS['public_key']
USER_ID = CREDENTIALS['user_id']
CLIENT_ID = CREDENTIALS['client_id']
ACCOUNT_ID = CREDENTIALS['account_id']
BASE_URL = CREDENTIALS['base_url']
configuration = DocuSign_eSign::Configuration.new
configuration.host = "#{BASE_URL}/restapi"
configuration.debugging = true
api_client = DocuSign_eSign::ApiClient.new(configuration)
api_client.base_path = BASE_URL
envelope_api = DocuSign_eSign::EnvelopesApi.new(api_client)
pre_signer_text = DocuSign_eSign::Text.new
pre_signer_text.value = 'Alpha'
pre_signer_text.tab_label = 'pre_value'
pre_signer = DocuSign_eSign::Signer.new
pre_signer.role_name = PRE_SIGNER
pre_signer.client_user_id = PRE_SIGNER
pre_signer.recipient_id = 1
pre_signer.name = 'Kevin Sylvestre'
pre_signer.email = 'kevin#fake.com'
pre_signer.tabs = DocuSign_eSign::Tabs.new
pre_signer.tabs.text_tabs = [pre_signer_text]
post_signer = DocuSign_eSign::Signer.new
post_signer.role_name = POST_SIGNER
post_signer.client_user_id = POST_SIGNER
post_signer.recipient_id = 2
post_signer.name = 'Kevin Sylvestre'
post_signer.email = 'kevin#fake.com'
post_signer.tabs = DocuSign_eSign::Tabs.new
post_signer.tabs.text_tabs = []
server_template = DocuSign_eSign::ServerTemplate.new
server_template.sequence = 0
server_template.template_id = TEMPLATE_ID
inline_template = DocuSign_eSign::InlineTemplate.new
inline_template.sequence = 0
inline_template.recipients = DocuSign_eSign::Recipients.new
inline_template.recipients.signers = [
pre_signer,
post_signer,
]
composite_template = DocuSign_eSign::CompositeTemplate.new
composite_template.server_templates = [server_template]
composite_template.inline_templates = [inline_template]
envelope_event = DocuSign_eSign::EnvelopeEvent.new
envelope_event.envelope_event_status_code = 'completed'
envelope_definition = DocuSign_eSign::EnvelopeDefinition.new
envelope_definition.status = 'sent'
envelope_definition.composite_templates = [composite_template]
api_client.request_jwt_user_token(CLIENT_ID, USER_ID, PRIVATE_KEY)
envelope = envelope_api.create_envelope(ACCOUNT_ID, envelope_definition)
pre_signer_recipient_view_request = DocuSign_eSign::RecipientViewRequest.new
pre_signer_recipient_view_request.authentication_method = 'none'
pre_signer_recipient_view_request.client_user_id = PRE_SIGNER
pre_signer_recipient_view_request.user_name = 'Kevin Sylvestre'
pre_signer_recipient_view_request.email = 'kevin#fake.com'
pre_signer_recipient_view_request.return_url = 'https://ksylvest.com'
pre_recipient_view = envelope_api.create_recipient_view(ACCOUNT_ID, envelope.envelope_id, pre_signer_recipient_view_request)
url = pre_recipient_view.url
`open #{url}`
puts "Continue?"
gets
# at this point I'd like to enter values for tabs...
post_signer_text = DocuSign_eSign::Text.new
post_signer_text.value = 'Omega'
post_signer_text.tab_label = 'post_value'
post_signer_recipient_view_request = DocuSign_eSign::RecipientViewRequest.new
post_signer_recipient_view_request.authentication_method = 'none'
post_signer_recipient_view_request.client_user_id = POST_SIGNER
post_signer_recipient_view_request.user_name = 'Kevin Sylvestre'
post_signer_recipient_view_request.email = 'kevin#fake.com'
post_signer_recipient_view_request.return_url = 'https://ksylvest.com'
post_recipient_view = envelope_api.create_recipient_view(ACCOUNT_ID, envelope.envelope_id, post_signer_recipient_view_request)
url = post_recipient_view.url
`open #{url}`
You could add the same person to sign twice, as two separate recipients that are the same person. You can generate different recipient views. You can set the routing order to be different. Only reason I didn't post this as an answer is that you may mean that you need to pause the envelope?
you can add tabs using your code where you have post_signer.tabs, but if you want to modify existing tabs that came from the template then you have to create the envelope in draft mode ("created") and then make a different API call to modify the tabs and then a final API call to send it. Another option is to pause the envelope and "correct" it.
Pause envelope workflow code examples
https://github.com/docusign/docusign-esign-ruby-client/blob/c477b07c2f578214fdf7d0c5a33355f01e9a0b4e/lib/docusign_esign/api/envelopes_api.rb#L6132 update_recipients() method should do the trick...
I have this structure:
$ArrayX = [8349310431,8349314513,......]
$ArrayY = [667984788,667987788,......]
$ArrayZ = [148507632380,153294624079,.....]
$range_map = $ArrayX.zip([$ArrayY.map(&:to_i),
$ArrayZ.map(&:to_i)].transpose).sort
puts $range_map ={[8349310431=>[667984788, 148507632380],
8349314513=>[667987788, 153294624079]}
I need the key to be compared with the rest of the keys and if the subtraction between keys is lower than 100, that key to print
I corrected your code also as per your need, and solved further,
$ArrayX = [8349310431,8349314513]
$ArrayY = [667984788,667987788]
$ArrayZ = [148507632380,153294624079]
$range_map = $ArrayX.zip([$ArrayY.map(&:to_i), $ArrayZ.map(&:to_i)].transpose).sort
$ArrayX = [8349310431,8349314513]
=> [8349310431, 8349314513]
$ArrayY = [667984788,667987788]
=> [667984788, 667987788]
$ArrayZ = [148507632380,153294624079]
=> [148507632380, 153294624079]
$range_map = Hash[$ArrayX.zip([$ArrayY.map(&:to_i), $ArrayZ.map(&:to_i)].transpose).sort]
=> {8349310431=>[667984788, 148507632380], 8349314513=>[667987788, 153294624079]}
keys = $range_map.keys
valid_keys = keys.select { |k| keys.detect { |x| (x-k).abs > 100 } }
$range_map.slice(*valid_keys)
If particular key is having difference more than 100 with one of rest of keys then it will be valid for filtering.
I am trying to crawl a website. I want to do it for different dates. So i am storing date in a list. But while trying to access items of list, crawler works only for 1st value in list. Please help. following is my code:
class SpidyQuotesViewStateSpider(scrapy.Spider):
name = 'retail_price'
def start_requests(self):
print "start request"
urls = "http://fcainfoweb.nic.in/PMSver2/Reports/Report_Menu_web.aspx"
yield scrapy.Request(url=urls, callback=self.parse)
def parse(self, response):
dated = ["05/03/2017","04/03/2017"]
urls = "http://fcainfoweb.nic.in/PMSver2/Reports/Report_Menu_web.aspx"
#frmdata =
cookies1 ={}
val = response.headers.getlist('Set-Cookie')
print "login session values" ,response.headers.getlist('Set-Cookie')
if(len(val) != 0):
cookies1['ASP.NET_SessionId'] = str(response.headers.getlist('Set-Cookie')[0].split(";")[0].split("=")[1])
cookies1['path'] = str(response.headers.getlist('Set-Cookie')[0].split(";")[1].split("=")[1])
print cookies1;
for i in range(len(dated)):
yield scrapy.FormRequest(url=urls, callback=self.parse1, formdata={'ctl00$MainContent$btn_getdata1':"Get Data",
'ctl00$MainContent$Txt_FrmDate':dated[i],
'ctl00$MainContent$Ddl_Rpt_Option0':"Daily Prices",
'ctl00$MainContent$Rbl_Rpt_type':"Price report",
'ctl00$MainContent$ddl_Language':"English",
'ctl00$MainContent$Ddl_Rpt_type':"Retail",
'__EVENTVALIDATION':"IwZyKgfTXVzxiHxiPXGk/W8XQZBDb0EOPxJh6s8hofq0ffqOpiHSH77CafcxySF3PbkYgSMNFCJhLM2cGnL6SxT0PJuGDCJtV0V8Y4a94UErUCiSANiin+4uKckk9v9Ux8JqTVeaipppmlH+wyks2U9SgPfkNUsqw4eHCkDyB5akNNZImRIixOHHVY3JSXGkwXn7ueK9w+AgnqJzpXaWdMr9J1++M4VAFImSNF8brFSfPHe5kb/qzkGIwUr/KRouaRYK8WLWZh/Mbl9xwREwhDSxWJSOdihSE0WWoaqSMtpaR99rDDCsD3mdJqfu0aPIlREupTZRzlrmztXU0eS3949YW+ywdTRvykaMNgOW2Q4saYP5j/niKbRW6GiDnaLV2A38X/HW80+trrsjwJr9tjTKVFyikf6s/3gzyiTp11ivSkwIY2b3hutjYn7OfTDo",
#'__EVENTVALIDATION':"HqVo2xHk04clYwnBposXbZGhbIr181A7RbyeZv74Cia7rXSKmpOpbeSnn3XXnoDJKRxMK0W9nxKZFfkNje+P/K7gE5HVjHJr9Gr0Gs46TntzKDsvzyii8jZ7e0fdZgQCJKoXxQNgR2vNkWqChKcEldBuMHCOgJRqCNCF/JPFKpdKZoIWr7GU8rhzwLijf/Gkm+FuTULs/fl2HHK6Z1QQEozzEHFsDwzl0G4IiN//eNYfHuUBXKZ3wdZzPqG0s53WHEuSBzhqBC9AtCJOs4ZZhdtwFh8iyTJ4PlsLP9DLHYHRCOAd72UO0UH8gT7gAkKVo1I4L540DilowOR9SttH7MM/oOs9qhKlnG61FgqkYGW8zGzF/yNEXO+beVAK1RVvuO+FDnuq/g36TRnUieei5GpAZ+96CSoCIxykdvHx8R+smTNF/5erlowV4ci+tcI7",
'__VIEWSTATEENCRYPTED':"",
'__VIEWSTATEGENERATOR':"85862B00",
'__VIEWSTATE':"+a+3jrBEKxDdkPOzx2wXwKaTMWvCB60WPaHRfJUAZQrdFIpxSqFr5VseTclpGzeHXdxaFnxJe/PkxDKYa7sj3Wiv/os1bNeX0IEB3s45eFsHYWGiU8cvsXCGa5z7rrGRDL5hotg7k/MuUWj8w27xXZO423MN5OsHS+wh+tC/5/Xix+w3zxuQhi8jR5DnreimHbhGZn1sYaKYIGCc8mDIDRNl+w1OZ058F+3LAx96QUu5BYiMYOmrlyxrb9b2yPTmmIrI4NtC4ClBQlxuST5wMDP3vUqqWMhn4auk8ev5gHyPestCRrsAXWs07wDNnikemMwo/4wPiTEbnZQV6SLcDUw0gZpXjXwLI7mhsVjEyVNaQnJp6+Wi6FLsAEEMlFYmQut3JecpVIUkjF9uYSN2GLIbXHPs37AiEXPeQ8E/GyBMx3z1X5l8sw/xSNmFgYQC3riajn8V0+SdkuV2PbNbYKtc+uoSCNLppLYCqiOv5eWanGvAQro2Q67FBA4w2xY+V/K8mzHaGMLoDBxJxLslWyJpL5cX0C6qoXVUu8B028auAQM4eVzH1YPF5qrJiCDo",
#'__VIEWSTATE':"W+m8kNAS6QHiRPo+zFj00EDs/Dbq+y/XvtCmSNwOIkGKlikAlphT8HBAWQDskSm1vdNterBuo0Hy7m4xPbXMOnyEm6IlseXO3jPw+ofnI2WHAKknLil+GeS0IfMWGeoD5aNyiz3zh1jZkKU7R7hQsxwARoHRyjhf8UCooFbkVvL6ddHVYZbH5LcocmCF1BTOCqYN5y5yzfDfYbp3KNW9kH53pdmwCsjiEirdxxUGDoG1Ke3JBEXfSl+4XubirHSR8z+VlFmPPXZGU8mMogwq9Eg822RYjvbwvZG74djcf7kdfB9KXCPO9u6cWIjLiW+cfXHSXD+1XYFVf9ATU2/NV4YbUzsI4PJRwoGD4BryUNIm2JFeT4c8F4REYTA16shxz5mDTFQ6rbmg6SmqP8G9gAc2Hr9ABD8+2BUNabGhNZ8wDIZArfYS4pl5DNrlPlpqeCjhmvv0znKAJSOac3pCUej8G90ZGwQKOPORWbNVzQShoH7QvrXV8pCklcia6psuAGO+Oj72oDWPxedE4DjdjX5TbLoW4bzsk/YNfUv4JpjGR8DWpG8IFYJG9CCjMEYb",
'__LASTFOCUS':"",
'__EVENTARGUMENT':"",
'__EVENTTARGET':"",
'ctl00_MainContent_ToolkitScriptManager1_HiddenField':";;AjaxControlToolkit,+Version=4.1.51116.0,+Culture=neutral,+PublicKeyToken=28f01b0e84b6d53e:en-US:fd384f95-1b49-47cf-9b47-2fa2a921a36a:475a4ef5:addc6819:5546a2b:d2e10b12:effe2a26:37e2e5c9:5a682656:c7029a2:e9e598a9"},method='POST',cookies = cookies1)
def parse1(self, response):
path1 = "id('Panel1')"
value1 = response.xpath(path1).extract_first()
print value1
First of all, you are sending the spider more time on the same site, though with different form parameters. You have therefore to use dont_filter=True in the request, otherwise Scrapy blocks duplicate calls.
Then it seems to me that the site you are scraping don't allow you to make more than one request during the same session. Try for example to go to http://fcainfoweb.nic.in/PMSver2/Reports/Report_Menu_web.aspx with your browser, compile the form, get the data and than to go back to the initial page: It's impossible. So you have to modify your spider. Here's a very rough code just to give an idea. It works for me, but please don't use it in production!
class SpidyQuotesViewStateSpider(scrapy.Spider):
name = 'retail_price'
urls = "http://fcainfoweb.nic.in/PMSver2/Reports/Report_Menu_web.aspx"
def start_requests(self):
dated = ["01/03/2017","05/03/2017","04/03/2017"]
for i in dated:
request = scrapy.Request(url=self.urls, dont_filter=True, callback=self.parse)
request.meta['question'] = i
yield request
def parse(self, response):
thedate = response.meta['question']
cookies1 ={}
val = response.headers.getlist('Set-Cookie')
print("login session values" ,response.headers.getlist('Set-Cookie'))
if(len(val) != 0):
cookies1['ASP.NET_SessionId'] = str(str(response.headers.getlist('Set-Cookie')[0]).split(";")[0].split("=")[1])
cookies1['path'] = str(str(response.headers.getlist('Set-Cookie')[0]).split(";")[1].split("=")[1])
yield scrapy.FormRequest(url=self.urls, dont_filter=True, callback=self.parse1, formdata={'ctl00$MainContent$btn_getdata1':"Get Data",
'ctl00$MainContent$Txt_FrmDate': thedate,
'ctl00$MainContent$Ddl_Rpt_Option0':"Daily Prices",
'ctl00$MainContent$Rbl_Rpt_type':"Price report",
'ctl00$MainContent$ddl_Language':"English",
'ctl00$MainContent$Ddl_Rpt_type':"Retail",
'__EVENTVALIDATION':"IwZyKgfTXVzxiHxiPXGk/W8XQZBDb0EOPxJh6s8hofq0ffqOpiHSH77CafcxySF3PbkYgSMNFCJhLM2cGnL6SxT0PJuGDCJtV0V8Y4a94UErUCiSANiin+4uKckk9v9Ux8JqTVeaipppmlH+wyks2U9SgPfkNUsqw4eHCkDyB5akNNZImRIixOHHVY3JSXGkwXn7ueK9w+AgnqJzpXaWdMr9J1++M4VAFImSNF8brFSfPHe5kb/qzkGIwUr/KRouaRYK8WLWZh/Mbl9xwREwhDSxWJSOdihSE0WWoaqSMtpaR99rDDCsD3mdJqfu0aPIlREupTZRzlrmztXU0eS3949YW+ywdTRvykaMNgOW2Q4saYP5j/niKbRW6GiDnaLV2A38X/HW80+trrsjwJr9tjTKVFyikf6s/3gzyiTp11ivSkwIY2b3hutjYn7OfTDo",
#'__EVENTVALIDATION':"HqVo2xHk04clYwnBposXbZGhbIr181A7RbyeZv74Cia7rXSKmpOpbeSnn3XXnoDJKRxMK0W9nxKZFfkNje+P/K7gE5HVjHJr9Gr0Gs46TntzKDsvzyii8jZ7e0fdZgQCJKoXxQNgR2vNkWqChKcEldBuMHCOgJRqCNCF/JPFKpdKZoIWr7GU8rhzwLijf/Gkm+FuTULs/fl2HHK6Z1QQEozzEHFsDwzl0G4IiN//eNYfHuUBXKZ3wdZzPqG0s53WHEuSBzhqBC9AtCJOs4ZZhdtwFh8iyTJ4PlsLP9DLHYHRCOAd72UO0UH8gT7gAkKVo1I4L540DilowOR9SttH7MM/oOs9qhKlnG61FgqkYGW8zGzF/yNEXO+beVAK1RVvuO+FDnuq/g36TRnUieei5GpAZ+96CSoCIxykdvHx8R+smTNF/5erlowV4ci+tcI7",
'__VIEWSTATEENCRYPTED':"",
'__VIEWSTATEGENERATOR':"85862B00",
'__VIEWSTATE':"+a+3jrBEKxDdkPOzx2wXwKaTMWvCB60WPaHRfJUAZQrdFIpxSqFr5VseTclpGzeHXdxaFnxJe/PkxDKYa7sj3Wiv/os1bNeX0IEB3s45eFsHYWGiU8cvsXCGa5z7rrGRDL5hotg7k/MuUWj8w27xXZO423MN5OsHS+wh+tC/5/Xix+w3zxuQhi8jR5DnreimHbhGZn1sYaKYIGCc8mDIDRNl+w1OZ058F+3LAx96QUu5BYiMYOmrlyxrb9b2yPTmmIrI4NtC4ClBQlxuST5wMDP3vUqqWMhn4auk8ev5gHyPestCRrsAXWs07wDNnikemMwo/4wPiTEbnZQV6SLcDUw0gZpXjXwLI7mhsVjEyVNaQnJp6+Wi6FLsAEEMlFYmQut3JecpVIUkjF9uYSN2GLIbXHPs37AiEXPeQ8E/GyBMx3z1X5l8sw/xSNmFgYQC3riajn8V0+SdkuV2PbNbYKtc+uoSCNLppLYCqiOv5eWanGvAQro2Q67FBA4w2xY+V/K8mzHaGMLoDBxJxLslWyJpL5cX0C6qoXVUu8B028auAQM4eVzH1YPF5qrJiCDo",
#'__VIEWSTATE':"W+m8kNAS6QHiRPo+zFj00EDs/Dbq+y/XvtCmSNwOIkGKlikAlphT8HBAWQDskSm1vdNterBuo0Hy7m4xPbXMOnyEm6IlseXO3jPw+ofnI2WHAKknLil+GeS0IfMWGeoD5aNyiz3zh1jZkKU7R7hQsxwARoHRyjhf8UCooFbkVvL6ddHVYZbH5LcocmCF1BTOCqYN5y5yzfDfYbp3KNW9kH53pdmwCsjiEirdxxUGDoG1Ke3JBEXfSl+4XubirHSR8z+VlFmPPXZGU8mMogwq9Eg822RYjvbwvZG74djcf7kdfB9KXCPO9u6cWIjLiW+cfXHSXD+1XYFVf9ATU2/NV4YbUzsI4PJRwoGD4BryUNIm2JFeT4c8F4REYTA16shxz5mDTFQ6rbmg6SmqP8G9gAc2Hr9ABD8+2BUNabGhNZ8wDIZArfYS4pl5DNrlPlpqeCjhmvv0znKAJSOac3pCUej8G90ZGwQKOPORWbNVzQShoH7QvrXV8pCklcia6psuAGO+Oj72oDWPxedE4DjdjX5TbLoW4bzsk/YNfUv4JpjGR8DWpG8IFYJG9CCjMEYb",
'__LASTFOCUS':"",
'__EVENTARGUMENT':"",
'__EVENTTARGET':"",
'ctl00_MainContent_ToolkitScriptManager1_HiddenField':";;AjaxControlToolkit,+Version=4.1.51116.0,+Culture=neutral,+PublicKeyToken=28f01b0e84b6d53e:en-US:fd384f95-1b49-47cf-9b47-2fa2a921a36a:475a4ef5:addc6819:5546a2b:d2e10b12:effe2a26:37e2e5c9:5a682656:c7029a2:e9e598a9"},method='POST',cookies = cookies1)
def parse1(self, response):
path1 = "id('Panel1')"
value1 = response.xpath(path1).extract_first()[:574]
print(value1)
I've been retrieving monthly invoice cost information on our SoftLayer accounts for quite some time using the Ruby softlayer gem. However, there is a concern in the team that we may be missing certain costs, such as any overages on network utilization. I'd like to have some piece of mind that what I'm doing is correctly gathering all costs and we are not missing anything. Here is my code/query:
account = SoftLayer::Service.new("SoftLayer_Account",:username => user, :api_key => api_key, :timeout => 999999999)
softlayer_client = SoftLayer::Client.new(:username => user, :api_key => api_key, :timeout => 999999999)
billing_invoice_service = softlayer_client.service_named("Billing_Invoice")
object_filter = SoftLayer::ObjectFilter.new
object_filter.set_criteria_for_key_path('invoices.createDate', 'operation' => 'betweenDate', 'options' => [{'name' => 'startDate', 'value' => ["#{startTime}"]}, {'name' => 'endDate', 'value' => ["#{endTime}"]}])
# Set startDate and endDate around the beginning of the month in search of the "Recurring" invoice that should appear on the 1st.
invoices = account.result_limit(0,10000).object_filter(object_filter).object_mask("mask[id,typeCode,itemCount,invoiceTotalAmount,closedDate,createDate]").getInvoices
invoices.each do | invoice |
if invoice["typeCode"] == "RECURRING"
invoice_reference = billing_invoice_service.object_with_id(invoice["id"])
invoice_object = invoice_reference.object_mask("mask[itemCount]").getObject
billing_items_count = invoice_object["itemCount"]
billing_machines_map = Hash.new
all_billing_items = Array.new
# Search for billing items containing a hostName value.
# The corresponding billing item ID will become the key of a new hash.
# Child costs will be added to the existing costs.
billing_items_retrieval_operation = proc {
for i in 0..(billing_items_count/8000.0).ceil - 1
billing_items = invoice_reference.result_limit(i*8000, 8000).object_mask("mask[id,resourceTableId,billingItemId,parentId,categoryCode,hostName,domainName,hourlyRecurringFee,laborFee,oneTimeFee,recurringFee,recurringTaxAmount,setupFee,setupTaxAmount,location[name]]").getItems()
billing_items.each do | billing_item |
if billing_item["hostName"]
billing_machines_map[billing_item["id"]] = billing_item
end
end
all_billing_items.concat(billing_items)
end
}
# Look for items with parentIds or resourceTableIds.
# Both Ids represent a "parent" of the item.
# Give higher importance to parentId.
billing_items_retrieval_callback = proc {
cost_of_billing_items_without_parent = BigDecimal.new("0.00")
all_billing_items.each do | billing_item |
if billing_item["parentId"] != ""
parent_billing_machine = billing_machines_map[billing_item["parentId"]]
if parent_billing_machine parent_billing_machine["recurringFee"] = (BigDecimal.new(parent_billing_machine["recurringFee"]) + BigDecimal.new(billing_item["recurringFee"])).to_s('F')
parent_billing_machine["setupFee"] = (BigDecimal.new(parent_billing_machine["setupFee"]) + BigDecimal.new(billing_item["setupFee"])).to_s('F')
parent_billing_machine["laborFee"] = (BigDecimal.new(parent_billing_machine["laborFee"]) + BigDecimal.new(billing_item["laborFee"])).to_s('F')
parent_billing_machine["oneTimeFee"] = (BigDecimal.new(parent_billing_machine["oneTimeFee"]) + BigDecimal.new(billing_item["oneTimeFee"])).to_s('F')
end
elsif billing_item["resourceTableId"] != ""
parent_billing_machine = billing_machines_map[billing_item["resourceTableId"]]
if parent_billing_machine
parent_billing_machine["recurringFee"] = (BigDecimal.new(parent_billing_machine["recurringFee"]) + BigDecimal.new(billing_item["recurringFee"])).to_s('F')
parent_billing_machine["setupFee"] = (BigDecimal.new(parent_billing_machine["setupFee"]) + BigDecimal.new(billing_item["setupFee"])).to_s('F')
parent_billing_machine["laborFee"] = (BigDecimal.new(parent_billing_machine["laborFee"]) + BigDecimal.new(billing_item["laborFee"])).to_s('F')
parent_billing_machine["oneTimeFee"] = (BigDecimal.new(parent_billing_machine["oneTimeFee"]) + BigDecimal.new(billing_item["oneTimeFee"])).to_s('F')
end
else
cost_of_billing_items_without_parent = (BigDecimal.new(cost_of_billing_items_without_parent) + BigDecimal.new(billing_item["recurringFee"])).to_s('F')
cost_of_billing_items_without_parent = (BigDecimal.new(cost_of_billing_items_without_parent) + BigDecimal.new(billing_item["setupFee"])).to_s('F')
cost_of_billing_items_without_parent = (BigDecimal.new(cost_of_billing_items_without_parent) + BigDecimal.new(billing_item["laborFee"])).to_s('F')
cost_of_billing_items_without_parent = (BigDecimal.new(cost_of_billing_items_without_parent) + BigDecimal.new(billing_item["oneTimeFee"])).to_s('F')
end
end
pp "INVOICE: Total cost of devices for account without a parent is:"
pp cost_of_billing_items_without_parent
end
end
end
After the above I make calls to getVirtualGuests and getHardware to get some additional meta information for each machine (I tie them together based on billingItem.id. Example:
billingItemId = billing_machine["billingItemId"]
account_service = softlayer_client.service_named("Account")
filter = SoftLayer::ObjectFilter.new {|f| f.accept("virtualGuests.billingItem.id").when_it is(billingItemId)}
virtual_guests_array = account_service.object_filter(filter).object_mask("mask[id, hostname, datacenter[name], billingItem[orderItem[order[userRecord[username]]]], tagReferences[tagId, tag[name]], primaryIpAddress, primaryBackendIpAddress]").getVirtualGuests()
As you can see I don't make any calls to capture bandwith overage charges. I have printed out the various "category" values I get from the above query but I am not seeing anything specific to network utilization (it's possible there are no extra network utilization costs but I am not certain).
Thank you.
Any extra costs such as bandwidth overages will be included in the billing item from the server. So you don't need to make any other call to the api to get it.