Fetching a single sheet from a Google Spreadsheet as bytes in Ruby - ruby

I'm trying to retrieve a single sheet from a google spreadsheet in excel format, I have all the access setup correctly and can run different google sheet v4 api functions on it.
I wanted to use the Google::Apis::SheetsV4::SheetsService::copy_spreadsheet function to copy a single sheet as mentioned in the Ruby example here - https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.sheets/copyTo
This is my code -
service = Google::Apis::SheetsV4::SheetsService.new
service.client_options.application_name = APPLICATION_NAME
service.authorization = authorize
spreadsheet_id = "<passing my spreadsheet id here>"
gid = "<setting this as my sheet id from the spreadsheet>"
request = Google::Apis::SheetsV4::CopySheetToAnotherSpreadsheetRequest.new(
destination_spreadsheet_id: "0",
)
response1 = service.copy_spreadsheet(spreadsheet_id,gid,request)
puts response1.to_json
This always fails with the following error -
/usr/local/lib/ruby/gems/3.1.0/gems/google-apis-core-0.4.2/lib/google/apis/core/http_command.rb:229:in `check_status': badRequest: Invalid destinationSpreadsheetId [0] (Google::Apis::ClientError)
from /usr/local/lib/ruby/gems/3.1.0/gems/google-apis-core-0.4.2/lib/google/apis/core/api_command.rb:134:in `check_status'
Would be great if someone can help me on how to use this properly, also if there's a better way to download/export a single sheet from a spreadsheet in Ruby let me know.

Answer for question 1
This always fails with the following error -
/usr/local/lib/ruby/gems/3.1.0/gems/google-apis-core-0.4.2/lib/google/apis/core/http_command.rb:229:in `check_status': badRequest: Invalid destinationSpreadsheetId [0] (Google::Apis::ClientError)
from /usr/local/lib/ruby/gems/3.1.0/gems/google-apis-core-0.4.2/lib/google/apis/core/api_command.rb:134:in `check_status'
Would be great if someone can help me on how to use this properly,
When I saw your error message and your script, I thought that destination_spreadsheet_id: "0", is not correct. In this case, please set the destination Spreadsheet ID. When this is reflected in your script, it becomes as follows.
src_spreadsheet_id = "###" # Please set the source Spreadsheet ID.
src_sheet_id = "###" # Please set the sheet ID of the source Spreadsheet.
dst_spreadsheet_id = "###" # Please set the destination Spreadsheet ID.
request = Google::Apis::SheetsV4::CopySheetToAnotherSpreadsheetRequest.new(
destination_spreadsheet_id: dst_spreadsheet_id,
)
response1 = service.copy_spreadsheet(src_spreadsheet_id,src_sheet_id,request)
puts response1.to_json
Answer for question 2
I'm trying to retrieve a single sheet from a google spreadsheet in excel format, I have all the access setup correctly and can run different google sheet v4 api functions on it.
also if there's a better way to download/export a single sheet from a spreadsheet in Ruby let me know.
In this case, how about the following sample script? In this script, a XLSX data including the specific sheet is downloaded using the endpoint of https://docs.google.com/spreadsheets/d/{spreadsheetId}/export?format=xlsx&gid={sheetId}. So, please set your Spreadsheet ID and sheet ID to the URL. And, in this case, the access token is retrieved from service you are using.
url = 'https://docs.google.com/spreadsheets/d/{spreadsheetId}/export?format=xlsx&gid={sheetId}'
filename = 'sample.xlsx' # Please set the saved filename.
access_token = service.request_options.authorization.access_token
open(
url,
"Authorization" => "Bearer " + access_token,
:redirect => true
) do |file|
open(filename, "w+b") do |out|
out.write(file.read)
end
end
When this script is run, the specific sheet of sheetId of spreadsheetId is downloaded as a XLSX data and save it.
This script uses require "open-uri".
Note:
When the Spreadsheet is downloaded as a XLSX data, when an error related to the scope, please add https://www.googleapis.com/auth/drive.readonly to the scopes and reauthorize again. By this, the script works.
Reference:
Method: spreadsheets.sheets.copyTo

Related

Google sheets IMPORTXML fails for ASX data

I am trying to extract the "Forward Dividend & Yield" value from https://finance.yahoo.com/ for multiple companies in different markets, into Google Sheets.
This is successful:
=IMPORTXML("https://finance.yahoo.com/quote/WBS", "//*[#id='quote-summary']/div[2]/table/tbody/tr[6]/td[2]")
But this fails with #N/A:
=IMPORTXML("https://finance.yahoo.com/quote/CBA.AX", "//*[#id='quote-summary']/div[2]/table/tbody/tr[6]/td[2]")
I cannot work out what needs to be different for ASX ticker codes, why does CBA.AX cause a problem?
Huge thanks for any help
When I tested the formula of =IMPORTXML("https://finance.yahoo.com/quote/CBA.AX", "//*"), an error of Error Resource at url not found. occurred. I thought that this might be the reason of your issue.
But, fortunately, when I try to retrieve the HTML from the same URL using Google Apps Script, the HTML could be retrieved. So, in this answer, I would like to propose to retrieve the value using the custom function created by Google Apps Script. The sample script is as follows.
Sample script:
Please copy and paste the following script to the script editor of Google Spreadsheet and save it. And, please put a formula of =SAMPLE("https://finance.yahoo.com/quote/CBA.AX") to a cell. By this, the value is retrieved.
function SAMPLE(url) {
const res = UrlFetchApp.fetch(url).getContentText().match(/DIVIDEND_AND_YIELD-value.+?>(.+?)</);
return res && res.length > 1 ? res[1] : "No value";
}
Result:
When above script is used, the following result is obtained.
Note:
When this script is used, you can also use =SAMPLE("https://finance.yahoo.com/quote/WBS").
In this case, when the HTML structure of the URL is changed, this script might not be able to be used. I think that this situation is the same with IMPORTXML and the xpath. So please be careful this.
References:
Custom Functions in Google Sheets
Class UrlFetchApp
An other solution is to decode the json contained in the source of the web page. Of course you can't use importxml since the web page is built on your side by javascript and not on server's side. You can access data by this way and get a lot of informations
var source = UrlFetchApp.fetch(url).getContentText()
var jsonString = source.match(/(?<=root.App.main = ).*(?=}}}})/g) + '}}}}'
i.e. for what you are looking for you can use
function trailingAnnualDividendRate(){
var url='https://finance.yahoo.com/quote/CBA.AX'
var source = UrlFetchApp.fetch(url).getContentText()
var jsonString = source.match(/(?<=root.App.main = ).*(?=}}}})/g) + '}}}}'
var data = JSON.parse(jsonString)
var dividendRate = data.context.dispatcher.stores.QuoteSummaryStore.summaryDetail.trailingAnnualDividendRate.raw
Logger.log(dividendRate)
}

How to scrape the data using requests module only in python

I am actually trying to parse a website using the requests module, and extract some text out of it.
Url : https://www.icsi.in/student/Members/MemberSearch.aspx
after hitting the url in the Cp Number text field input : 16803
hit search,
on the bottom you can see some data, I want that data, let's say a name.
I am successfully able to get the data using selenium, but can't able to get it using requests module.
I have tried the requests module giving parameters, sessions, cookies etc.
but nothing worked.
url = "https://www.icsi.in/student/Members/MemberSearch.aspx"
ss = {'dnn$ctr410$MemberSearch$txtCpNumber':'16803',
'__EVENTTARGET':'dnn$ctr410$MemberSearch$btnSearch',
'__VIEWSTATEGENERATOR':'6A295697',
'dnn$ctlHeader$dnnSearch$Search':'SiteRadioButton'}
session = requests.Session()
cookies = session.cookies.get_dict()
for cookie in cookies:
session.cookies.set(cookie['name'], cookie['value'])
response = requests.post(url, data=ss)
print(response)
HTMLTree = html.fromstring(response.content)
name = HTMLTree.xpath('//div[#class="name_head"]//text()')
print(name)
I expect the output of the name of the person.
Anyone out there please help me.
If you don't mind using C# code I would be more than happy to help you otherwise it's a very lengthy process. If you choose that python is the only road you're willing to take then you should try grabbing the encrypted value within C:\User[USERNAME]\Appdata\Local\Google\Chrome\User Data\Default\Cookies You can change the file path accordingly to your OS. You can use SQLite to read and modify the encrypted values.
cookie = Decrypt(Encoding.Default.GetBytes(SQLDatabase1.GetValue(i, "encrypted_value")
if (cookie.Contains(".ASPXANONYMOUS")):
Step1 = cookie + "END"
Step2 = (step1 + ".ASPXANONYMOUS")
The following code above may help you with your journey.

Using DocuSign API and Ruby

I am getting started with Docusign, and even with Ruby. I am trying to get my head wrapped around something and I need your help.
I am trying in Ruby to create a helper in order to prefill a contract with the information I posses such as the client's information etc. And have that sent to the 2 people who need the contract. I am a little stuck in the steps I need to take in order to get the communication working and grabbing the information I need.
Any help or tips is appreciated.
I have installed the gem for the docusign API, I have the docusign-ruby-client code, I have also created my template, got my integrator key. I am stuck with the next step. How do I actually use in my project all these things together in order to automatically fill in the template with the information from the client ? –
EDITS:
I am unsure on how to use this below in order to get my template, fill it automatically with my client info, and when done, send the pre-filled contract to 2 people.
host = 'https://demo.docusign.net/restapi'
integrator_key = 'e5fdb0cd-a206-4934-91f1-5d00000000'
user_id = 'myemail#gmail.com'
expires_in_seconds = 3600 #1 hour
auth_server = 'account-d.docusign.com'
private_key_filename = '[REQUIRED]'
# STEP 1: Initialize API Client
configuration = DocuSign_eSign::Configuration.new
configuration.host = host
api_client = DocuSign_eSign::ApiClient.new(configuration)
api_client.configure_jwt_authorization_flow(private_key_filename, auth_server, integrator_key, user_id, expires_in_seconds)
# STEP 2: Initialize Authentication API using the API Client
authentication_api = DocuSign_eSign::AuthenticationApi.new(api_client)
# STEP 3: Make the login call
login_options = DocuSign_eSign::LoginOptions.new
login_information = authentication_api.login(login_options)
if !login_information.nil?
login_information.login_accounts.each do |login_account|
if login_account.is_default == "true"
# STEP 4: Extract the user information
base_url = login_account.base_url
account_id = login_account.account_id
puts base_url
puts account_id
# IMPORTANT: Use the base url from the login account to update the api client which will be used in future api calls
base_uri = URI.parse(base_url)
api_client.config.host = "%s://%s/restapi" % [base_uri.scheme, base_uri.host]

Scrapy xpath aria-select=false

I am trying to get the transcription info from some Khan Academy videos using scrapy.
For example: https://www.khanacademy.org/math/algebra-basics/basic-alg-foundations/alg-basics-negative-numbers/v/opposite-of-a-number
When I Tried to select the Transcript button through xpath response.xpath('//div[contains(#role, "tablist")]/a').extract() I only got the information about the tab has the aria-selected="true" which is the About section. I would need to use scrapy to change the aria-selected from false to true in the Transcript button and then retrieve the necessary information.
Could anyone please clarify how I would be able to accomplish this?
Much appreciated !
If you take a look at your network inspect you can see that an AJAX request is being made to retrieve the transcript once the page loads:
In this case it's https://www.khanacademy.org/api/internal/videos/2Zk6u7Uk5ow/transcript?casing=camel&locale=en&lang=en
It seems to use youtube video url id to create this api url. So you can recreate it really easily:
import json
import scrapy
class MySpider(scrapy.Spider):
#...
transcript_url_template = 'https://www.khanacademy.org/api/internal/videos/{}/transcript?locale=en&lang=en'
def parse(self, response):
# find youtube id
youtube_id = response.xpath("//meta[#property='og:video']/#content").re_first('v/(.+)')
# create transcript API url using the youtube id
url = self.transcript_url_template.format(youtube_id)
# download the data and parse it
yield Request(url, self.parse_transript)
def parse_transcript(self, response):
# convert json data to python dictionary
data = json.loads(response.body)
# parse your data!

Google Spreadsheet API - returns remote 500 error

Has anyone battled 500 errors with the Google spreadsheet API for google domains?
I have copied the code in this post (2-legged OAuth): http://code.google.com/p/google-gdata/source/browse/trunk/clients/cs/samples/OAuth/Program.cs, substituted in my domain;s API id and secret and my own credentials, and it works.
So it appears my domain setup is fine (at least for the contacts/calendar apis).
However swapping the code out for a new Spreadsheet service / query instead, it reverts to type: remote server returned an internal server error (500).
var ssq = new SpreadsheetQuery();
ssq.Uri = new OAuthUri("https://spreadsheets.google.com/feeds/spreadsheets/private/full", "me", "mydomain.com");
ssq.OAuthRequestorId = "me#mydomain.com"; // can do this instead of using OAuthUri for queries
var feed = ssservice.Query(ssq); //boom 500
Console.WriteLine("ss:" + feed.Entries.Count);
I are befuddled
I had to make sure to use the "correct" class:
not
//using SpreadsheetQuery = Google.GData.Spreadsheets.SpreadsheetQuery;
but
using SpreadsheetQuery = Google.GData.Documents.SpreadsheetQuery;
stinky-malinky
Seems you need the gdocs api to query for spreadsheets, but the spreadsheet api to query inside of a spreadsheet but nowhere on the internet until now will you find this undeniably important tit-bit. Google sucks hard on that one.

Resources