I'm trying to scrape this page:
https://github.com/search?p=1&q=https%3A%2F%2Fsonarcloud.io%2Fdashboard&type=Code
and I need to authenticate with my email and password.
I have tried to do that:
auth = {:usarname => "username", :password => "password"}
a = HTTParty.get(url, :basic_auth)
but this didn't authenticate me as expected.
Why isn't this working, and how can I fix it?
I want retrive that information, and isn't available on the Github API:
Don't scrape GitHub. Scraping is fragile, and very awkward with sites that make heavy use of JavaScript.
Use its API instead:
https://api.github.com/search/code?q=https%3A%2F%2Fsonarcloud.io%2Fdashboard
To search across all repositories you'll still need to authenticate, though. You need to pass your auth hash into HTTParty.get():
auth = {:username => "username", :password => "password"}
a = HTTParty.get(url, :basic_auth => auth)
# ^ Here
More idiomatically, this might look like
auth = {username: "username", password: "password"}
a = HTTParty.get(url, basic_auth: auth)
You also have a typo—usarname instead of username—which I've fixed in my version.
Edit: If you want to retrieve the specific matched text, file, and lines you still don't have to scrape their HTML. Instead, you can set your Accept header to application/vnd.github.v3.text-match+json:
url = "https://api.github.com/search/code"
query = {q: "https://sonarcloud.io/dashboard"}
auth = {username: "username", password: "password"}
headers = {"Accept" => "application/vnd.github.v3.text-match+json"}
a = HTTParty.get(url, query: query, basic_auth: auth, headers: headers)
The response should now give a text_matches key containing hashes with fragments showing the matched text as well as object_types (e.g. "FileContent"), object_urls, and indices.
This is also mentioned in the search code link I already provided:
When searching for code, you can get text match metadata for the file content and file path fields when you pass the text-match media type. For more details about how to receive highlighted search results, see Text match metadata.
Related
The Google API client has Class: Google::Apis::PeopleV1::Name and Google::Apis::PeopleV1::Photo how do I hit these endpoints? I tried
p = Google::Apis::PeopleV1::Photo.new
response = p.url
But it returns nil. Opposed to this GmailService has instance methods like get_user_message which I can call like this
#service = Google::Apis::GmailV1::GmailService.new
# authorization is done
response = #service.get_user_message(user_id, message_id)
It returns a response object which has the message, now I want to fetch the User's name and Photo, but I did not find any method to access this information. How do I retrieve the User's name and photo using the API client?
I think that the value retrieved with the method of users.messages.get in Gmail API doesn't include "User Profile Picture". Ref So for example, in order to retrieve the URL of user's profile photo, how about using the method of people.connections.list in People API? The sample script is as follows.
Sample script:
service = Google::Apis::PeopleV1::PeopleServiceService.new
service.authorization = authorize
response = service.list_person_connections(
'people/me',
'page_size': 1000,
'person_fields': "names,emailAddresses,photos"
)
puts "No connections" if response.connections.empty?
result = response.connections.map {|person| {
'resourceName': person.resource_name,
'email': person.email_addresses[0].value,
'photos': person.photos.map {|e| e.url}
}}
puts result
Result:
When above script is run, the following result is returned.
[
{"resourceName": "people/###", "email": "###", "photos": ["https://###"]},
{"resourceName": "people/###", "email": "###", "photos": ["https://###", "https://###"]},
,
,
,
]
The URLs of phtots are the URLs of the user's photo images.
Note:
In this case, it seems that the method of otherContacts.list doesn't return the user's photo data.
Reference:
Method: people.connections.list
I am trying to send a patch request to edit a video using vimeo api using ruby. The request is successful but vimeo is not able to read my the payload(the title and description), that is, the the title and the description is not getting changed. I have used HTTParty, RestClient as well as Net::HTTP but none of it works. Below are my code that i have implemented to send a patch request,
RestClient
payload = {"description" => "Test Description", "name" => "Test Video"}
res = RestClient.patch(
vimeo_edit_url,
payload.to_s,
{ "Authorization" => auth })
NET::HTTP
options = {'description' => "Test Description", 'name' => "Test Video"}
edit_req = Net::HTTP::Patch.new(vimeo_edit_url, initheader = { "Authorization" => auth})
edit_req.data = options.to_s
edit_uri = URI(vimeo_edit_url)
edit_http = Net::HTTP.new(edit_uri.host, edit_uri.port)
edit_http.use_ssl = true
edit_http.verify_mode = OpenSSL::SSL::VERIFY_PEER
edit_response = edit_http.request(edit_req)
Usually the response will be a 200 OK with the updated video details but I get a 200 OK with video details(title and description not being changed). It is as if like vimeo is not able to read my payload.
You probably want to be passing payload, not payload.to_s.
That .to_s is turning your nicely arranged hash into a weird rubyified string, and all of the HTTP libraries you mentioned will accept a hash and handle the conversion for you.
Here's what some different representations look like in an irb session:
>> payload = {"description" => "Test Description", "name" => "Test Video"}
>> payload
=> {"description"=>"Test Description", "name"=>"Test Video"}
>> payload.to_s
=> "{"description"=>"Test Description", "name"=>"Test Video"}"
>> require 'json'
=> true
>> payload.to_json
=> "{"description":"Test Description","name":"Test Video"}"
I'm having troubles sending a draft in Gmail through their API and the documentation doesn't help very much, especially since I'm working with Ruby.
I can create a draft without any issue, but then when I try to send the newly created draft, I get an error saying:
ArgumentError (wrong number of arguments (0 for 1))
The involved code is as follows:
#gmail = client.discovered_api('gmail', 'v1')
#send_result = client.execute(
:api_method => #gmail.users.drafts.send,
:parameters => { 'userId' => 'me' },
:body_object => { 'id' => '<message_id>' }
)
Taking a look at the debugger, the error seems to appear because of this:
#gmail.users.drafts.send
What am I missing here? I haven't seen anywhere that I should be passing parameters into the api_method? Also where can I find where this is documented and what is the parameter supposed to be?
Thanks!
The question is pretty old at this point, but I just ran into the same problem and figured it's better to answer late than never.
#gmail.users.drafts.send is colliding with Ruby's Object#send. You can work around the collision by converting the Google::APIClient::Resource item to a hash and then reading the value by key:
:api_method => #gmail.users.drafts.to_h["gmail.users.drafts.send"]
Your example, including the workaround:
#gmail = client.discovered_api('gmail', 'v1')
#send_result = client.execute(
:api_method => #gmail.users.drafts.to_h["gmail.users.drafts.send"],
:parameters => { 'userId' => 'me' },
:body_object => { 'id' => '<message_id>' }
)
I hope that helps!
I'm just going off of:
https://developers.google.com/gmail/api/v1/reference/users/drafts/send
But I think you have it right. The userId should be a parameter (e.g. in the URL) and the draft ID should be in the (POST) body. Can you confirm you're actually providing a draft ID and not the message.id?
Are you able to get an HTTP trace of the actual request, that would help immensely (you should likely be able to set this on the client or underlying http library your client uses, etc).
I am having trouble figuring out how to add a calendar to all of a domain's users' CalendarList.
So far I can successfully create the Calendar and create a domain wide ACL but I don't know how to insert the calendar into domain users' lists.
Using the ruby client API it looks something like this:
client = Google::APIClient.new
service = client.discovered_api("calendar", "v3")
calendar_response = JSON.parse(client.execute(:api_method => service.calendars.insert,
:body => JSON.dump({"summary" => "events"}),
:headers => { "Content-Type" => "application/json" }).response.body)
rule = {
"scope" => {
"type" => "domain",
"value" => domain,
},
"role" => "writer"
}
acl_result = client.execute(:api_method => service.acl.insert,
:parameters => { "calendarId" => calendar_response["id"] },
:body => JSON.dump(rule),
:headers => { "Content-Type" => "application/json" })
This all works fine. It creates the calendar and it is shared with everyone in the domain. What I can't figure out is how to explicitly added it to the users' list of Calendars.
To add to the single authed user's calendar list would look like this:
list_params = {
"calendarId" => calendar_response["id"],
"hidden" => false,
"selected" => true
}
calendar_list_result = client.execute(:api_method => service.calendar_list.insert,
:body => JSON.dump(list_params),
:headers => { "Content-Type" => "application/json" })
Questions:
If I am authenticated as a domain admin, can I create CalendarList item for all users?
If so, can it be done with a single command or do I need to make a call with every user id?
If I need to do a command for each user, how do I get the user ids?
If I am authenticated as a domain admin, can I create CalendarList item for all users?
No you can't :(.
If so, can it be done with a single command or do I need to make a call with every user id?
You have to make a call for each user id.
If I need to do a command for each user, how do I get the user ids?
To retrieve the user id you first must list all the users in the domain with the Admin SDK API, https://developers.google.com/admin-sdk/directory/v1/guides/manage-users#get_all_domain_users , then for each INSERT command in the calendar you change the user id, for the email address.
I believe that i have followed the documentation on SendGrid's site but so far when I receive an email that I've sent through their API it never substitutes the replacements i've specified in the x-smtpapi headers. I am using HTTParty to send the request like this:
HTTParty.post(Sendgrid::Postman.api_url, {
:query => params.merge({ "api_user" => #config[:api_user], "api_key" => #config[:api_key] }),
:headers => headers, :format => :json
})
the "params" look like this:
{"from"=>"noreply#foo.com", "text"=>"Happy Holidays -first_name- -last_name-,\nI hope this message finds you in good health and high spirits.", "to"=>["foo#gmail.com"], "subject"=>"foo"}
The Headers look like this:
{"X-SMTPAPI"=>"{\"sub\": {\"-first_name-\": [\"Foo\"], \"-email-\": [\"foo#gmail.com\"], \"-login-\": [\"heavysixer\"], \"-last_name-\": [\"Bar\"]}, \"to\": [\"foo#gmail.com\"]}"}
The mail is always successfully delivered but when it arrives in the inbox the values that were supposed to be substituted still appear like -first_name- & -last_name-
What am I missing? I've been messing around with this for a solid day now?
-----------------------------------------------------------
UPDATE:
Per the advice below I have tried to push the x-smtpapi params into the form post yet I am getting the same result. The query string for my post now looks like this:
params = {"api_user" => 'foo', "api_key" => 'bar', "from"=>"noreply#foo.com", "text"=>"Happy Holidays -first_name- -last_name-,\nI hope this message finds you in good health and high spirits.", "to"=>["foo#gmail.com"], "subject"=>"foo", "x-smtpapi"=>{"sub"=>{"-first_name-"=>["foo"], "-email-"=>["foo#gmail.com"], "-login-"=>["foo"], "-last_name-"=>["bar"]}}}
HTTParty.post(Sendgrid::Postman.api_url, :query => params, :format => :json)
Their documentation implies that x-smtpapi should be one of the posted parameters, not an http header.