Ruby & IMAP - Accessing Office 365 with Oauth 2.0 - ruby

So MS disabled IMAP for basic auth as we all know.
I am trying to figure out how to get the OAUTH 2.0 working using ruby (not ruby on rails).
I have Azure APP and everything needed (I think), but I can not find any code related to ruby and getting the access token.
First step is completed, but next step is to get the access token.
https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
I need to read different Outlook mailboxes.
Could someone please explain how to do this?

SOLUTION for me!
Steps I took.
Made an Azure app ('Device Flow' was the easiest way to go for me) Check the Steps in the link. You also need to change some settings in your APP if you want to use IMAP. See the youtube link here between 2:50 - 4:30
Get the postman requests from this link (scroll down a little) (click here)
From postman you can use "Device Flow" requests.
Start with Device Authorization Request (you need a scope and client_id for this) I used https://outlook.office.com/IMAP.AccessAsUser.All scope.
go to the link that you got back from the request and enter the required code.
now go to Device Access Token Request and use the "device_code" from the last request and put that under code, under body.
You should get an access_token
Connect using ruby
require 'gmail_xoauth' # MUST HAVE! otherwise XOAUTH2 auth wont work
require 'net/imap'
imap = Net::IMAP.new(HOST, PORT, true)
access_token = "XXXXX"
user_name = "email#outlook.com"
p imap.authenticate('XOAUTH2',"#{user_name}", "#{access_token}")
# example
imap.list('','*').each do |folders|
p folders
end
XOAUTH2 Returns
#<struct Net::IMAP::TaggedResponse tag="RUBY0001", name="OK", data=#<struct Net::IMAP::ResponseText code=nil, text="AUTHENTICATE completed.">, raw_data="RUBY0001 OK AUTHENTICATE completed.\r\n
Just to specify
HOST = 'outlook.office365.com'
PORT = 993
UPDATE 25.01.2023
class Oauth2
require 'selenium-webdriver'
require 'webdrivers'
require 'net/http'
# Use: Oauth2.new.get_access_code
# Grants access to Office 365 emails.
def get_access_code
p "### Access Request Started #{Time.now} ###"
begin
codes = device_auth_request
authorize_device_code(codes[:user_code])
access_code = device_access_token(codes[:device_code])
access_code
rescue => e
p e
p "Something went wrong with authorizing"
end
end
def device_auth_request # Returns user_code and device_code
url = URI('https://login.microsoftonline.com/organizations/oauth2/v2.0/devicecode')
https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true
request = Net::HTTP::Post.new(url)
request.body = "client_id=YOUR_CLIENT_ID&scope=%09https%3A%2F%2Foutlook.office.com%2FIMAP.AccessAsUser.All"
response = https.request(request)
{
user_code: JSON.parse(response.read_body)["user_code"],
device_code: JSON.parse(response.read_body)["device_code"]
}
end
def device_access_token(device_code)
url = URI('https://login.microsoftonline.com/organizations/oauth2/v2.0/token')
https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true
request = Net::HTTP::Post.new(url)
request.body = "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code&code=#{device_code}&client_id=YOUR_CLIENT_ID"
response = https.request(request)
JSON.parse(response.read_body)["access_token"]
end
def authorize_device_code(device_code)
# SELENIUM SETUP
driver = setup_selenium
driver.get "https://microsoft.com/devicelogin"
sleep(4)
# ------------------------------------------
# Give Access
element = driver.find_element(:class, "form-control")
element.send_keys(device_code)
sleep(2)
element = driver.find_element(:id, "idSIButton9")
element.submit
sleep(2)
element = driver.find_element(:id, "i0116")
element.send_keys("YOUR OUTLOOK ACCOUNT EMAIL")
sleep(2)
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
element = driver.find_element(:id, "i0118")
element.send_keys("YOUR OUTLOOK PASSWORD")
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
element = driver.find_element(:class, "button_primary")
element.click
sleep(2)
# ------------------------------------------
driver.quit
end
def setup_selenium
require 'selenium-webdriver'
# set up Selenium
options = Selenium::WebDriver::Chrome::Options.new(
prefs: {
download: {
prompt_for_download: false
},
plugins: {
'always_open_pdf_externally' => true
}
}
)
options.add_argument('--headless')
options.add_argument('--no-sandbox')
# options.add_argument('-incognito')
options.add_argument('disable-popup-blocking')
Selenium::WebDriver.for :chrome, options: options
end
end

Related

Unable to upload and image in googledrive using goolge ruby API

I am first time using Google APIs. I am unable to upload any file in Google Drive. I tried below complete code.
require 'google/apis/drive_v2'
require 'google/api_client/client_secrets'
# I downloaded 'client_secrets.json' file from 'https://console.developers.google.com/projectselector/apis/library' and put in lib folder
CLIENT_SECRETS_FILE = "client_secrets.json"
client_secrets_filepath = File.expand_path(CLIENT_SECRETS_FILE ,"#{File.dirname(__FILE__)}/../../lib/")
CLIENT_SECRETS = Google::APIClient::ClientSecrets.load(client_secrets_filepath)
authorization = CLIENT_SECRETS.to_authorization
Drive = Google::Apis::DriveV2
#drive = Drive::DriveService.new
#drive.authorization = authorization
file_path = File.expand_path(#ScreenShot_dir)+'/'+"imageName" +'.png'
metadata = Drive::File.new(title: 'My document')
metadata = #drive.insert_file(metadata, upload_source: file_path, content_type: 'image/png')
It is not uploading the file in drive but giving an error like "missing authorization code".
my client_secrets.json look like below:
{"installed":{
"client_id":"<some digits>.apps.googleusercontent.com",
"project_id":"<projectname>","auth_uri":"https://accounts.google.com/o/oauth2/auth",
"token_uri":"https://accounts.google.com/o/oauth2/token",
"auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs",
"client_secret":"<secret key>",
"redirect_uris":["urn:ietf:wg:oauth:2.0:oob","http://localhost"]}}
I am not sure what I am missing in it. Appreciate any help on this issue.
"missing authorization code"
Means that you have not properly authenticated your code.
You should check the Ouath2 documentation for the client library. Get one of the samples there working then you should be able to alter it for Drive without to much trouble.
# AdSense Management API command-line sample.
require 'google/apis/adsense_v1_4'
require 'google/api_client/client_secrets'
require 'google/api_client/auth/installed_app'
require 'google/api_client/auth/storage'
require 'google/api_client/auth/storages/file_store'
require 'logger'
require 'json'
CREDENTIAL_STORE_FILE = "#{$0}-oauth2.json"
# Handles authentication and loading of the API.
def setup
log_file = File.open('adsense.log', 'a+')
log_file.sync = true
logger = Logger.new(log_file)
logger.level = Logger::DEBUG
adsense = Google::Apis::AdsenseV1_4::AdSenseService.new
# Stores auth credentials in a file, so they survive multiple runs
# of the application. This avoids prompting the user for authorization every
# time the access token expires, by remembering the refresh token.
# Note: FileStorage is not suitable for multi-user applications.
storage = Google::APIClient::Storage.new(
Google::APIClient::FileStore.new(CREDENTIAL_STORE_FILE))
adsense.authorization = storage.authorize
if storage.authorization.nil?
client_secrets = Google::APIClient::ClientSecrets.load
# The InstalledAppFlow is a helper class to handle the OAuth 2.0 installed
# application flow, which ties in with Stroage to store credentials
# between runs.
flow = Google::APIClient::InstalledAppFlow.new(
:client_id => client_secrets.client_id,
:client_secret => client_secrets.client_secret,
:scope => ['https://www.googleapis.com/auth/adsense.readonly']
)
adsense.authorization = flow.authorize(storage)
end
return adsense
end
# Generates a report for the default account.
def generate_report(adsense)
report = adsense.generate_report(start_date: '2011-01-01', end_date: '2011-08-31',
metric: %w(PAGE_VIEWS AD_REQUESTS AD_REQUESTS_COVERAGE
CLICKS AD_REQUESTS_CTR COST_PER_CLICK
AD_REQUESTS_RPM EARNINGS),
dimension: %w(DATE),
sort: %w(+DATE))
# Display headers.
report.headers.each do |header|
print '%25s' % header.name
end
puts
# Display results.
report.rows.each do |row|
row.each do |column|
print '%25s' % column
end
puts
end
end
if __FILE__ == $0
adsense = setup()
generate_report(adsense)
end

Ruby Gmail API with OAUTH2 responds with Invalid credentials (Failure) when logging in

I am trying to connect to the Gmail api using the Gmail for Ruby gem. I'm following this google oauth2 guide for installed applications.
I have set my app up in the Google Developer's Console, I am able to send a request with my client_id and client_secret to obtain an authorization code. I am then able to send a request with my authorization code to obtain an access token and a refresh token. I am able to successfully send a request to refresh my access token, and it returns a new access token.
The problem arises when I try to connect to Gmail. First I set an instance variable #gmail = Gmail.connect(:xoauth2, #email, #client.access_token). Then, I attempt to login with #gmail.login(true). However, when I try that, I get the following error:
Couldn't login to given Gmail account: caiden.robinson35#gmail.com (Invalid credentials (Failure)) (Gmail::Client::AuthorizationError)
I am at a loss here, everything suggests I'm executing the oauth2 flow correctly, except the fact that when it comes time to login, I get invalid credentials. When generating my authorization code, I specifically click my email and allow my app to have access. The API is also enabled in my developers console. Here is the full code:
class GmailClient
def initialize
load_client_info
#email = "caiden.robinson35#gmail.com"
load_and_set_oauth2_tokens
sign_in_gmail
binding.pry
end
private
def sign_in_gmail
binding.pry
#gmail = Gmail.connect(:xoauth2, #email, #client.access_token)
######################
# RIGHT HERE IS WHERE IT FAIL
######################
#gmail.login true
binding.pry
end
def load_client_info
gmail_credentials = YAML.load_file('config/gmail.yml')
#client_id = gmail_credentials["client_id"]
#client_secret = gmail_credentials["client_secret"]
#redirect_uri = 'urn:ietf:wg:oauth:2.0:oob'
end
def load_and_set_oauth2_tokens use_cached_tokens = true
if use_cached_tokens && File.exist?("config/tokens.yml")
token_hash = YAML.load_file('config/tokens.yml')
#authorization_code = { code: token_hash["authorization_code"],
is_cached: true }
#client = signet_client(token_hash)
#token_hash = #client.refresh!
else
if !(instance_variable_defined?("#authorization_code") && #authorization_code[:is_cached] == false)
retrieve_and_set_authorization_code
end
#token_hash = set_client_and_retrieve_oauth2_tokens
end
write_tokens_to_file
end
def retrieve_and_set_authorization_code
puts "Go to the following url to enable the gmail cli app:"
puts "https://accounts.google.com/o/oauth2/auth?scope=email&redirect_uri=urn:ietf:wg:oauth:2.0:oob&response_type=code&client_id=#{#client_id}"
print "Paste your authorization code here: "
#authorization_code = { code: gets.chomp,
is_cached: false }
end
def set_client_and_retrieve_oauth2_tokens
#client = signet_client
#client.fetch_access_token!
end
def signet_client token_hash = nil
client = Signet::OAuth2::Client.new(
client_id: #client_id,
client_secret: #client_secret,
redirect_uri: #redirect_uri,
scope: 'email',
token_credential_uri: 'https://www.googleapis.com/oauth2/v4/token'
)
if token_hash.present?
client.refresh_token = token_hash["refresh_token"]
else
client.authorization_uri = 'https://accounts.google.com/o/oauth2/auth'
client.code = #authorization_code[:code]
end
client
end
def write_tokens_to_file
if File.exist?("config/tokens.yml")
data = YAML.load_file "config/tokens.yml"
#token_hash.each { |k, v| data[k] = v }
File.open('config/tokens.yml', 'w') do |file|
YAML.dump(data, file)
end
else
File.open('config/tokens.yml', 'w') do |file|
#token_hash.each { |k, v| file.write("#{k}: #{v}\n") }
file.write("authorization_code: #{#authorization_code[:code]}\n")
end
end
end
end
If my question is lacking any info, please just ask, I am eager to solve this.
Scopes matter. Here are right ones:
scope: ['https://mail.google.com/', 'https://www.googleapis.com/auth/userinfo.email' #,'https://www.googleapis.com/auth/gmail.send' - if you'd like to send emails as well]

Ruby 2.2.0 Google Drive OAuth refresh

I am trying to setup a command line backup app in ruby which accesses Google Drive using Oauth. I have set everything up in the account, created my Client ID and Secret. I seem to remember doing this before but cannot find the code. I used this answer before I think: Ruby google_drive gem oAuth2 saving
I have made a class to handle the Google Drive stuff then there is the applications main file which if given "hard" as an argument will do the initial setup where you have to copy and paste the link into the web browser in order to get a code which you can then paste into the CLI to get the initial access token and refresh token. This works and following these steps my list method (when not commented out) correctly lists everything in Google Drive. When I do the initial setup I am manually copying the access and refresh tokens to .access_token and .refresh_token, these are loading in the code fine.
What is not working, is refreshing the token which I understand I need to do otherwise it will expire, meaning I will have to go through the initial setup again which is obviously a pain (and not suitable for a CRON job). I am getting the following error when I run #auth.refresh!
/home/user/.rvm/gems/ruby-2.2.0/gems/signet-0.6.0/lib/signet/oauth_2/client.rb:947:in `fetch_access_token': Authorization failed. Server message: (Signet::AuthorizationError)
{
"error" : "invalid_grant"
}
from /home/user/.rvm/gems/ruby-2.2.0/gems/signet-0.6.0/lib/signet/oauth_2/client.rb:964:in `fetch_access_token!'
from /home/user/.rvm/gems/ruby-2.2.0/gems/signet-0.6.0/lib/signet/oauth_2/client.rb:981:in `refresh!'
from /home/user/Development/BackupBadger/Sources/Mechanisms/GoogleDriveMechanism.rb:62:in `connect'
from BackupBadger.rb:9:in `<main>'
I have had a look to see what it might be but am for the moment stuck on why this error is triggering when I can seemingly authenticate (since I can list all files on the drive)
My main file
$root=File.join('/home/user/Development/BackupBadger')
$sources=File.join($root,'Sources')
require File.join($sources,'Mechanisms','GoogleDriveMechanism.rb')
badger = BackupBadger::GoogleDriveMechanism.new
if ARGV[0] == "hard" then
badger.hard_setup
else
badger.connect
#badger.list
end
My class Google Drive
module BackupBadger
require 'google/api_client'
require 'google_drive'
require 'pp'
require File.join($sources,'Mechanism.rb')
class GoogleDriveMechanism
def initialize()
#client = Google::APIClient.new(
:application_name => 'Backup Badger',
:application_version => '0.0.1'
)
#access_token_path = File.join($root,'.access_token')
#refresh_token_path = File.join($root,'.refresh_token')
#auth = nil
#access_token = File.open(#access_token_path, "rb").read
#refresh_token = File.open(#refresh_token_path, "rb").read
#session = nil
#client_id = 'CLIENT_ID'
#client_secret = 'CLIENT_SECRET'
#redirect_uri = 'urn:ietf:wg:oauth:2.0:oob'
#scope = "https://www.googleapis.com/auth/drive " +
"https://spreadsheets.google.com/feeds/"
end
# Call this to do the initial setup, which requires pasting a url into a web broswer
def hard_setup
#auth = #client.authorization
#auth.client_id = #client_id
#auth.client_secret = #client_secret
#auth.scope = #scope
#auth.redirect_uri = #redirect_uri
print("1. Open this page:\n%s\n\n" % #auth.authorization_uri)
print("2. Enter the authorization code shown in the page: ")
#auth.code = $stdin.gets.chomp
#auth.fetch_access_token!
#access_token = #auth.access_token
system'clear'
print "Save your access token\n\n"
print #access_token
print "\nSave your refresh token\n\n"
print #auth.refresh_token
end
def connect
#auth = #client.authorization
#auth.client_id = #client_id
#auth.client_secret = #client_secret
#auth.scope = #scope
#auth.redirect_uri = #redirect_uri
#auth.refresh_token = #refresh_token
puts #access_token
puts #refresh_token
# Error is here
#auth.refresh!
#refresh_token = #auth.refresh_token
#access_token = #auth.access_token
File.write(#refresh_token_path, #refresh_token) if #refresh_token
File.write(#access_token_path, #access_token) if #access_token
puts #access_token
puts #refresh_token
end
def list
#session = GoogleDrive.login_with_oauth(#access_token)
for file in #session.files
p file.title
end
end
end
end
If the tl;dr is simply "how do I use a refresh token to get a new access token", then the answer is https://developers.google.com/accounts/docs/OAuth2WebServer#refresh
I won't paste the code coz it's likely to change, but in essence you simply POST to a Google URL and the JSON response is a shiny new access token.

HTTP Basic Authentication with Anemone Web Spider

I need collect all "title" from all pages from site.
Site have HTTP Basic Auth configuration.
Without auth I do next:
require 'anemone'
Anemone.crawl("http://example.com/") do |anemone|
anemone.on_every_page do |page|
puts page.doc.at('title').inner_html rescue nil
end
end
But I have some problem with HTTP Basic Auth...
How I can collected titles from site with HTTP Basic Auth?
If I try use "Anemone.crawl("http://username:password#example.com/")" then I have only first page title, but other links have http://example.com/ style and I received 401 error.
HTTP Basic Auth works via HTTP headers. Client, willing to access restricted resource, must provide authentication header, like this one:
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
It contains name and password, Base64-encoded. More info is in Wikipedia article: Basic Access Authentication.
I googled a little bit and didn't find a way to make Anemone accept custom request headers. Maybe you'll have more luck.
But I found another crawler that claims it can do it: Messie. Maybe you should give it a try
Update
Here's the place where Anemone sets its request headers: Anemone::HTTP. Indeed, there's no customization there. You can monkeypatch it. Something like this should work (put this somewhere in your app):
module Anemone
class HTTP
def get_response(url, referer = nil)
full_path = url.query.nil? ? url.path : "#{url.path}?#{url.query}"
opts = {}
opts['User-Agent'] = user_agent if user_agent
opts['Referer'] = referer.to_s if referer
opts['Cookie'] = #cookie_store.to_s unless #cookie_store.empty? || (!accept_cookies? && #opts[:cookies].nil?)
retries = 0
begin
start = Time.now()
# format request
req = Net::HTTP::Get.new(full_path, opts)
response = connection(url).request(req)
finish = Time.now()
# HTTP Basic authentication
req.basic_auth 'your username', 'your password' # <<== tweak here
response_time = ((finish - start) * 1000).round
#cookie_store.merge!(response['Set-Cookie']) if accept_cookies?
return response, response_time
rescue Timeout::Error, Net::HTTPBadResponse, EOFError => e
puts e.inspect if verbose?
refresh_connection(url)
retries += 1
retry unless retries > 3
end
end
end
end
Obviously, you should provide your own values for the username and password params to the basic_auth method call. It's quick and dirty and hardcode, yes. But sometimes you don't have time to do things in a proper manner. :)

Ruby - remote file download ...timeout?

Excuse the tabs. I'm trying to download a file from remote to local and I keep getting one back that is exactly 310 bytes (regardless of what file I choose to download). I tried setting the timeout to 0, but this isn't working. What am I doing wrong? Thanks!
#downloadUrl = 'https://username:password#api.net'
Net::HTTP.start(#downloadUrl) do |http|
response = http.get('/file.ext')
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
EDIT: I don't want to use httpclient, I want to use standard net/http. I am almost there, but I keep getting initialize': getaddrinfo: nodename nor servname provided, or not known (SocketError) thrown at Net::HTTP.start(url.path). But when I remote "https", I get ECONNREFUSED. ...Getting closer?
url = URI.parse('https://api.net/file.ext')
#request = Net::HTTP.start(url.path)
#request.basic_auth 'username', 'password'
sock = Net::HTTP.new(url.host, 443)
sock.use_ssl = true
sock.ssl_version='SSLv3'
sock.start do |http|
response = http.get(#request)
open('/Users/me/file.ext', "wb", :read_timeout=>0) do |file|
file.write(response.body)
end
end
Using httpclient is much simpler when accessing via SSL.
gem install httpclient
I haven't tried this out, but this should work for you. Here is the rdoc.
require 'httpclient'
domain = "https://api.net/file.ext"
username = "username"
password = "password"
clnt = HTTPClient.new
clnt.set_auth(domain, username, password)
res = clnt.get_content(https_url)
You can refer to the "writing a binary file in ruby" question for saving your response to a file.
Updated Using net/http
You should be doing Net::HTTP.start(url.host, url.port) instead of Net:HTTP.start(url.path). See RDoc for Net::HTTP.start.
url = URI.parse("https://api.net/file.ext")
download_file = opne("/Users/me/file.ext", "wb")
request = Net::HTTP.start(url.host, url.port)
# .. set basic auth, verify peer etc
begin
request.request_get(url.path) do |resp|
resp.read_body { |segment| download_file.write(segment) }
end
ensure
download_file.close
end

Resources