We have some apps (or maybe we should call them a handful of scripts) that use Google APIs to facilitate some administrative tasks. Recently, after making another client_id in the same project, I started getting an error message similar to the one described in localhost redirect_uri does not work for Google Oauth2 (results in 400: invalid_request error). I.e.,
Error 400: invalid_request
You can't sign in to this app because it doesn't comply with Google's
OAuth 2.0 policy for keeping apps secure.
You can let the app developer know that this app doesn't comply with
one or more Google validation rules.
Request details:
The content in this section has been provided by the app developer.
This content has not been reviewed or verified by Google.
If you’re the app developer, make sure that these request details
comply with Google policies.
redirect_uri: urn:ietf:wg:oauth:2.0:oob
How do I get through this error? It is important to note that:
The OAuth consent screen for this project is marked as "Internal". Therefore any mentions of Google review of the project, or publishing status are irrelevant
I do have "Trust internal, domain-owned apps" enabled for the domain
Another client id in the same project works and there are no obvious differences between the client IDs - they are both "Desktop" type which only gives me a Client ID and Client secret that are different
This is a command line script, so I use the "copy/paste" verification method as documented here hence the urn:ietf:wg:oauth:2.0:oob redirect URI (copy/paste is the only friendly way to run this on a headless machine which has no browser).
I was able to reproduce the same problem in a dev domain. I have three client ids. The oldest one is from January 2021, another one from December 2021, and one I created today - March 2022. Of those, only the December 2021 works and lets me choose which account to authenticate with before it either accepts it or rejects it with "Error 403: org_internal" (this is expected). The other two give me an "Error 400: invalid_request" and do not even let me choose the "internal" account. Here are the URLs generated by my app (I use the ruby google client APIs) and the only difference between them is the client_id - January 2021, December 2021, March 2022.
Here is the part of the code around the authorization flow, and the URLs for the different client IDs are what was produced on the $stderr.puts url line. It is pretty much the same thing as documented in the official example here (version as of this writing).
OOB_URI = 'urn:ietf:wg:oauth:2.0:oob'
def user_credentials_for(scope, user_id = 'default')
token_store = Google::Auth::Stores::FileTokenStore.new(:file => token_store_path)
authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store)
credentials = authorizer.get_credentials(user_id)
if credentials.nil?
url = authorizer.get_authorization_url(base_url: OOB_URI)
$stderr.puts ""
$stderr.puts "-----------------------------------------------"
$stderr.puts "Requesting authorization for '#{user_id}'"
$stderr.puts "Open the following URL in your browser and authorize the application."
$stderr.puts url
code = $stdin.readline.chomp
$stderr.puts "-----------------------------------------------"
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id, code: code, base_url: OOB_URI)
end
credentials
end
Please see https://stackoverflow.com/a/71491500/1213346 for a "proper" solution. This answer is just an ugly workaround that the community seems to like.
...
Here is a cringy workaround for this situation:
Replace urn:ietf:wg:oauth:2.0:oob with http://localhost:1/ in the code posted in the question. This makes the flow go through, my browser gets redirected and fails and I get an error messages like:
This site can’t be reached
The webpage at http://localhost:1/oauth2callback?
code=4/a3MU9MlhWxit8P7N8QsGtT0ye8GJygOeCa3MU9MlhWxit8P7N8QsGtT0y
e8GJygOeC&scope=email%20profile%20https... might be temporarily
down or it may have moved permanently to a new web address.
ERR_UNSAFE_PORT
Now copy the code code value from the failing URL, paste it into the app, and voila... same as before :)
P.S. Here is the updated "working" version:
def user_credentials_for(scope, user_id = 'default')
token_store = Google::Auth::Stores::FileTokenStore.new(:file => token_store_path)
authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store, "http://localhost:1/")
credentials = authorizer.get_credentials(user_id)
if credentials.nil?
url = authorizer.get_authorization_url
$stderr.puts ""
$stderr.puts "-----------------------------------------------"
$stderr.puts "Requesting authorization for '#{user_id}'"
$stderr.puts "Open the following URL in your browser and authorize the application."
$stderr.puts url
$stderr.puts
$stderr.puts "At the end the browser will fail to connect to http://localhost:1/?code=SOMECODE&scope=..."
$stderr.puts "Copy the value of SOMECODE from the address and paste it below"
code = $stdin.readline.chomp
$stderr.puts "-----------------------------------------------"
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id, code: code)
end
credentials
end ```
I sent off an email to someone on the Google OAuth team. This is the gist of their response.
As I feared your issue is related to Making Google OAuth interactions safer by using more secure OAuth flows
The current recommendation from google is to move to use localhost/loopback redirects as recommended here: instructions-oob or use the OAuth for devices flow if you are using non-sensitive scopes and need a headless solution.
A solution for python.
As google_auth_oauthlib shows, InstalledAppFlow.run_console has been deprecated after Feb 28, 2022. And if you are using google-ads-python, you can just replace flow.run_console() by flow.run_local_server().
Let me post the "proper" solution as a separate answer, which is to actually follow the recommended procedure by implementing an HTTP listener in the ruby app. If this is running on an offline machine the listener will never get the code, but you can still paste the code from the failing URL.
require 'colorize'
require 'sinatra/base'
# A simplistic local server to receive authorization tokens from the browser
def run_local_server(authorizer, port, user_id)
require 'thin'
Thin::Logging.silent = true
Thread.new {
Thread.current[:server] = Sinatra.new do
enable :quiet
disable :logging
set :port, port
set :server, %w[ thin ]
get "/" do
request = Rack::Request.new env
state = {
code: request["code"],
error: request["error"],
scope: request["scope"]
}
raise Signet::AuthorizationError, ("Authorization error: %s" % [ state[:error] ] ) if state[:error]
raise Signet::AuthorizationError, "Authorization code missing from the request" if state[:code].nil?
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id,
code: state[:code],
scope: state[:scope],
)
[
200,
{ "Content-Type" => "text/plain" },
"All seems to be OK. You can close this window and press ENTER in the application to proceed.",
]
end
end
Thread.current[:server].run!
}
end
# Returns user credentials for the given scope. Requests authorization
# if requrired.
def user_credentials_for(scope, user_id = 'default')
client_id = Google::Auth::ClientId.new(ENV['GOOGLE_CLIENT_ID'], ENV['GOOGLE_CLIENT_SECRET'])
token_store = Google::Auth::Stores::FileTokenStore.new(:file => ENV['GOOGLE_CREDENTIAL_STORE'])
port = 6969
redirect_uri = "http://localhost:#{port}/"
authorizer = Google::Auth::UserAuthorizer.new(client_id, scope, token_store, redirect_uri)
credentials = authorizer.get_credentials(user_id)
if credentials.nil? then
server_thread = run_local_server(authorizer, port, user_id)
url = authorizer.get_authorization_url
$stderr.puts ""
$stderr.puts "-----------------------------------------------"
$stderr.puts "Requesting authorization for '#{user_id.yellow}'"
$stderr.puts "Open the following URL in your browser and authorize the application."
$stderr.puts
$stderr.puts url.yellow.bold
$stderr.puts
$stderr.puts "⚠️ If you are authorizing on a different machine, you will have to port-forward"
$stderr.puts "so your browser can reach #{redirect_uri.yellow}"
$stderr.puts
$stderr.puts "⚠️ If you get a " << "This site can't be reached".red << " error in the browser,"
$stderr.puts "just copy the failing URL below. Copy the whole thing, starting with #{redirect_uri.yellow}."
$stderr.puts "-----------------------------------------------"
code = $stdin.readline.chomp
server_thread[:server].stop!
server_thread.join
credentials = authorizer.get_credentials(user_id)
# If the redirect failed, the user must have provided us with a code on their own
if credentials.nil? then
begin
require 'uri'
require 'cgi'
code = CGI.parse(URI.parse(code).query)['code'][0]
rescue StandardException
# Noop, if we could not get a code out of the URL, maybe it was
# not the URL but the actual code.
end
credentials = authorizer.get_and_store_credentials_from_code(
user_id: user_id,
code: code,
scope: scope,
)
end
end
credentials
end
credentials = user_credentials_for(['https://www.googleapis.com/auth/drive.readonly'])
In short, we run a web server expecting the redirect from the browser. It takes the code the browser sent, or it takes the code pasted by the user.
For headless Python scripts that need sensitive scopes, continuing to use run_console now produces the following (and the flow likely fails):
DeprecationWarning: New clients will be unable to use `InstalledAppFlow.run_console` starting on Feb 28, 2022. All clients will be unable to use this method starting on Oct 3, 2022. Use `InstalledAppFlow.run_local_server` instead. For details on the OOB flow deprecation, see https://developers.googleblog.com/2022/02/making-oauth-flows-safer.html?m=1#disallowed-oob
The official solution is to migrate to a flow that spins up a local server to handle the OAuth redirect, but this will not work on remote headless systems.
The solution Google adopted in gcloud is to run a local server on the same machine as the user's browser and then have the user copy the redirect URL requested from this local server back to the remote machine. Note that this requires having gcloud installed both on the remote machine and on the user's workstation.
As a hack for situations where installing a script to echo back the redirect URL on the workstation is not practical, we can use a redirect URL that is guaranteed to fail and just have the user copy back the URL of the error page on which they will land after authorization is complete.
import urllib
from google_auth_oauthlib.flow import InstalledAppFlow
def run_console_hack(flow):
flow.redirect_uri = 'http://localhost:1'
auth_url, _ = flow.authorization_url()
print(
"Visit the following URL:",
auth_url,
"After granting permissions, you will be redirected to an error page",
"Copy the URL of that error page (http://localhost:1/?state=...)",
sep="\n"
)
redir_url = input("URL: ")
query = urllib.parse.urlparse(redir_url).query
code = urllib.parse.parse_qs(query)['code'][0]
flow.fetch_token(code=code)
return flow.credentials
scopes = ['https://www.googleapis.com/auth/drive.file']
flow = InstalledAppFlow.from_client_secrets_file(secrets_file, scopes)
credentials = run_console_hack(flow)
We could also ask the user to pass back the code query string parameter directly but that is likely to be confusing and error-prone.
The use of 1 as the port number means that the request is guaranteed to fail, rather than potentially hit some service that happens to be running on that port. (e.g. Chrome will fail with ERR_UNSAFE_PORT without even trying to connect)
"Hello world" for this error:
Generating an authentication URL
https://github.com/googleapis/google-api-nodejs-client#generating-an-authentication-url
const {google} = require('googleapis');
const oauth2Client = new google.auth.OAuth2(
YOUR_CLIENT_ID,
YOUR_CLIENT_SECRET,
YOUR_REDIRECT_URL
);
// generate a url that asks permissions for Blogger and Google Calendar scopes
const scopes = [
'https://www.googleapis.com/auth/blogger',
'https://www.googleapis.com/auth/calendar'
];
const url = oauth2Client.generateAuthUrl({
// 'online' (default) or 'offline' (gets refresh_token)
access_type: 'offline',
// If you only need one scope you can pass it as a string
scope: scopes
});
If something goes wrong the first step is to Re Check again the three values of the google.auth.OAuth2 function.
1 of 2
Compare to the store values under Google APIs console:
YOUR_CLIENT_ID
YOUR_CLIENT_SECRET
YOUR_REDIRECT_URL -
For example http://localhost:3000/login
2 of 2 (environment variables)
A lot of times the values store inside .env. So re-check the env and the output under your files - for example index.ts (Even use console.log).
.env
# Google Sign-In (OAuth)
G_CLIENT_ID=some_id_1234
G_CLIENT_SECRET=some_secret_1234
PUBLIC_URL=http://localhost:3000
index
const auth = new google.auth.OAuth2(
process.env.G_CLIENT_ID,
process.env.G_CLIENT_SECRET,
`${process.env.PUBLIC_URL}/login`
);
SUM:
Something like this will not work
const oauth2Client = new google.auth.OAuth2(
"no_such_id",
"no_such_secret",
"http://localhost:3000/i_forgot_to_Authorised_this_url"
);
I've fixed this problem with recreate my App in google console. And I think the problem was with redirect_url. I had this problem when I was using 'Android' type of App in google console (in this case you can't configure redirect url). In my android App I'm using google auth with WebView so the best option here use use 'Web' type for your app in google console.
In my case, had to update plugins. by running following command-
bundle exec fastlane update_plugins
With this redirect uri was getting created properly as
https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=563584335869-fgrhgmd47bqnekij5i8b5pr03ho849e6.apps.googleusercontent.com&include_granted_scopes=true&redirect_uri=http://localhost:8081&response_type=code&scope=https://www.googleapis.com/auth/cloud-platform&state=2ce8a59b2d403f3a89fa635402bfc5c4
steps.oauth.v2.invalid_request 400 This error name is used for multiple different kinds of errors, typically for missing or incorrect parameters sent in the request. If is set to false, use fault variables (described below) to retrieve details about the error, such as the fault name and cause.
GenerateAccessToken GenerateAuthorizationCode
GenerateAccessTokenImplicitGrant
RefreshAccessToken
Google Oauth Policy
I'm trying to create a script to access the Quizlet API
Those API are protected with OAuth2 and I'm using this oauth2 ruby gem https://github.com/intridea/oauth2
The gem's GitHub page shows an example but for me doesn't work and I feel I'm missing some pieces.
This is part of the example:
client.auth_code.authorize_url(:redirect_uri => 'http://localhost:8080/oauth2/callback')
# => "https://example.org/oauth/authorization?response_type=code&client_id=client_id&redirect_uri=http://localhost:8080/oauth2/callback"
token = client.auth_code.get_token('authorization_code_value', :redirect_uri => 'http://localhost:8080/oauth2/callback', :headers => {'Authorization' => 'Basic some_password'})
Two quetions:
What should I do with the result of authorize_url? This url should be opened on a browser to proceed with the login, but the gem doesn't open it automatically. Should I ask the user to do so?
What about the callback URL? The example uses http://localhost:8080/oauth2/callback but is not clear if the gem itelf is in charge of listening on that port.
Any advices?
It might be easier to look at what the Quizlet API is asking you to do, and skipping the integration with the oauth gem. I could be wrong and I'd be interested to see how the the gem is used, but it's still a good idea to look at how to implement an oauth client step-by-step.
This is taken from the Quizlet API docs:
Send the user to https://quizlet.com/authorize. This can be done by displaying a link for the user to click, or redirecting server-side.
Once the user accepts/denies your oauth (this happens on the quizlet website, not your own), quizlet will send a request to your server. Because of the /authorize call, Quizlet will now have a redirect_uri which they will use to get in touch with your server. In your server's action for this route, you can get the authorization_code.
Send the authorization_code to https://api.quizlet.com/oauth/token, and in the response get an access_token whic you use for the rest of the api requests.
About the gem
You should send the authorize url to users. This is step 1 above
the gem is not in charge of listening for the redirect. You need to use a web server for this.
I'm building an API with Sinatra (using Angular for the client side and want others to have access to API) and have it also be an OAuth provider. I am wondering what the best route to take (work off existing gems or roll own solution off Warden or something).
Have used devise and doorkeeper for authentication and oauth before with Rails, wondering what best solution for Sinatra is.
Ideally I don't want the views or be able to extend/mod the actions of an existing solution, as I'm interacting with it purely as an API.
I just recently did the same thing using the following answer from S/O
What is a very simple authentication scheme for Sinatra/Rack
It implies a user model, but instead of using that, I just set a user and admin password in my config file. Then I had a login form that just took a password. When the user enters that password, I checked it against the one in settings and set the session['user'] to :admin or :user according to whichever it matched (or nil if none). Then on each of my routes, I called auth: :user or auth: :admin accordingly.
APIs normally accept your login request and send you an authentication token which you need to pass back in each call. This is very similar to cookie based sessions where your browser automatically passes back the cookie which is acquired on initial website visit.
From what I've seen in Sinatra's docs, you could make a session-based authentication system like this:
enable :session
disable :show_exceptions
use Rack::Session::Pool,
key: 'session_id'
post '/login' do
user = User.login_success(params)
halt 401 if user.nil?
session[:user] = user
200
end
get '/fun' do
user = session[:user]
halt 401 if user.nil?
halt 403 if !user.has_permission_for '/fun'
"fun was had"
end
Now all you need to do in your client is to pass back the cookie token returned in response to initial visit when requesting an API function. This can be done with any web client library that supports cookie stores (such as libcurl) or by inserting the session cookie into the request header manually. Rack::Minitest functionality also supports cookies, so you can test your API with minitest.
See Sinatra API Authentication.
Quick summary:
Sinatra has no built-in auth.
It's best to build auth yourself (see the link).
There are gems available, but you probably won't need them for something as simple as an API.
The issue here is with multiple instances of the same Sinatra (Rack) app deployed on Passenger+Apache on different sub URIs with HTTP basic auth to keep away unwanted access:
I have 4 instances of a Sinatra app deployed on my domain as:
example.com/private/foo
example.com/private/moo
...
...
Access to all of them is protected by HTTP basic authentication using the Rack::Auth::Basic middleware. config.ru for all of them look like:
# ...
users = {'user' => 'password'}
use Rack::Auth::Basic, 'realm' do |username, password|
users.key?(username) && users[username] == password
end
run MyApp
The only thing the changes from one config.ru to another is the 'realm' parameter.
Now the issue is that once I have logged into one of the apps, say private/foo, Chrome doesn't prompt me for a username and password for other apps (private/moo etc.). This is counterintuitive since all instances are uniquiely identified by their URLs. Using different credentials for each instance does work, but shouldn't Chrome request credentials at least once for each instance? One thing I noticed is that the first time I log into one of the instances, Chrome says 'The server at example.com:80 requires a username and password'. I would have expected 'The resource example.com/private/foo requires a username and password'. Isn't that how it is supposed to work?
I checked Rack::Auth::Basic source code and Wikipedia's article on HTTP Basic Auth and came up with nothing to help my case :(.
In basic authentication, the realm parameter isn't send back to the server. So the server can't really check if the client is sending authorization header for the same realm or not. It depends on the client. Rack's implementation of HTTP basic authentication is correct. So:
Now the issue is that once I have logged into one of the apps, say private/foo, Chrome doesn't prompt me for a username and password for other apps (private/moo etc.). This is counterintuitive since all instances are uniquiely identified by their URLs.
As Andrew pointed out and is clear from the RFC, URL doesn't play a role there. But if '/foo' is protected, '/foo/moo' is protected under the same realm.
Using different credentials for each instance does work, but shouldn't Chrome request credentials at least once for each instance?
Under the scenes what is happening (on inspecting with debugger tools) is that, after I have logged once into one of the apps, say private/foo, Chrome re-sends the same authorization header to other apps, say private/moo, without being challenged first.
The RFC says that the client may send the corresponding authorization header for a realm without being challenged by the server first.
Looks like Chrome is either treating all my apps to be under the same realm or re-sending the same authorization header across different realms. I don't think that is the expected behavior but I could be missing something. Firefox behaves same. Anyway, that wasn't the essence of the question.
The theme of the question was "How do I get Chrome to request me username and password at least once for each instance? Basic auth isn't working the way I expected it to; why?"
Use Digest authentication (RFC 2617 again). Rack implements the MD5 algorithm version under Rack::Auth::Digest::MD5. Set different opaque for each instance and you are good to go:
# ...
realm = "Description of the protected area."
opaque = "Secret key that uniquely identifies a realm."
users = {'user' => 'password'}
use Rack::Auth::Digest::MD5, realm, opaque do |username|
users[username]
end
opaque is sent back by the client and can be verified on the server side that the authorization request is for the correct resource. Job of realm seems descriptory in nature -- which area or resource are you trying to protect? what id do I flash?
RFC: https://www.rfc-editor.org/rfc/rfc2617
I'm trying to create a ruby-based twitter client where I can post my status from command-line using ruby. I'm trying to understand the oauth right now, and it confused me a little bit. If I'm building a web application, I can provide a callback url when the request token is complete. How would I do that from the command-line? I don't want ruby to print out the authorized and copy and paste the url and click 'Allow' to get the token. I found something about out-of-band exchange or PIN. How would I do that with OAuth library in ruby, please thank you very much.
To use the callback url mechanism, you application should be a web application. It seems you are developing desktop application and if that's the case, you should follow "PIN code" flow by supplying an oauth_callback_url of "oob" (out-of-band) when you request token. Like this,
https://api.twitter.com/oauth/request_token?oauth_callback=oob
If you properly set a header of this HTTP request (setting HTTP header is the key part of OAuth and I think you already know how to do this), Twitter will give oauth_token, oauth_token_secret and oauth_verifier. Let's call this token "request_token". You need it to get "access_token" later.
Once you have request_token, you need to open web page with the below url
http://api.twitter.com/oauth/authorize?oauth_token=request_token
This will open the authorization page and let a user to decide whether the user wants to allow your application to access his or her Twitter account. If the use says okay, then Twitter gives PIN code. You need to allow a user to type the PIN code so that you can save it.
Now, it's time to get another token ("access_token") by using your comsumer_key / secret, request_token and the PIN code. You should set header with all these values correctly and do HTTP request again with this url,
https://api.twitter.com/oauth/access_token
If Twitter accepts your "access_token" request, it will give you oauth_token, oauth_token_secret, user_id and screen_name. Let's call this token "access_token". Now, you can perform any OAuth required Twitter API by using access_token and its secret (oauth_token_secret). You can save the two values in a file and keep using them whenever you need to access the user's Twitter account. The values will be always valid until the user revokes the access to your application.
I don't know Ruby but if you know how to perform HTTP/HTTPS requests (GET / POST) with custom headers in Ruby, this PIN code flow should work fine if you follow Twitter API document carefully. Good Luck!
It will ask for the PIN code until you specify the oauth_callback when getting the request token, not when forwarding the user to the authorization url
#consumer = OAuth::Consumer.new(
TWITTER_CONSUMER_KEY,
TWITTER_CONSUMER_SECRET,
{:site=>"https://api.twitter.com"})
#request_token = #consumer.get_request_token( :oauth_callback => CALLBACK_URL )
This was the result of surfing several hours of incomplete documentation.
use Twitter gem, it will make things easier for you. http://rdoc.info/gems/tweeter/2.0.0/frames