Sending automated emails using Gmail API with Java and Oauth authentication - google-api

I have a web app which sends emails (gmail) in name of my users
When a user registers, she supplies gmail account and password. Also she has to enable access for Less Secure Apps (I recommend to create a new account for this)
Then I can open a gmail session
session = Session.getInstance(props, new javax.mail.Authenticator() {
protected PasswordAuthentication getPasswordAuthentication() {
return new PasswordAuthentication(user.getEmail(), user.getPassword());
}
});
and send emails on her behalf.
Unfortunately this is going to stop working next 30th May, when Google will allow only OAUTH2 access
I have followed Java Quickstart for Gmail API and I have code up and running for sending emails with OAUTH2: enable gmail api, create an application on google cloud platform, grant send permission, oauth2 client id credential created...
The problem I have is I can't see a way to automatize this task because when creating an authorized credential, a consent screen displays on browser and you have to select the account to be granted manually (maybe because my app in google cloud platform is still pending to be reviewed)
Is there a way to infer the gmail account you want to access from the credentials file (client_secret.json)? Is there a way to automatize this?

No, or yes. It depends.
The whole point of OAuth2 is to improve security by working with authorization tokens rather than asking for user credentials. To do this the user has to consent to the app's access request, and thus the OAuth consent screen cannot be bypassed. This is
explained in Google's documentation. It's not related to your app's review status but rather it's the way OAuth works.
You can still work in a similar way, though . Instead of asking for username and password upon the user's registration you can redirect them to the OAuth consent screen so they can authorize your app. Make sure that your app is requesting offline access type and then you can retrieve an access_token and a refresh_token. These will essentially work as your credentials and you can use the refresh token to generate new access tokens when needed without having the user go through the consent screen each time.
The refresh token doesn't have a "natural" expiration so you can keep using it indefinitely, but there are a few scenarios where it will become invalid, such as it not being used for six months, the user changing passwords (if using Gmail scopes), the user manually revoking access, etc. In these cases you will need to direct them to the consent screen again to reauthorize your app.
In this sense, your app can still work automatically without user input except the initial setup, which you already had to deal with when they supplied you with their credentials. The refresh token expiration can even be compared to what you had to do when the users changed their passwords in your current workflow.
One exception to this are service accounts. If you and your users are part of a Google Workspace domain you can delegate domain-wide access to it, then the service account will be able to access user data without any manual input. Of course, this is because as the domain administrator you pretty much own all the accounts under it. But if you're working with a publicly available application you will have to deal with the limitations I mentioned above.
Sources:
Google's auth overview
Using OAuth 2.0 to access Google APIs
OAuth 2.0 for web applications
The OAuth consent screen

Related

How to programmatically request domain-wide delegation for Google Calendar API?

I've spent a few hours now looking through the docs + reading other SO posts, and it's still not clear to me how to simply request domain-wide credentials for Google Calendar API (for other domains; not my own).
Our current web app kicks off an OAuth flow to request Calendar API credentials for a single user; however, there doesn't seem to be a simple way like this to request credentials for an entire domain. It seems that acquiring domain-level access requires the admin to manually set up a service account and then pass that information over (https://developers.google.com/admin-sdk/directory/v1/guides/delegation) which is incredibly cumbersome.
With Microsoft's Calendar API, this is a very straightforward process where you simply specify "Application Permissions" and then the OAuth flow must be completed by an Admin to get the expected set of access and refresh tokens. For GSuite, I can find no such equivalent...
To clarify based on some comments, I'm trying to figure out if GSuite has anything equivalent to what Microsoft Graph offers as described here:
https://learn.microsoft.com/en-us/graph/auth-v2-service
The closest I've seen requires publishing to the Google Apps Marketplace; however, the documentation here seems to be quite lacking and it's unclear how things like redirect_uri for handing over credentials to the backend server are handled.
If I've missed any documentation and someone can just point me in the right direction then that would be greatly appreciated.
The best documentation for how to set up domain wide delegation to a service account is -> Perform Google Workspace Domain-Wide Delegation of Authority
request domain-wide credentials for Google Calendar API (for other domains; not my own).
You can not. The admin of the google workspace domain sets up domain wide delegation for service accounts owned by the domain.
there doesn't seem to be a simple way like this to request credentials for an entire domain.
Service account authorization is very different from Oauth2 authorization. Service account authorization is is intended for backend systems that need access to data without requesting user permission.
Oauth2 allows you to grant authorization on a per user basis. The user must grant an application access to their data by accepting the consent screen.
There is no oauth2 flow that would grant your application to all the users on a workspace domain. TBH I think thats a good thing.
Only super administrators can configure domain-wide delegation, this is done in the Google Workspace Admin console
When you enable domain-wide delegation for a service account, basically you are giving it the permission to impersonate any user within your domain
If you need to make Calendar requests on behalf your users, in your app you will need to implement impersonation, that way you will use the service account with wide-domain delegation to make the requests on behalf your users
Here you can find the documentation explaining how to set up wide-domain delegation and an example for making the API calls implementing impersonation https://developers.google.com/identity/protocols/oauth2/service-account#authorizingrequests

Userless Automated server to server Oauth2 2 legged authentication to Gmail

I've found plenty of information on implementing Oauth2 using a user authorization step, but I'm trying to run a container that automatically scrapes a gmail inbox for attachments transforms them, and exports to prometheus, and I'm having trouble figuring out how to implement this library: https://pkg.go.dev/golang.org/x/oauth2/clientcredentials#Config or any other for that matter to retrieve a token without involving a manual user step.
Will doing this in Go require writing direct API calls since I can't find an existing library to handle this scenario? Would it make more sense to create a Google App password and use generic user/pass SMTP authentication?
First off i understand what you are trying to do.
You have a backend system running in a container which will access a single gmail account and process the emails.
Now you need to understand the limitations of the API you are working with.
There are two types of authorization used to access private user data
service account - server to server interaction only works with workspace domains. No authorization popup required.
Oauth2 - authorize normal user gmail accounts, requires user interaction to authorize the consent screen
If you do not have a workspace account and this is a normal gmail user then you have no choice you must use Oauth2, which will require that a user authorize the application at least once.
Using Oauth2 you can request offline access and receive a refresh token which you can use to request new access tokens when ever you wish. The catch is that your application will need to be in production and verified, because your refresh token will only work for seven days and then it will expire. To fix this and get a refresh token that does not expire means that your application must in production and verified. This means you need to go though Googles verification process with a restricted gmail scope which requires third party security check and costs between 15k - 75k depending upon your application.
I understand that this is a single user system but that does not mean that you still need to go though verification. When google added the need for application verification they did not take into account single user systems like yours.
Option
Have you considered going directly though the SMPT server instead of using the Gmail api? If you use an apps password you should bypass everything by loging in using the login and the apps password.

In the bot framework OAuth Flow with Teams, what is the correct way to consent to changed permissions?

I've implemented the OAuth flow in a Teams bot similar to the sample, and I'm able to get a user token and make calls against the graph, but I'm running into an issue with changing permissions. Here's what I did:
Created a bot and gave it mail.read
Add the bot for a user and trigger the OAuth dialog. User is prompted for consent, and signed in.
Successfully read mail
In AAD, added the Mail.ReadWrite and Mail.Send permissions to the app
Next time the OAuthPrompt was triggered, token was returned immediately. No prompt for consent.
Got an access denied exception when trying to write or send mail
So the issue is that changing permissions on the app does not automatically trigger the consent flow again in the OAuthPrompt. I also tried these troubleshooting steps:
I manually signed the user out (botAdapter.SignOutUserAsync) and signed back in, but was not prompted for consent again.
Same as above, but signed out of Teams as well. Launched the web client in a new incognito window, and still no prompt for consent.
Uninstalled and re-installed the bot on the user's account. No change.
Tried the above after waiting a few hours, but still no change.
The only way I found to trigger the consent flow again was to have the user go to https://account.activedirectory.windowsazure.com/r#/applications and delete the consent from there. Even then I had to call botAdapter.SignOutUserAsync (since the bot service still returned a token with the old permissions). Once I did that, and triggered the OAuth prompt again, I was able to get the consent flow with the new permissions to trigger.
So my question is, is there a better way to handle this? If a new version of the bot requires new permissions, shouldn't the OAuthPrompt from Microsoft.Bot.Builder.Dialogs and the bot service handle re-prompting for consent?
Short answer
Bot Framework's auth provider for AAD v1 isn't flexible enough to support adding scopes.
You can switch to the AAD v2 provider and specify your new scopes in the OAuth Connection Settings. Then, if you force the user to sign in again (after SignOutUserAsync), you will get a consent screen including the new scopes. To set up the AAD v2 provider, see the Bot Framework docs on adding authentication to a bot, which has steps for both the AAD v1 and AAD v2 providers.
Long answer
The key to this behavior is how the AAD v1 and AAD v2 endpoints handle permissions and consent differently.
The problem with v1
In v1, the permissions (such as Mail.Send) are pre-registered in the AAD app registration. During sign-in, AAD checks if the user has already consented to any scopes for the app.
If yes, AAD skips the consent screen and eventually provides an access token for the scopes that had already been consented.
If no, AAD shows the consent screen for all registered scopes and eventually provides an access token for all of those scopes.
This explains the behavior you're seeing. After you tell Bot Framework to forget the current access token (via SignOutUserAsync) and force the user to login again, AAD sees that the user has consented previously, so it skips the consent screen and gives you a new token with the old scopes.
Then how do you add permissions for users who have already consented, without forcing the user to delete consent for the app? AAD's login endpoint has an optional prompt parameter that you can set to prompt=consent. This will force AAD to show the consent screen as if the user had not previously consented, and it will contain all registered permissions. So if you try to use an access token and get a 403 Forbidden error (or an equivalent exception), you can take the user through the login flow using prompt=consent.
Unfortunately, with Azure Bot Service's AAD v1 provider, you don't have enough control over the login URL to dynamically set the prompt parameter, so there isn't an easy way to achieve this.
Enter v2
But there's hope! The AAD v2 endpoints have a much more flexible way of adding scopes incrementally. In v2, for delegated permissions, the permissions do not have to be pre-registered in the AAD app registration. Instead, you specify the scopes in the scope parameter of the login URL. During sign-in, AAD checks if the user has consented to the scopes you specified in the URL.
If yes, AAD skips the consent screen and eventually provides an access token for the scopes you specified.
If no, AAD shows the consent screen for all the scopes you specified and eventually provides an access token for all of those scopes.
Either way, you end up with an access token containing all the scopes you specified in the URL.
Bot Framework sets the scope parameter using the scopes you specify in the OAuth Connection Settings. So if you add a new scope there, then the next time the user signs in, they will get a consent screen with the new permissions, and you will get an access token with the new scopes. (Note: To re-trigger the sign-in, you will still have to sign the user out using SignOutUserAsync. Otherwise, Bot Framework will continue giving you the access token it already has, instead of performing a new sign-in flow with AAD.)

send emails from MY gmail account with OAuth2 and nodemailer

I want to send emails from my gmail address through my own server. I'm using nodemailer and using account credentials is flaky, and often times doesn't work and leads to this thread
I've implemented everything on that thread many times, and still it's flaky, and also I know OAuth2 is the way to go.
I have a project with cliendID and clientSecret in google developer console, as you can see:
But how do I get an access token WITHOUT any browser interaction?
I seem to be missing something trivial here ...
I've went through all google tutorials and docs I could find about OAuth2, tokens, and APIs, but all guides go through the browser in one point.
Go to the OAuth Playground, click the cog on the top right, check the Use your own OAuth credentials and insert your clientID and clientSecret.
Then select the Gmail API v1 scopes you want in the list to the left and follow the outlined steps and you will get an access_token and a refresh_token.
Google Oauth2 actually all Oauth2 implementations I am aware of require that a user grant an application access via a web browser.
There is an alternative type called service accounts this is more like oauth1 service accounts are preauthorized. You can grant a service account access to your google drive by sharing folders and files with the service account like you would any other user. Because they are preauthorized there is no browser window pop up with service accounts.
You can only user service accounts with Gmail if you have a Google domains account Gsuite. The admin can go in and grant the service account access to the Gmail account in question. Perform G Suite Domain-Wide Delegation of Authority
If this is a normal user Gmail account you cant use a service account. You will have to use Oauth2 popup the request and save the refresh token so that you can gain access at a later date.

Access email from Gmail from server

I am trying to be able to set up a cron job to read contents from a certain email in my gmail inbox daily. I lookeed up gmail api documentation and noticed that the only way to authenticate my requests to access email data is via OAuth 2.0 which requires user authorization. Is there a way to authorize my app to access emails from a particular email id without the need for the user to manually take any actions.
I found this: https://developers.google.com/identity/sign-in/web/server-side-flow. I was wondering if there is any way to follow this workflow without having to build the UI?
Technically speaking you can use Oauth2 you just have to have the user authentication your application once. You will get a refresh token then you can use the refresh token to get a new access token from cron. Unless this is a Google domain account you cant use service accounts. There is no way to pre authorize a service account to access a normal user gmail.
Alternative: have you considered going directly though the mail server? Skip the rest api. https://developers.google.com/gmail/oauth_overview Note: That page also speaks of XOauth2 I haven't tried it yet you can still access SMTP and IMAP using username and password.

Resources