Google Admin API with service account -- bad credentials - google-api

I'm trying to write console app code to update Google Directory with values pulled from a SQL database. I can't seem to make the API connect successfully when using a service account. Any ideas? I've whittled the code down to just the essentials here.
static void Main(string[] args)
{
try
{
// Create a ServiceAccountCredential credential
var xCred = new ServiceAccountCredential(new ServiceAccountCredential.Initializer("saxxxxxxxx#directorysync-xxxxxx.iam.gserviceaccount.com")
{
Scopes = new[] {
DirectoryService.Scope.AdminDirectoryUser,
DirectoryService.Scope.AdminDirectoryUserReadonly
}
}.FromPrivateKey("-----BEGIN PRIVATE KEY-----\nMI...p9XnI4DZFO/QQJc=\n-----END PRIVATE KEY-----\n"));
// Create the service
DirectoryService service = new DirectoryService(
new BaseClientService.Initializer()
{
HttpClientInitializer = xCred,
}
);
var listReq = service.Users.List();
listReq.Domain = "mycompany.com";
listReq.MaxResults = 100;
listReq.OrderBy = UsersResource.ListRequest.OrderByEnum.Email;
Users results = listReq.Execute();
// process the users list here...
}
catch (Exception e)
{ Console.WriteLine(e.Message); }
}
The error happens at the .Execute() line:
Google.Apis.Requests.RequestError
Not Authorized to access this resource/api [403]
Errors [
Message[Not Authorized to access this resource/api] Location[ - ] Reason[forbidden] Domain[global]
]
I've tried code seen elsewhere (How to login to Google API with Service Account in C# - Invalid Credentials) to bring in the whole contents of the .JSON file that contains the credentials for the service account; that made no difference. I'm not the google domain admin, but the admin built the credential and promises that it does, indeed, have rights to the user resources. I'm utterly lost at what's not right.

Either:
You're missing the email address of an admin user to impersonate.
An Admin of the domain needs to assign user management privileges to the service account.
Background
There's two HTTP requests involved in making a Google API request with a service account:
Using the service account's credentials to request an access token from the Google OAuth 2.0 server. A HTTP POST is sent with a JWT signed with the private key of the service account. If successful, an access token is returned which is valid for one hour.
Making the service API request (Admin SDK Directory API in this case) using the OAuth access token obtained from the last step.
The error message you provided is not listed in the JWT error codes page so step 1 is working, and the error is coming from step 2 - the request to Directory API.
You should be able to confirm this using an HTTPS request interceptor like mitmproxy.
You'd get a 403 error for the Directory API users.list method for a few reasons:
You've authenticated as a service account, but that service account has no admin privileges which are sufficient for the request.
You've authenticated as a user in the domain (either using a service account with impersonation with the sub parameter in the JWT request, or using three-legged interactive OAuth), but that user has no admin privileges which are sufficient for the request.
You've authenticated as a service account or user with sufficient privileges, but the domain parameter is not owned by the customer for which the service account or user is associated with.
In your sample code, there is no email address of a domain user (admin) specified (the email address for ServiceAccountCredential.Initializer is actually the OAuth client name of the service account, not a real email address), so it is either case 1 or 3.
To address case 1
A service account has no association with a domain or access privileges by default.
An admin can open Admin console, go to "Account > Admin roles > "User Management Admin" > Admins > Assign service accounts", then:
Add the service account name (looks like an email address, ending #gserviceaccount.com).
Click "Assign role".
Now, the service account does not need to impersonate another admin in the domain, it will directly have admin privileges.
A couple of side-effects of this:
The service account cannot be "suspended" as such, only removed from the admin role.
Audit logging of the actions will show the service account name in the "Reporting > Audit > Admin" section of Admin console.
To address case 2
You may have meant to impersonate an admin user in the domain, which you could do by adding:
, User = "admin#example.com"
- after the Scopes array passed to ServiceAccountCredential.Initializer.
This adds the email address of a user to the JWT request to retrieve an access token for that user (by adding the sub field to the claim-set of the JWT assertion).
To address case 3
Replace the "mycompany.com" on the line:
listReq.Domain = "mycompany.com";
- with a domain that is associated with the customer, or instead, remove that line and add:
listReq.Customer = "my_customer";
(literally my_customer - see users.list query-parameters)
- Which will list users on all domains associated with the customer (Google Workspace and Cloud Identity customers can have many secondary domains).

Related

Google Admin SDK with service account without impersonation

The goal is to establish a Service Account in the G-Suite platform that an application can use to perform actions without prompting users to authenticate.
We are experiencing issues very similar to the following postings, but slightly differently.
Google Admin SDK authentication with service account
How to authorise a service account to access the Google Admin API
As long as we follow the 'impersonation" flow, things work. With impersonation, the authorization rules follow the user being impersonated. Ideally, we want to SCOPE the service account accordingly and not impersonate a user. When we do this, we constantly receive a 403.
We've tried our best to follow the instructions here:
https://developers.google.com/admin-sdk/directory/v1/guides/delegation
https://developers.google.com/identity/protocols/OAuth2ServiceAccount?hl=en_US#delegatingauthority
https://medium.com/#Skaaptjop/access-gsuite-apis-on-your-domain-using-a-service-account-e2a8dbda287c
Our Java snippet is similar to this:
final NetHttpTransport HTTP_TRANSPORT = GoogleNetHttpTransport.newTrustedTransport();
File file = new File(MdmResource.class.getClassLoader().getResource("myproject.p12").getFile());
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId("105601508644514129999")
.setServiceAccountPrivateKeyFromP12File(file)
.setServiceAccountScopes(
Collections.singletonList(DirectoryScopes.ADMIN_DIRECTORY_USER_READONLY))
.setServiceAccountUser("user#domain.com") // 403 when this is commented out
.build();
Directory dir = new Directory(HTTP_TRANSPORT, JSON_FACTORY, credential);
Get result = dir.users().get("dude#domain.com");
User user = result.execute();
System.out.println(user.get("customerId"));
In GCP, we created the ServiceAccount while logged in as a superadmin. We enabled Domain-Wide delegation and generated the keys (in this example, the p12 type and used the corresponding file as input into the Java app).
Then, in the API Library, we enabled Admin SDK.
In the Google Admin, under Security/Advanced settings, we setup the scopes. We used 105601508644514129999 and https://www.googleapis.com/auth/admin.directory.device.mobile,https://www.googleapis.com/auth/admin.directory.user.readonly,https://www.googleapis.com/auth/admin.directory.user
When we run the Java code, we see the following when "setServiceAccountUser" is commented out (works fine when impersonating, as long as the user has the right permissions):
{
"code" : 403,
"errors" : [ {
"domain" : "global",
"message" : "Not Authorized to access this resource/api",
"reason" : "forbidden"
} ],
"message" : "Not Authorized to access this resource/api"
}
So, it seems like our SA is not correctly connected to scopes, but I'm out of ideas how to do this.
BTW, the 105601508644514129999 is the SA Unique ID and Client ID of the OAuth 2.0 credentials that were automatically generated during the SA creation process. We also used the SA Email for the "setServiceAccountId" as shown in the following example, but still getting 403. https://developers.google.com/admin-sdk/directory/v1/guides/delegation. Actually, I think this example has another issue with .setServiceAccountScopes too.
Finally...
<dependency>
<groupId>com.google.auth</groupId>
<artifactId>google-auth-library-oauth2-http</artifactId>
<version>0.20.0</version>
</dependency>
<!-- Used to navigate Google Directory services, like https://developers.google.com/admin-sdk/directory -->
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-admin-directory</artifactId>
<version>directory_v1-rev117-1.25.0</version>
</dependency>
Any thoughts?
Service accounts are not in the G Suite domain (even if they are owned by a user / project in it) and thus cannot be granted admin access to the G Suite domain. All they can do is impersonate users in the domain with Domain-Wide Delegation.
What you likely want to settle on is granting regular OAuth as an admin user using a client ID and secret and being sure to get a long-lived refresh token access_type=offline. This will allow you to refresh the access token whenever it expires and continue acting as the single admin user as long as they don't revoke your access.

How to debug Authentication configuration?

We are experiencing problems with Authentication of Service Accounts for domain-wide delegation.
The main problem is it's hard to investigate and debug the auth configuration so
we would like to ask for some tips how to debug the configuration.
Or maybe we are missing some configuration options and you can point us to them.
Our process is:
Create SA(=Service Account) with enabled domain-wide delegation.
Authenticate SA in GSuite admin console(https://support.google.com/a/answer/162106?hl=en).
use client_id from the credentials file. (now email)
scopes are comma-separated without spaces between.
Ensure the "Security > API Reference > API Reference -> 'Enable API Access'" is checked.
For some GSuite domains this is working configuration, but we got some domains where this configuration results in:
google.auth.exceptions.RefreshError: ('unauthorized_client: Client is unauthorized to retrieve access tokens using this method.', '{\n "error": "unauthorized_client",\n "error_description": "Client is unauthorized to retrieve access tokens using this method."\n}')
In our understanding, this is the error saying the client_id and scopes were not added to the "Manage API client access" page. (=List of authenticated clients)
We really ensured that the GSuite domain we are requesting has the proper client_id and scopes added in the list of authenticated clients + has the 'Enabled API Access'.
We even created Shared Desktop with them and did it by ourselves to be fully sure of it.
But the error still persists.
However, we are not able to replicate this problem on our test GSuite domain.
We tried couple of options using same SA as the client:
The impersonated account hasn't permissions to access the resource.
This result in:
googleapiclient.errors.HttpError: https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&alt=json returned "Not Authorized to access this resource/api">
The scopes are just partial:
google.auth.exceptions.RefreshError: ('access_denied: Requested client not authorized.', '{\n "error": "access_denied",\n "error_description": "Requested client not authorized."\n}')
The 'Enabled API Access' is not checked.
googleapiclient.errors.HttpError: https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&alt=json returned "Domain cannot use apis.">
The error we are receiving from the client("Client is unauthorized to retrieve access tokens using this method."), we are able to replicate only if the client_id is not in the list of authenticated clients at all.
But we are sure, the problematic GSuite domains have the SA authenticated in "Manage API client access" page.
We are using these scopes: https://www.googleapis.com/auth/userinfo.profile,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/plus.login,https://www.googleapis.com/auth/calendar.readonly,https://www.googleapis.com/auth/contacts.readonly, https://www.googleapis.com/auth/admin.directory.user.readonly
Do you have any ideas how to debug/solve this issue?
Here is what you need to do. Double check each step. If in doubt, start over.
Enable "Admin SDK API. This is enabled on a per project basis.
Create a service account. Do not add or remove any privileges. Don't change the service account in any way. If you do you will get an error that you are not authorized.
Enable Domain-wide Delegation on the service account.
Follow this document to delegate domain-wide authority to your service account:
Delegate domain-wide authority to your service account
When creating the service account credentials (from the downloaded Json) you will need the following scopes for full G Suite management:
"https://www.googleapis.com/auth/admin.directory.group",
"https://www.googleapis.com/auth/admin.directory.user"
Impersonate a user account which creates new credentials. The user account needs to be a G Suite superadmin. This account must have logged into G Suite at least once and accepted the Terms of Service.
Create your client using the credentials from step #5.
Working Python Example:
from googleapiclient.discovery import build
from google.oauth2 import service_account
# This is the service account credentials file
credentials_file = 'google-directory-api.json'
# In this example I only need to send email
credentials = service_account.Credentials.from_service_account_file(
credentials_file,
scopes=['https://www.googleapis.com/auth/gmail.send'])
# This user is a G Suite superadmin
impersonate = 'username#example.com'
credentials = credentials.with_subject(impersonate)
service = build('gmail', 'v1', credentials=credentials)
I think we are going to need to take this in stages. Lets start with your first error and see if that fixes your issue.
Background info
There are several types of clients that you can create in Google developer console. Here are the top three.
Browser client: Used for web applications
Native client (other): used for installed desktop applications
Service account: used for server to server communication.
The clients are different the client.json file you download is different and the code used to authenticated to the authentication server is also different.
Error 1: code / client missmatch
unauthorized_client: Client is unauthorized to retrieve access tokens using this method.
Can mean one of two things. Either you have created a service account client and you are not using service account code to authenticate or you are are using the code to authenticate with a service account but you have not created a service account client. You haven't posted what language you are using or any code so I cant tell you if the code you are using is intended to be used with a service account or not. Your going to have to look into this one a bit.
Check in developer console make sure your client is like this If it is check your code. If it isnt then create a real service account client and try again.

Google Drive API - Transfer ownership as delegated admin

I am trying to transfer ownership from a normal user to a delegated admin user which has both the Drive service privilege and the Data Transfer privilege, which is required according to the documentation.
I'm trying to do this on behalf of the delegated admin user.
The reason for this is, that I want to make sure that every file that gets created by any user in our domain, will transfer its ownership to one admin user.
So I have an (offline) access token and refresh token stored in my backend. I use the Push Notification service from the Google Drive API to respond to new files created by users in my domain using this (stripped down) code:
$client = new \Google_Client();
$client->setAuthConfigFile('client_config.json');
// This is the access token json for the delegated admin user.
$client->setAccessToken('{access_token_json_string}');
$drive = new \Google_Service_Drive($client);
// This is the file created by the user. It has two permissions: The user is the 'owner' and the admin user is the 'writer'
$fileId = 'XXX_XXxXXXXxxXXXxxX';
// this is the admin user's permission id, of role 'writer'. This is the one I am trying to change to 'owner'.
$permissionId = '1234567890';
// I've tried setting all values, but that doesn't help
$permission = new \Google_Service_Drive_Permission();
$permission->setId($permissionId);
$permission->setEmailAddress('admin#domain.com');
$permission->setValue('admin#domain.com');
$permission->setType('user');
$permission->setRole('owner');
// Both 'put' and 'update' raise the same error
$drive->permissions->update($fileId, $permission->getId(), $permission, array(
'transferOwnership' => true
));
The response I am getting:
Error calling PUT https://www.googleapis.com/drive/v2/files/XXX_XXxXXXXxxXXXxxX/permissions/1234567890?transferOwnership=true: (403) Insufficient permissions for this file
Note that all other API calls work properly, so the access token is valid. There's also no option available within Google Drive itself to transfer the ownership (as being the admin user). Transferring it from the normal user to the admin user works as expected.

Re-validating users when using OpenId Connect implicit flow

I'm trying to understand how to make sure that a logged on user's account is still "valid" (where valid means for example not locked out, not deleted)
I've set up an identity provider using IdentityServer v3. On the "relying party"-side, I have an ASP.NET WebApi hosted using Owin. On the RP-side, I'm using UseOpenIdConnectAuthentication to install the OpenIdConnectAuthenticationMiddleware in the Owin pipeline.
What's working so far:
Any unauthenticated user visiting my web app is redirected to the login page on IdentityServer
The user logs on (I'm using the implicit flow)
The user is redirected back to my web app
My web app receives the JWT containing the id token and access token
My web app calls the user info endpoint to retrieve the claims using the access token
My web app creates a new ClaimsIdentity containing the claims my app is interested of. This is then persisted in a cookie, using:
app.UseCookieAuthentication(new CookieAuthenticationOptions
{
AuthenticationType = "Cookies",
SlidingExpiration = true
});
This works fine, but I want some kind of hourly validation that the user is still "valid" from the identity servers perspective.
Is there some standard pattern how I should re-validate that a user account is valid? I don't want to force the user to log on again, I just want to confirm that a user can't stay on forever even if his user account is deleted.
You can send the user to the Authorization Server again with an OpenID Connect authentication request but with the additional parameter prompt=none as documented in the spec: http://openid.net/specs/openid-connect-core-1_0.html#AuthorizationEndpoint If that returns successfully, the user is still logged in, else an error will be returned. The user will not be prompted in either case.

Accessing Google Contacts Api via OAuth 2.0 and private key aka Service Account

I am currently implementing access to Google Contacts via OAuth 2.0 and a so called Service Account. The service account is generated for an ordinary user like "My.Name#gmail.com".
The code to generate the OAuth 2.0 credentials is:
public static GoogleCredential getCredentials() throws GeneralSecurityException, IOException {
GoogleCredential credential = new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(SingleUserCredentials.SERVICE_ACCOUNT_EMAIL)
.setServiceAccountScopes("https://www.google.com/m8/feeds")
.setServiceAccountPrivateKeyFromP12File(new File(SingleUserCredentials.SERVICE_ACCOUNT_PKCS12_FILE_PATH))
.build();
credential.refreshToken();
return credential;
}
I am then trying to retrieve the contacts via:
ContactsService myService = new ContactsService(
"myApp");
myService.setOAuth2Credentials(getCredentials());
URL feedUrl = new URL("https://www.google.com/m8/feeds/contacts/default/full");
Query myQuery = new Query(feedUrl);
ContactFeed resultFeed = myService.query(myQuery, ContactFeed.class);
// Print the results
for (ContactEntry entry : resultFeed.getEntries()) {
System.out.println(entry.getName().getFullName().getValue());
System.out.println("Updated on: " + entry.getUpdated().toStringRfc822());
}
The problem is that I do not get any a single contact from my account. The feed is always empty. There is no error. Nothing.
When accessing a Google Apps managed domain via the same approach it works nicely.
I am wondering if the Contacts Api supports OAuth 2.0 for ordinary (aka #gmail.com) accounts when using a p12 key file and a service account.
I ran into that same problem myself.
I tried both the email address that I received when I setup the key and the email address of a domain administrator.
When I use the email from the key setup, I don't receive anything at all -- no warnings, no exceptions, and no data.
When I use the email address of a domain administrator, I receive an exception:
com.google.api.client.auth.oauth2.TokenResponseException: 400 OK
[java] {
[java] "error" : "invalid_grant"
[java] }
[java] Feb 5, 2013 5:16:48 PM com.google.appengine.repackaged.org.apache.http.impl.client.DefaultRequestDirector handleResponse
[java] WARNING: Authentication error: Unable to respond to any of these challenges: {}
[java] Feb 5, 2013 5:16:48 PM com.google.apphosting.utils.jetty.JettyLogger warn
[java] WARNING: /
[java] java.lang.NullPointerException: No authentication header information
...
So, I figured that the domain administrator's email address wasn't what I needed.
Next, I Googled around for a while before finding this page:
http://javadoc.google-api-java-client.googlecode.com/hg/1.13.2-beta/com/google/api/client/googleapis/auth/oauth2/GoogleCredential.html
I saw in there getServiceAccountUser (). The description of the field was:
Returns the email address of the user the application is trying to impersonate in the service account flow or null for none or if not using the service account flow.
Sure enough, there's a corresponding setServiceAccountUser (String) which accepts the username (email address) of the user you're using the service account to impersonate.
I set that field to an appropriate value and I was able to proceed.
In retrospect, it all makes sense -- if I don't supply an account that I'm trying to work from, I can't pull down the contacts for that account.
It is currently not possible to access Contacts using a service account as it is not supported in the Google APIs Console at Google APIs Console.
See also: Service Accounts
Second, it would only work with a Google managed domain because the Admin of the domain must grant access to the service account via the process below:
Delegate domain-wide authority to your service account
The service account that you created now needs to be granted access to the Google Apps domain’s user data that you want to access. The following tasks have to be performed by an administrator of the Google Apps domain:
Go to your Google Apps domain’s control panel. The URL should look like: "www.google.com/a/cpanel/mydomain.com"
Go to Advanced tools... > Manage third party OAuth Client access.
In the Client name field enter the service account's Client ID.
In the One or More API Scopes field enter the list of scopes that your application should be granted access to.
Click the Authorize button.
I've run across this very same error today but have given up on using Service Accounts for now, which I assume is not currently supported in the Contacts API. And so I am using Contacts API v3 with OAuth 1.0 and am getting the expected results.
ContactsService contactsService = new ContactsService(APPLICATION_NAME);
contactsService.setUserCredentials(CLIENT_USERNAME, CLIENT_SECRET);
URL contactFeedURL = new URL("https://www.google.com/m8/feeds/contacts/default/full");
Query contactFeedQuery = new Query(contactFeedURL);
ContactFeed contactFeed = contactsService.getFeed(contactFeedQuery, ContactFeed.class);

Resources