Can I limit the available scopes on a Google API service account? - google-api

I have done the following:
Created a project in Google API console
Enabled the Google Drive API in the project
Created a service account
Shared a Google Drive folder with the service account
Connected successfully to Google Drive and retrieved the list of folders and files shared with the service account.
When you create an OAuth client ID, you can limit that to predefined scopes. As far as I can tell, the service account has access to any Google Drive scope. I wanted to tighten that down to the following scope: https://www.googleapis.com/auth/drive.readonly just as a reassurance that there's no way the Google Drive app I'm making unintentionally adds/edits/deletes any files.
I know I can add the account to different roles. However, I looked through the list multiple times and none of them are related to Google Drive. I attempted to make my own role, but the available permissions on that screen do not reference Google Drive either. It's possible I missed something or there's another place I could look. Any suggestions?

To limit the scope a Service Account, you have to specify the scope on the server-side.
Service accounts are special Google accounts that can be used by
applications to access Google APIs programmatically via OAuth 2.0. A
service account uses an OAuth 2.0 flow that does not require human
authorization. Instead, it uses a key file that only your application
can access.
For example:
In python, you can specify the scope of a service account by creating a list of scopes and use it as parameter when getting the credentials.
Folder and Files:
python:
Search all image with jpeg extension:
import httplib2
import os
from apiclient import discovery
from google.oauth2 import service_account
scopes = ["https://www.googleapis.com/auth/drive.readonly"]
secret_file = os.path.join(os.getcwd(), 'client_secret.json')
credentials = service_account.Credentials.from_service_account_file(secret_file, scopes=scopes)
service = discovery.build('drive', 'v3', credentials=credentials)
page_token = None
while True:
response = service.files().list(q="mimeType='image/jpeg'",
spaces='drive',
fields='nextPageToken, files(id, name)',
pageToken=page_token).execute()
for file in response.get('files', []):
# Process change
print('Found file: %s' % (file.get('name')))
page_token = response.get('nextPageToken', None)
if page_token is None:
break
Output:
Found file: cute-puppy.jpg
Creating folder with readonly scope:
import httplib2
import os
from apiclient import discovery
from google.oauth2 import service_account
scopes = ["https://www.googleapis.com/auth/drive.readonly"]
secret_file = os.path.join(os.getcwd(), 'client_secret.json')
credentials = service_account.Credentials.from_service_account_file(secret_file, scopes=scopes)
service = discovery.build('drive', 'v3', credentials=credentials)
file_metadata = {
'name': 'Invoices',
'mimeType': 'application/vnd.google-apps.folder'
}
file = service.files().create(body=file_metadata,
fields='id').execute()
Error message:
<HttpError 403 when requesting https://www.googleapis.com/drive/v3/files?fields=id&alt=json returned "Insufficient Permission: Request had insufficient authentication scopes.". Details: "Insufficient Permission: Request had insufficient authentication scopes.">
References:
Google Auth Python
OAuth Scopes

Related

errors when trying to tweet using tweepy

I am working on an academic research project and I am trying to send out tweets using the the Twitter API. The error I am receiving repeatedly is
Forbidden: 403 Forbidden
Your client app is not configured with the appropriate oauth1 app permissions for this endpoint.
import tweepy
#from tweepy import OAuthHandler
ACCESS_KEY = 'xxx'
ACCESS_SECRET = 'xxx'
CONSUMER_KEY = 'xxx'
CONSUMER_SECRET = 'xxx'
api = tweepy.Client(bearer_token='xxx',
access_token=ACCESS_KEY,
access_token_secret=ACCESS_SECRET,
consumer_key=CONSUMER_KEY,
consumer_secret=CONSUMER_SECRET)
api.create_tweet(text='I want to Tweet')
Here is my code. The authentication raises no errors. Just the attempt at tweeting.
You can fix the problem by activating Read / Write in the Oauth section of your application, and then you shall regenerate the "Access Token and Secret".
You can check that are properly recreated when you see:
Created with Read and Write permissions
EDIT as of 10/February/2023: You are now required to ask for Elevated access if you want to have read + write permission. You only have read access from the V2 API Endpoints as of today

How can I authenticate against ADXProxy using app key authentication?

I am trying to access an Azure Application Insights resource via Redash, using the (preview) ADXProxy feature.
I've created an App Registration in Azure, and I've got some proof-of-concept python code which can successfully access my Application Insights resource and execute a Kusto query (traces | take 1) using an application token:
import azure.kusto
import azure.kusto.data.request
import msal
cluster = 'https://ade.applicationinsights.io/subscriptions/<MY_SUBSCRIPTION>/resourcegroups/<MY_RESOURCE_GROUP>/providers/microsoft.insights/components/<MY_APP_INSIGHTS_RESOURCE>'
app_id = '<MY_APP_ID>'
app_key = '<MY_SECRET>'
authority_id = '<MY_AAD_SUBSCRIPTION_ID>'
def run():
app = msal.ConfidentialClientApplication(
client_id=app_id,
client_credential=app_key,
authority='https://login.microsoftonline.com/<MY_AAD_SUBSCRIPTION_ID>')
token = app.acquire_token_for_client(['https://help.kusto.windows.net/.default'])
kcsb = azure.kusto.data.request.KustoConnectionStringBuilder.with_aad_application_token_authentication(
connection_string=cluster,
application_token=token['access_token']
)
client = azure.kusto.data.request.KustoClient(kcsb)
result = client.execute('<MY_APP_INSIGHTS_RESOURCE>', 'traces | take 1')
for res in result.primary_results:
print(res)
return 1
if __name__ == "__main__":
run()
However, Redash doesn't support application token authentication: it uses application key authentication, making a call like:
kcsb = azure.kusto.data.request.KustoConnectionStringBuilder.with_aad_application_key_authentication(
connection_string = cluster,
aad_app_id = app_id,
app_key = app_key,
authority_id = '<MY_AAD_SUBSCRIPTION_ID>'
)
I can't successfully connect to my App Insights resource using this type of flow. If I substitute this KustoConnectionStringBuilder into my program above, I get an exception telling me:
The resource principal named https://ade.applicationinsights.io was not found in the tenant named <MY_AAD_SUBSCRIPTION_ID>. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.
Is there something I can do in code or Azure Portal configuration to connect my 'tenant' to the ade.applicationinsights.io resource principal and get this connection working?
Adxproxy supports only tokens minted by Azure Active Directory (AAD). The token must be created for an Azure Data Explorer cluster (ADX), that you own. If you don't have your own ADX cluster, and for whatever reason you want to access your Application Insights resources via Adxproxy, you can always authenticate to 'https://help.kusto.windows.net' and use that token.

How to debug Authentication configuration?

We are experiencing problems with Authentication of Service Accounts for domain-wide delegation.
The main problem is it's hard to investigate and debug the auth configuration so
we would like to ask for some tips how to debug the configuration.
Or maybe we are missing some configuration options and you can point us to them.
Our process is:
Create SA(=Service Account) with enabled domain-wide delegation.
Authenticate SA in GSuite admin console(https://support.google.com/a/answer/162106?hl=en).
use client_id from the credentials file. (now email)
scopes are comma-separated without spaces between.
Ensure the "Security > API Reference > API Reference -> 'Enable API Access'" is checked.
For some GSuite domains this is working configuration, but we got some domains where this configuration results in:
google.auth.exceptions.RefreshError: ('unauthorized_client: Client is unauthorized to retrieve access tokens using this method.', '{\n "error": "unauthorized_client",\n "error_description": "Client is unauthorized to retrieve access tokens using this method."\n}')
In our understanding, this is the error saying the client_id and scopes were not added to the "Manage API client access" page. (=List of authenticated clients)
We really ensured that the GSuite domain we are requesting has the proper client_id and scopes added in the list of authenticated clients + has the 'Enabled API Access'.
We even created Shared Desktop with them and did it by ourselves to be fully sure of it.
But the error still persists.
However, we are not able to replicate this problem on our test GSuite domain.
We tried couple of options using same SA as the client:
The impersonated account hasn't permissions to access the resource.
This result in:
googleapiclient.errors.HttpError: https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&alt=json returned "Not Authorized to access this resource/api">
The scopes are just partial:
google.auth.exceptions.RefreshError: ('access_denied: Requested client not authorized.', '{\n "error": "access_denied",\n "error_description": "Requested client not authorized."\n}')
The 'Enabled API Access' is not checked.
googleapiclient.errors.HttpError: https://www.googleapis.com/admin/directory/v1/users?customer=my_customer&alt=json returned "Domain cannot use apis.">
The error we are receiving from the client("Client is unauthorized to retrieve access tokens using this method."), we are able to replicate only if the client_id is not in the list of authenticated clients at all.
But we are sure, the problematic GSuite domains have the SA authenticated in "Manage API client access" page.
We are using these scopes: https://www.googleapis.com/auth/userinfo.profile,https://www.googleapis.com/auth/userinfo.email,https://www.googleapis.com/auth/gmail.readonly,https://www.googleapis.com/auth/plus.login,https://www.googleapis.com/auth/calendar.readonly,https://www.googleapis.com/auth/contacts.readonly, https://www.googleapis.com/auth/admin.directory.user.readonly
Do you have any ideas how to debug/solve this issue?
Here is what you need to do. Double check each step. If in doubt, start over.
Enable "Admin SDK API. This is enabled on a per project basis.
Create a service account. Do not add or remove any privileges. Don't change the service account in any way. If you do you will get an error that you are not authorized.
Enable Domain-wide Delegation on the service account.
Follow this document to delegate domain-wide authority to your service account:
Delegate domain-wide authority to your service account
When creating the service account credentials (from the downloaded Json) you will need the following scopes for full G Suite management:
"https://www.googleapis.com/auth/admin.directory.group",
"https://www.googleapis.com/auth/admin.directory.user"
Impersonate a user account which creates new credentials. The user account needs to be a G Suite superadmin. This account must have logged into G Suite at least once and accepted the Terms of Service.
Create your client using the credentials from step #5.
Working Python Example:
from googleapiclient.discovery import build
from google.oauth2 import service_account
# This is the service account credentials file
credentials_file = 'google-directory-api.json'
# In this example I only need to send email
credentials = service_account.Credentials.from_service_account_file(
credentials_file,
scopes=['https://www.googleapis.com/auth/gmail.send'])
# This user is a G Suite superadmin
impersonate = 'username#example.com'
credentials = credentials.with_subject(impersonate)
service = build('gmail', 'v1', credentials=credentials)
I think we are going to need to take this in stages. Lets start with your first error and see if that fixes your issue.
Background info
There are several types of clients that you can create in Google developer console. Here are the top three.
Browser client: Used for web applications
Native client (other): used for installed desktop applications
Service account: used for server to server communication.
The clients are different the client.json file you download is different and the code used to authenticated to the authentication server is also different.
Error 1: code / client missmatch
unauthorized_client: Client is unauthorized to retrieve access tokens using this method.
Can mean one of two things. Either you have created a service account client and you are not using service account code to authenticate or you are are using the code to authenticate with a service account but you have not created a service account client. You haven't posted what language you are using or any code so I cant tell you if the code you are using is intended to be used with a service account or not. Your going to have to look into this one a bit.
Check in developer console make sure your client is like this If it is check your code. If it isnt then create a real service account client and try again.

Updating Google Drive Metadata with googleapiclient in python

I am trying to update a Test file's metadata (specifically the file's name) in Google Drive using python's Google API client library. I have used the following google resources:
https://developers.google.com/drive/api/v3/reference/files/update
I haven't had had trouble connecting to googles drive API. I am merely having issues updating the filename or any other metadata that I would want to update.
My code is below:
from googleapiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools
from googleapiclient import errors
from googleapiclient.http import MediaFileUpload
# This is to connect to Google Drive's API
SCOPES = 'https://www.googleapis.com/auth/drive.metadata' # drive.metadata gives me the access to read and writeover metadata
CLIENT_SECRET = 'client_id.json' # this is from Oauth when you create credentials for a good project
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET, SCOPES)
creds = tools.run_flow(flow, store)
DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http()))
files = DRIVE.files().list().execute().get('files', [])
print()
print("File Specific metadata")
for f in files:
if f['name']=='Test' and f['mimeType']=='application/vnd.google-apps.document':
print(f['name'], f['id'], f['mimeType'])
file_id = f['id']
print(file_id)
# First retrieve the file from the API.
print()
file = DRIVE.files().get(fileId=file_id).execute()
print(file)
file['name'] = 'A'
print(file)
# Send the request to the API.
updated_file = DRIVE.files().update(fileId=file_id, body=file).execute()
But when I do this, I get an insufficient permissions error. I think it has something to do with the SCOPES but I am not sure.

Creating email drafts using Gmail API on Ruby (google-api-ruby-client 0.9)

I am currently trying out Ruby and the Google API for Ruby and I am having difficulties accessing my Gmail account and creating drafts with it (via create_user_draft) using a Service Account. I have successfully authenticated my Service Account with the API (Access Tokens are being generated).
I can use it with the Google::Apis::DriveV2::DriveService::list_files but not on GmailV1 methods.
I use this code to authorise the service account and the scope https://www.googleapis.com/auth/gmail.compose
Authorisation
def authorise
#jsonKeyIo = self.loadCredentialsFile
gAuthDefaultCreds = ##gAuthDefaultCreds
serviceAccountCredentials = gAuthDefaultCreds.make_creds(
{json_key_io: #jsonKeyIo, scope: #scope})
#service.authorization = serviceAccountCredentials
#service.authorization.fetch_access_token!
end
It generates an access token with this format:
{"access_token"=>"ya29.access_token_codes_here", "token_type"=>"Bearer", "expires_in"=>3600}
Draft creator snippet
##gmail = Google::Apis::GmailV1
##service = ##gmail::GmailService.new
def createDraft(draftTitle, draftMessage)
draft = ##gmail::Draft.new
draft.message = draftMessage
#service.create_user_draft('my.email#gmail.com', draft)
end
It throws a failedPrecondition: Bad Request (Google::Apis::ClientError) with the above code but when I added options: {authorization: #accessToken } as a third parameter of create_user_draft, the exception becomes Unauthorized (Google::Apis::AuthorizationError).
Can you help me go to the right path? I find the API documentation, on the Google API sites and on the source code itself, lackluster.
UPDATE
I have read here that in order for Service Accounts to work on the Gmail API, a paid Google Apps account is required (normal #gmail.com accounts won't work) since on the Admin Console is where we should have to enable the scopes for our Service Accounts.
Currently trying out JWT Credentials login.

Resources