google-api-nodejs-client - Service Account credentials authentication issues - google-api

I am trying to use the google-api-nodejs library to manage some resources in the google Campaign Manager API.
I have confirmed that we currently have a project configured, and that this project has the google Campaign Manager API enabled (see screenshot at the bottom).
I have tried several ways of authenticating myself (particularly API keys, OAuth2, and Service account credentials). This question will focus on using a Service Account for authentication purposes.
Now, I have generated a new service account keyfile (see screenshot at the bottom)), and I configured my code as follows, following the service-account-credentials section of the library's repo. I've also extended the auth scope to include the necessary scope according to this endpoint API docs
import { assert } from "chai";
import { google } from "googleapis";
it("can query userProfiles using service account keyfile", async () => {
try {
const auth = new google.auth.GoogleAuth({
keyFile:
"/full-path-to/credentials-service-account.json",
scopes: [
"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/dfatrafficking",
"https://www.googleapis.com/auth/ddmconversions",
"https://www.googleapis.com/auth/dfareporting",
],
});
const authClient = await auth.getClient();
// set auth as a global default
google.options({
auth: authClient,
});
const df = google.dfareporting("v3.5");
const res = await df.userProfiles.list({});
console.log("res: ", res);
assert(true);
} catch (e) {
console.error("error: ", e);
assert(false);
}
});
This results in the following error:
{
"code": 403,
"errors": [
{
"message": "Version v3.5 is no longer supported. Please upgrade to the latest version of the API.",
"domain": "global",
"reason": "forbidden"
}
]
}
This is an interesting error, because v3.5 is the latest version of that API (as of 14 April 2022) (This page shows the deprecation schedule: https://developers.google.com/doubleclick-advertisers/deprecation. Notice that v3.3 and v3.4 are deprecated, while v3.5 is not.)
In any case, using a different version of the dfareporting API still result in error:
// error thrown: "Version v3.5 is no longer supported. Please upgrade to the latest version of the API."
const df = google.dfareporting("v3.5");
// error thrown: "Version v3.4 is no longer supported. Please upgrade to the latest version of the API."
const df = google.dfareporting("v3.4");
// error thrown: 404 "The requested URL <code>/dfareporting/v3.3/userprofiles</code> was not found on this server"
const df = google.dfareporting("v3.3");
// Note 1: There are no other versions available
// Note 2: It is not possible to leave the version blank
const df = google.dfareporting();
// results in typescript error: "An argument for 'version' was not provided."
I also tried to query the floodlightActivities API, which failed with an authentication error.
// const res = await df.userProfiles.list({});
const res = await df.floodlightActivities.list({
profileId: "7474579",
});
This, in it's turn, results in the following error:
{
"code": 401,
"errors": [
{
"message": "1075 : Failed to authenticate. Google account can not access the user profile/account requested.",
"domain": "global",
"reason": "authError",
"location": "Authorization",
"locationType": "header"
}
]
}
Now, my question is:
am I doing something wrong while trying to authenticate using the service account credentials?
Or, is it possible that these endpoints do not support service-account-credentials?
Or, is something else going wrong here?

Related

MsalClientException IDW10104 from GetAccessTokenForAppAsync

I have an ASP.NET Core Web API set up as App Service in Azure with an App Registration in our AzureAd
In appsettings.json I have (anonimized)
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"Domain": "ourdomain.co.uk",
"TenantId": "n9n999n9-9999-nnnn-9n9n9-9n9n9n9n9n9",
"ClientId": "81933a15-157f-45b0-bc32-3d7d6d62f4a7",
"Audience": "https://ourdomain.co.uk/breathe.notifications-service",
"ClientSecret": "a6a6a6a~EEizqWNa8itAAAjcrycxnCtxaVgKTFx"
},
That app has an API permission in Azure Ad that allows me to call another app service, Audit. The audit service does not have any specific scopes defined but it does have an app role called Audit.Write
In the calling API i need to get a token to call audit so I run this code
var accessToken = await this.tokenAcquisition.GetAccessTokenForAppAsync(this.auditApiScope);
this.httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
this.httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
Note the call to GetAccessTokenForAppAsync rather than the more common GetAccessTokenForUserAsync
The scope string that I am passing is
https://ourdomain.co.uk/us.audit-service/.default
When I call GetAccessTokenForAppAsync it is failing with MSALException
IDW10104: Both client secret and client certificate cannot be null or
whitespace, and only ONE must be included in the configuration of the
web app when calling a web API. For instance, in the appsettings.json
file.
The client secret is in the AzureAd config, I am not specifying a certificate.
I now have this working and have two options but before I outline those I need to offer some extra background.
This Web Api and others we have created offer functionality to Azure Ad users and Azure B2C users. This functionality was first possible with Microsoft.Identity.Web 1.11.0 and we hjave been using 1.11.0 since it was released. However we always had an issue where we would generate thousands of exceptions because MSAL was getting confused ny which scheme to use.
We came across this blog post, Removing misleading IDX10501 logs when using multiple authentication schemes in ASP.NET Core 3.1 there is more detail in this github thread, https://github.com/oliviervaillancourt/blog/issues/3.
Our Startup.cs Configure Services looks like this
public void ConfigureServices(IServiceCollection services)
{
services.AddMicrosoftIdentityWebApiAuthentication(this.configuration)
.EnableTokenAcquisitionToCallDownstreamApi()
.AddInMemoryTokenCaches();
services.AddAuthentication()
.AddMicrosoftIdentityWebApi(this.configuration, "AzureAdB2C", "B2CScheme", true);
services.AddAuthentication("AzureAD_OR_AzureAdB2C")
.AddMicrosoftIdentityWebApi(
jwtBearerOptions =>
{
var azureAdB2CConfig = this.configuration.GetSection("AzureAdB2C");
jwtBearerOptions.ForwardDefaultSelector = context =>
{
var token = string.Empty;
if (context.Request.Headers.TryGetValue("Authorization", out var value))
{
string authorization = value;
if (authorization.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
{
token = authorization.Substring("Bearer ".Length).Trim();
}
}
if (token == null)
{
this.logger.LogInformation($"Cannot get the Token out of the Authorization header");
}
var jwtHandler = new JwtSecurityTokenHandler();
if (jwtHandler.CanReadToken(token))
{
var jwtToken = jwtHandler.ReadJwtToken(token);
var expectedB2CIssuer = $"{azureAdB2CConfig.GetValue<string>("Instance")}/{azureAdB2CConfig.GetValue<string>("TenantId")}/v2.0/";
if (string.Compare(jwtToken.Issuer, expectedB2CIssuer, true) == 0)
{
// Claim is from B2C so this request should be validated against the B2C scheme.
this.logger.LogInformation($"Request is with a B2C issued token so refer to B2CScheme. Token issuer: {jwtToken.Issuer} B2C Issuer: {expectedB2CIssuer}");
return "B2CScheme";
}
else
{
this.logger.LogInformation($"Request is not with a B2C issued token so refer to Bearer scheme. Token issuer: {jwtToken.Issuer} B2C Issuer: {expectedB2CIssuer}");
}
}
else
{
this.logger.LogInformation("Request token could not be read so refer to Bearer scheme");
}
return "Bearer";
};
},
identityOptions =>
{
var azureAdB2CConfig = this.configuration.GetSection("AzureAdB2C");
identityOptions.Instance = azureAdB2CConfig.GetValue<string>("Instance");
identityOptions.TenantId = "AzureAD_OR_AzureAdB2C";
identityOptions.ClientId = "AzureAD_OR_AzureAdB2C";
},
"AzureAD_OR_AzureAdB2C",
false);
services.AddControllers()
.AddNewtonsoftJson();
services.AddLogging(options =>
{
// hook the Console Log Provider
options.AddConsole();
options.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
// hook the Application Insights Provider
options.AddFilter<ApplicationInsightsLoggerProvider>(string.Empty, Microsoft.Extensions.Logging.LogLevel.Trace);
// pass the InstrumentationKey provided under the appsettings
options.AddApplicationInsights(this.configuration["APPINSIGHTS_INSTRUMENTATIONKEY"]);
});
}
The logic used by the ForwardDefaultSelector is what helps us work with multiple schemes and forward ASP.NET to the right scheme.
Now back to the answer.
If I remove the ForwardDefaultSelector I no longer get the IDW10104 however that is what we use to remopve all the extraneous exceptions schemes so that is not really going to be workable.
The only viable option is to move the Web Api from the latest version of Microsoft.Identity.Web 1.21.1 to 1.16.0. The issue that is causing us to get the exception was introduced in 1.16.1. I will raise an issue on the MSAL github for 1.16.1. We were previously using 1.11.0.

Bot framework SDK NodeJS and Custom Question Answering: Failed to generate answers: [object Object]

After an upgrade of one of my knowledgebases from QnaMaker to a language resource with custom question answering enable (FKA QnAMaker Managed), my chatbot does not function anymore and returns an error: DialogContextError: Failed to generate answers: [object Object]
My .env has:
QnAKnowledgebaseId=<name of my knowledgebase>
QnAEndpointKey=<the Ocp-Apim-Subscription-Key from the prediction url
QnAEndpointHostName=https://<languageresourcename>.cognitiveservices.azure.com/ (I also tried to add language/ to the url)
QnaMaker initialization:
const { QnAMaker } = require('botbuilder-ai');
try {
this.qnaMaker = new QnAMaker({
knowledgeBaseId: process.env.QnAKnowledgebaseId,
endpointKey: process.env.QnAEndpointKey,
host: process.env.QnAEndpointHostName
});
} catch (err) {
console.warn(`QnAMaker Exception: ${ err } Check your QnAMaker configuration in .env`);
}
And called by:
const qnaResults = await this.qnaMaker.getAnswers(step.context);
dependencies:
"dependencies": {
"adaptivecards-templating": "^1.4.0",
"applicationinsights": "^1.8.10",
"azure-storage": "^2.10.5",
"body-parser": "^1.19.0",
"botbuilder": "^4.15.0",
"botbuilder-ai": "^4.15.0",
"botbuilder-azure-blobs": "4.14.1-preview",
"botbuilder-dialogs": "^4.15.0",
"botbuilder-lg": "^4.15.0",
"botframework-config": "^4.11.2",
...
}
Using the key, knowledgebaseid and hostname from my old QnAMaker KB works just fine. Struggling to translate the new prediction url into an object for the QnAMaker.
Update
The class GenerateAnswerUtils (called by QnaMaker) is expecting the following:
const url = `${endpoint.host}/knowledgebases/${endpoint.knowledgeBaseId}/generateanswer`;
This will not work with a QnaMaker KB migrated to Custom Questions which expects an url like this:
'https://<languageresourcename>.cognitiveservices.azure.com/language/:query-knowledgebases?projectName=<projectname>&api-version=2021-10-01&deploymentName=production'
Not sure if this should be a bugreport or feature request. I will file an issue anyway.
The BotFramework SDK does not support the new Custom Questions feature yet. This was causing the error. A feature request is already filed.

YouTube Data API credential isn't working when using YouTube Data API

I used YouTube Data API to enable a web-based client I produced for a customer to connect to youtube and list out the videos on her account. This was working fine until the customer left those services unused for 90 days after which the credential was canceled.
I created a new credential but now it doesn't work and I can't figure out the solution. The client authorizes using OAuth then does a YouTube Youtube.Search.list
try{
youtube = getYouTubeService(XXXXXX);
}catch(NullPointerException e){
throw new BasicException("You must connect this system to YouTube before you can load videos. You may do this in Settings.");
}
HashMap<String, String> parameters = new HashMap<String,String>();
parameters.clear();
parameters.put("part", "id,snippet");
parameters.put("forMine", "true");
parameters.put("type", "video");
parameters.put("maxResults", "50");
YouTube.Search.List searchListMineRequest = youtube.search().list(parameters.get("part").toString());
if (parameters.containsKey("maxResults")) {
searchListMineRequest.setMaxResults(Long.parseLong(parameters.get("maxResults").toString()));
}
if (parameters.containsKey("forMine") && parameters.get("forMine") != "") {
boolean forMine = (parameters.get("forMine") == "true") ? true : false;
searchListMineRequest.setForMine(forMine);
}
if (parameters.containsKey("type") && parameters.get("type") != "") {
searchListMineRequest.setType(parameters.get("type").toString());
}
SearchListResponse response = searchListMineRequest.execute();
It is the execute command that produces the problem. The user connects using OAuth in another part of the program and the refresh token is stored for use with the above code. I am new to the google api so excuse if I'm getting some code wrong. I am getting a 403 error. However since the code worked before the credential was cancelled, then I conclude that the problem is with the new credential.
I get the following error message:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 403 Forbidden
{
"code" : 403,
"errors" : [ {
"domain" : "usageLimits",
"message" : "Access Not Configured. YouTube Data API has not been used in project XXXXXXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=XXXXXXXXXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.",
"reason" : "accessNotConfigured",
"extendedHelp" : "https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=XXXXXXXXXXXXX"
} ],
"message" : "Access Not Configured. YouTube Data API has not been used in project X before or it is disabled. Enable it by visiting.......
}
I found the solution. Once the project credential reached 90 days, my google console project was no longer a functioning project. Some things would work, others not. I had to completely delete the project, connect the youtube data API again and issue a new credential. This resolved the problem.

Failure connecting to S/4 Hana via Cloud SDK

I'm following the OData tutorial at https://blogs.sap.com/2017/05/21/step-4-with-sap-s4hana-cloud-sdk-calling-an-odata-service/comment-page-1/ and I'm getting an error when I try to retrieve business partners using DefaultBusinessPartnerService.
The relevant piece of code is:
DefaultBusinessPartnerService businessPartnerService = new DefaultBusinessPartnerService();
System.err.println("criated default business partner");
List<BusinessPartner> partners = businessPartnerService
.getAllBusinessPartner()
.select(BusinessPartner.BUSINESS_PARTNER,
BusinessPartner.LAST_NAME,
BusinessPartner.FIRST_NAME)
//.filter(BusinessPartner.BUSINESS_PARTNER_CATEGORY.eq(CATEGORY_VENDOR))
.orderBy(BusinessPartner.LAST_NAME, Order.ASC)
.execute(new ErpEndpoint(new ErpConfigContext()));
response.setContentType("application/json");
response.getWriter().write(new Gson().toJson(partners));
My ErpQueryEndpoint configuration is as follows:
#Mon May 14 15:27:09 BRT 2018
URL=https\://host\:port
Name=ErpQueryEndpoint
TrustAll=TRUE
Type=HTTP
Password=Password
Authentication=BasicAuthentication
User=Username
Where host, port, Username and Password have been replaced by the correct values.
When I query http://localhost:8080/s4integration-application/businesspartners I get the following error:
The endpoint responded with HTTP error code 403.
No service found for namespace , name API_BUSINESS_PARTNER, version 0001
Full error message:
{
"error": {
"code": "/IWFND/MED/170",
"message": {
"lang": "en",
"value": "No service found for namespace , name API_BUSINESS_PARTNER, version 0001"
},
"innererror": {
"application": {
"component_id": "",
"service_namespace": "/SAP/",
"service_id": "API_BUSINESS_PARTNER",
"service_version": "0001"
},
"transactionid": "C83CB3D2A1420000E005AF97B0836AD5",
"timestamp": "20180514182746.3576100",
"Error_Resolution": {
"SAP_Transaction": "Run transaction /IWFND/ERROR_LOG on SAP Gateway hub system (System Alias ) and search for entries with the timestamp above for more details",
"SAP_Note": "See SAP Note 1797736 for error analysis (https://service.sap.com/sap/support/notes/1797736)"
},
"errordetails": []
}
}
}
It seems that the endpoint is not configured on the SAP system (it's an S/4 Hana system). I'm not sure if I have to add something else to the URL besides the host and port or if there is some other configuration that has to be done on the SAP system.
Instructions for activating OData APIs from the SAP S/4HANA backend can be found here: help.sap.com/viewer/cdc25c83b63e482586b31b8acd49cf2f/1610%20003/… Just ignore the notion of the Fiori app.

Google Apps API - Getting a list of subscriptions or users

I am having trouble retrieving a list of subscriptions, or a list of users, to sync all the data with our own business system.
{
"error": {
"errors": [
{
"domain": "global",
"reason": "conditionNotMet",
"message": "Provisioning API is disabled for your domain. Please enable it in your Google Apps Control Panel.",
"locationType": "header",
"location": "If-Match"
}
],
"code": 412,
"message": "Provisioning API is disabled for your domain. Please enable it in your Google Apps Control Panel."
}
}
I am using the google PHP library, and this is basically the code I am using, which is a slightly modified example from googles example library.
$client->setScopes(array(
'https://www.googleapis.com/auth/apps.order', // As required by the google api docs for /subscription
));
/**
* Google's own example code
*/
if (isset($_GET['code'])) {
$client->authenticate();
$_SESSION['access_token'] = $client->getAccessToken();
$redirect = 'http://' . $_SERVER['HTTP_HOST'] . $_SERVER['PHP_SELF'];
header('Location: ' . filter_var($redirect, FILTER_SANITIZE_URL));
}
if (isset($_SESSION['access_token'])) {
$client->setAccessToken($_SESSION['access_token']);
}
if ($client->getAccessToken()) {
$url = 'https://www.googleapis.com/apps/reseller/v1';
//$url = 'https://www.googleapis.com/apps/reseller/v1sandbox/'; // sandbox mode overwrite
$req = new Google_HttpRequest($url .'/subscriptions', 'GET');
$resp = $client::getIo()->authenticatedRequest($req);
print "<h1>Subscriptions</h1>: <pre>" . $resp->getResponseBody() . "</pre>";
Since I'm failing at finding a solution to my problem I'm asking here hoping someone might know why I am getting a Provisioning API error, as the Provisioning API has been deprecated.
Enable API access from the control panel, it's in the error description.
Control panel -> otehr controls -> security -> api reference -> enable api access.
I'm trying to do the same thing and get an insufficient permissions... :(

Resources