InvalidOperationException: At least one client secrets (Installed or Web) should be set - youtube-data-api

I am using Google Youtube Api to authenticate my application using service account
my Json looks like
{
"type": "service_account",
"project_id": "uityityiuy",
"private_key_id": "hjklhkkkkkkkkkkh",
"private_key": "sdfggfdsgsdf",
"client_email": "asdfas",
"client_id": "asdfasasd",
"auth_uri": "qrwqerweq",
"token_uri": "qrwerq",
"auth_provider_x509_cert_url": "eryter",
"client_x509_cert_url": "ertyertytter"
}
Authentication script
using Google.Apis.YouTube.v3;
UserCredential credential;
using (var stream = new FileStream(keyFilePath, FileMode.Open, FileAccess.Read))
{
credential = await GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
scopes,
"user",
CancellationToken.None,
new FileDataStore("somename"));
var youtubeService = new YouTubeService(new
BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = "sosadfas"
});
return youtubeService;
}
I get
InvalidOperationException: At least one client secrets (Installed or
Web) should be set
Google.Apis.Auth.OAuth2.GoogleClientSecrets.get_Secrets() in
GoogleClientSecrets.cs, line 45
Any One With Ideas?

After going through google developer site they say
Service Accounts do not work with the YouTube API

Related

MsalClientException IDW10104 from GetAccessTokenForAppAsync

I have an ASP.NET Core Web API set up as App Service in Azure with an App Registration in our AzureAd
In appsettings.json I have (anonimized)
"AzureAd": {
"Instance": "https://login.microsoftonline.com/",
"Domain": "ourdomain.co.uk",
"TenantId": "n9n999n9-9999-nnnn-9n9n9-9n9n9n9n9n9",
"ClientId": "81933a15-157f-45b0-bc32-3d7d6d62f4a7",
"Audience": "https://ourdomain.co.uk/breathe.notifications-service",
"ClientSecret": "a6a6a6a~EEizqWNa8itAAAjcrycxnCtxaVgKTFx"
},
That app has an API permission in Azure Ad that allows me to call another app service, Audit. The audit service does not have any specific scopes defined but it does have an app role called Audit.Write
In the calling API i need to get a token to call audit so I run this code
var accessToken = await this.tokenAcquisition.GetAccessTokenForAppAsync(this.auditApiScope);
this.httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", accessToken);
this.httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
Note the call to GetAccessTokenForAppAsync rather than the more common GetAccessTokenForUserAsync
The scope string that I am passing is
https://ourdomain.co.uk/us.audit-service/.default
When I call GetAccessTokenForAppAsync it is failing with MSALException
IDW10104: Both client secret and client certificate cannot be null or
whitespace, and only ONE must be included in the configuration of the
web app when calling a web API. For instance, in the appsettings.json
file.
The client secret is in the AzureAd config, I am not specifying a certificate.
I now have this working and have two options but before I outline those I need to offer some extra background.
This Web Api and others we have created offer functionality to Azure Ad users and Azure B2C users. This functionality was first possible with Microsoft.Identity.Web 1.11.0 and we hjave been using 1.11.0 since it was released. However we always had an issue where we would generate thousands of exceptions because MSAL was getting confused ny which scheme to use.
We came across this blog post, Removing misleading IDX10501 logs when using multiple authentication schemes in ASP.NET Core 3.1 there is more detail in this github thread, https://github.com/oliviervaillancourt/blog/issues/3.
Our Startup.cs Configure Services looks like this
public void ConfigureServices(IServiceCollection services)
{
services.AddMicrosoftIdentityWebApiAuthentication(this.configuration)
.EnableTokenAcquisitionToCallDownstreamApi()
.AddInMemoryTokenCaches();
services.AddAuthentication()
.AddMicrosoftIdentityWebApi(this.configuration, "AzureAdB2C", "B2CScheme", true);
services.AddAuthentication("AzureAD_OR_AzureAdB2C")
.AddMicrosoftIdentityWebApi(
jwtBearerOptions =>
{
var azureAdB2CConfig = this.configuration.GetSection("AzureAdB2C");
jwtBearerOptions.ForwardDefaultSelector = context =>
{
var token = string.Empty;
if (context.Request.Headers.TryGetValue("Authorization", out var value))
{
string authorization = value;
if (authorization.StartsWith("Bearer ", StringComparison.OrdinalIgnoreCase))
{
token = authorization.Substring("Bearer ".Length).Trim();
}
}
if (token == null)
{
this.logger.LogInformation($"Cannot get the Token out of the Authorization header");
}
var jwtHandler = new JwtSecurityTokenHandler();
if (jwtHandler.CanReadToken(token))
{
var jwtToken = jwtHandler.ReadJwtToken(token);
var expectedB2CIssuer = $"{azureAdB2CConfig.GetValue<string>("Instance")}/{azureAdB2CConfig.GetValue<string>("TenantId")}/v2.0/";
if (string.Compare(jwtToken.Issuer, expectedB2CIssuer, true) == 0)
{
// Claim is from B2C so this request should be validated against the B2C scheme.
this.logger.LogInformation($"Request is with a B2C issued token so refer to B2CScheme. Token issuer: {jwtToken.Issuer} B2C Issuer: {expectedB2CIssuer}");
return "B2CScheme";
}
else
{
this.logger.LogInformation($"Request is not with a B2C issued token so refer to Bearer scheme. Token issuer: {jwtToken.Issuer} B2C Issuer: {expectedB2CIssuer}");
}
}
else
{
this.logger.LogInformation("Request token could not be read so refer to Bearer scheme");
}
return "Bearer";
};
},
identityOptions =>
{
var azureAdB2CConfig = this.configuration.GetSection("AzureAdB2C");
identityOptions.Instance = azureAdB2CConfig.GetValue<string>("Instance");
identityOptions.TenantId = "AzureAD_OR_AzureAdB2C";
identityOptions.ClientId = "AzureAD_OR_AzureAdB2C";
},
"AzureAD_OR_AzureAdB2C",
false);
services.AddControllers()
.AddNewtonsoftJson();
services.AddLogging(options =>
{
// hook the Console Log Provider
options.AddConsole();
options.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Trace);
// hook the Application Insights Provider
options.AddFilter<ApplicationInsightsLoggerProvider>(string.Empty, Microsoft.Extensions.Logging.LogLevel.Trace);
// pass the InstrumentationKey provided under the appsettings
options.AddApplicationInsights(this.configuration["APPINSIGHTS_INSTRUMENTATIONKEY"]);
});
}
The logic used by the ForwardDefaultSelector is what helps us work with multiple schemes and forward ASP.NET to the right scheme.
Now back to the answer.
If I remove the ForwardDefaultSelector I no longer get the IDW10104 however that is what we use to remopve all the extraneous exceptions schemes so that is not really going to be workable.
The only viable option is to move the Web Api from the latest version of Microsoft.Identity.Web 1.21.1 to 1.16.0. The issue that is causing us to get the exception was introduced in 1.16.1. I will raise an issue on the MSAL github for 1.16.1. We were previously using 1.11.0.

Authenticating to Google.Cloud.Storage.V1 using OAUTH with C#, but getting Error creating credential from JSON. Unrecognized credential type

I am trying to simply authenticate to the Google Cloud Storage API using OAuth2.0 and C# with a JSON file from Google.
here is my OAuth JSON file:
{
"client_id": "ClientIdHere",
"project_id": "ProjectName",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_secret": "ClientSecretHere",
"redirect_uris": [ "https://localhost:44349//RedirectPage.aspx" ]
}
Here is the code that I am using, I borrowed it from this post
protected void DoTheThingsAgain()
{
try
{
string bucketName = "Daves-Bucket";
string sharedkeyFilePath = "C:\\inetpub\\wwwroot\\GoogleAPITest\\GoogleAPITest\\App_Data\\JSONFile.json";
GoogleCredential credential = null;
using (var jsonStream = new FileStream(sharedkeyFilePath, FileMode.Open,
FileAccess.Read, FileShare.Read))
{
credential = GoogleCredential.FromStream(jsonStream);
}
}
catch (Exception ex)
{
throw ex;
}
}
Everything works fine until it hits this line:
**credential = GoogleCredential.FromStream(jsonStream);**
then it gives me the error: Error creating credential from JSON. Unrecognized credential type .
How do I specify the credential type for OAuth?
I have seen other solutions for this but not while using OAuth, only with ServiceAccounts.
That you in advance for any help you all can provide!

Access denied when using OAuth client credentials flow to create shared mailbox

Short version:
How to correctly set up application permissions and/or role assignments and/or something else that I'm missing, so that application id/secret (OAuth client credentials) can be used to create shared mailboxes?
So far I've tried couple combinations of permissions/roles, e.g. Exchange.ManageAsApp with User Administrator (fe930be7-5e62-47db-91af-98c3a49a38b1), Exchange administrator (29232cdf-9323-42fd-ade2-1d097af3e4de) and bunch of other.
Details:
I have a bunch of powershell scripts used to automate various tasks on Exchange Online. So far I've been using basic auth, which I was able to successfully convert into OAuth password flow.
But to get rid of dependency on service account completely, I'd prefer to use credentials flow. In background I'm trying to do something like this:
var authenticationContext = new AuthenticationContext($"https://login.microsoftonline.com/{TenantId}", false, _tokenCache);
var clientCredential = new ClientCredential(ClientId, ClientSecret);
var authenticationResult = await authenticationContext.AcquireTokenAsync(Resource, clientCredential);
var username = "OAuthUser#" + TenantId;
var password = authenticationResult.CreateAuthorizationHeader();
var executor = new ExolExecutor(username, password);
await executor.Execute(Script, cancellationToken);
where executor does the regular thing:
Create PSSession to https://outlook.office365.com/powershell-liveid?BasicAuthToOAuthConversion=true
Executes powershell script using
using PowerShell powershell = PowerShell.Create();
powershell.Runspace = runspace;
powershell.AddScript(script);
...
await Task.Factory.FromAsync(powershell.BeginInvoke(input, output), powershell.EndInvoke);
Remove PSSession
So far so good. Works perfectly fine with Get-Mailbox -ResultSize 1.
But when trying to create new shared mailbox New-Mailbox -Name "pko222" -DisplayName "pko222" -Alias "pko222" -Shared, I'm getting
CategoryInfo.Activity: New-Mailbox
CategoryInfo.Category: 1001
CategoryInfo.Reason: ADOperationException
CategoryInfo.TargetName:
CategoryInfo.TargetType:
ErrorDetails.Message:
ErrorDetails.RecommendedAction:
Exception.Message: Active Directory operation failed on DB7PR01A03DC005.EURPR01A003.prod.outlook.com. This error is not retriable. Additional information: Access is denied.
Active directory response: 00000005: SecErr: DSID-03152612, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0
FullyQualifiedErrorId: [Server=BEXP281MB0087,RequestId=88419a8e-78a4-4967-9bca-71d40feb5150,TimeStamp=10/6/2020 11:57:38 AM] [FailureCategory=Cmdlet-ADOperationException] 2C0312E5,Microsoft.Exchange.Management.RecipientTasks.NewMailbox
JWT token looks something like this:
{
"aud": "https://outlook.office365.com",
"iss": "https://sts.windows.net/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/",
"iat": 1601985127,
"nbf": 1601985127,
"exp": 1601989027,
"aio": "E2RgYFCOsw1iZj34elV49CH5zyd5AQ==",
"app_displayname": "XXXXXXXXXXX",
"appid": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"appidacr": "1",
"idp": "https://sts.windows.net/yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy/",
"oid": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz",
"rh": "0.AAAAv9y4fwZQ_0G6_d1kLKJ_sarAXb_REQFHhc2EM1FNn9tIAAA.",
"roles": ["User.Read.All", "full_access_as_app", "Mail.ReadWrite", "MailboxSettings.ReadWrite", "User.ReadBasic.All", "Mailbox.Migration", "Mail.Read", "Mail.Send", "MailboxSettings.Read", "Exchange.ManageAsApp"],
"sid": "qqqqqqqq-qqqq-qqqq-qqqq-qqqqqqqqqqqq",
"sub": "zzzzzzzz-zzzz-zzzz-zzzz-zzzzzzzzzzzz",
"tid": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy",
"uti": "CRytfXbD80y3ATmQvd-VAQ",
"ver": "1.0",
"wids": ["29232cdf-9323-42fd-ade2-1d097af3e4de", "88d8e3e3-8f55-4a1e-953a-9b9898b8876b", "fe930be7-5e62-47db-91af-98c3a49a38b1", "9360feb5-f418-4baa-8175-e2a00bac4301", "62e90394-69f5-4237-9190-012177145e10", "0997a1d0-0d1d-4acb-b408-d5ca73121e90"]
}
Fyi i managed to make it work on my side.
you just need to add the following param in the connection uri
&email=SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}#yourtenantname.onmicrosoft.com
so the connection uri looks like :
https://outlook.office365.com/PowerShell-LiveId?BasicAuthToOAuthConversion=true&email=SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}#yourtenantname.onmicrosoft.com
just change the suffix from 'yourtenantname' with ... your tenant name! dont put the tenant guid !
https://learn.microsoft.com/en-us/answers/questions/451006/pssession-and-modern-auth.html

Override default Identity Server 4 Client

I created a .NET Core 3.0 Angular project with Identity Server. I want to add claims and roles to my app.
My Identity Server is mostly out of the box with some simple route changes:
services.AddIdentity<ApplicationUser, IdentityRole>()
.AddDefaultTokenProviders()
.AddEntityFrameworkStores<ApplicationDbContext>();
services.AddIdentityServer(options =>
{
options.UserInteraction.LoginUrl = "/auth/login";
options.UserInteraction.LogoutUrl = "/auth/logout";
})
.AddApiAuthorization<ApplicationUser, ApplicationDbContext>();
I currently add my a simple policy in startup.cs
services.AddAuthorization(options =>
{
options.AddPolicy("RequireAdministratorRole", policy =>
{
policy.RequireRole("Admin");
policy.RequireClaim("CreateUser");
});
});
At the bottom of my Configure() method I call this method to add roles:
private async Task CreateUserRoles(IServiceProvider serviceProvider)
{
var RoleManager = serviceProvider.GetRequiredService<RoleManager<IdentityRole>>();
var UserManager = serviceProvider.GetRequiredService<UserManager<ApplicationUser>>();
IdentityResult adminRoleResult;
//Adding Admin Role
var adminRoleCheck = await RoleManager.RoleExistsAsync("Admin");
if (!adminRoleCheck)
{
//create the roles and seed them to the database
adminRoleResult = await RoleManager.CreateAsync(new IdentityRole("Admin"));
await RoleManager.AddClaimAsync(new IdentityRole("Admin"), new Claim("CreateUser", "Create User"));
}
}
In my register.cshtml.cs I've successfully set a role and a claim via:
var roleResult = await _userManager.AddToRoleAsync(user, "Admin");
var claimResult = await _userManager.AddClaimAsync(user, new Claim("CreateUser", "Create User"));
I've confirmed that the new user has that claim and role.
The client userinfo call returns correctly but when I look at the id_token I dont have those claims:
{
"nbf": 1574787988,
"exp": 1574788288,
"iss": "https://localhost:5001",
"aud": "MyAppSpa",
"iat": 1574787988,
"at_hash": "ndzldxAE3EiVzI4PeThNPQ",
"s_hash": "dIqJXx372XhOESn1XYH36A",
"sid": "VQLp--MHdoOoxXiVASWZ0g",
"sub": "4a0450dd-fe4f-4b3d-ac12-3f70876183e1",
"auth_time": 1574787983,
"idp": "local",
"amr": [
"pwd"
]
}
According to oidc-client-js is not getting claims correctly from Identity Server 4, I should just need to include AlwaysIncludeUserClaimsInIdToken = true to the client configuration.
However, the template doesnt have a configuration. Its all under the hood.
Questions:
1) Will adding AlwaysIncludeUserClaimsInIdToken = true to the client fix my issue?
2) How do I add it given my current configuration?
Concern about the size of ID Token , by default the claims won't include in ID token .You can get the claims from userinfo endpoint with ID token(read from user.profile) . That is by design .
The new .net core 3.0 angular authentication template just configures IdentityServer to use template supported configuration , but the configuration is not fully customize compare to Identity server provided configuration ,such as AlwaysIncludeUserClaimsInIdToken . So the workaround is not use ApiAuthorization service, the full power of IdentityServer is still available to customize authentication to suit your needs.

Why do I get 403 forbidden error while posting a json to elasticsearch endpoint on AWS?

I am posting a json to AWS elasticsearch,using a java lambda function.
public Object handleRequest(DynamodbEvent dynamodbEvent, Context context) {
//code to general the json document
AmazonDynamoDBClient amazonDynamoDBClient = new AmazonDynamoDBClient();
List<DynamodbEvent.DynamodbStreamRecord> dynamodbStreamRecordlist = dynamodbEvent.getRecords();
if (!dynamodbStreamRecordlist.isEmpty()) {
DynamodbEvent.DynamodbStreamRecord record = dynamodbStreamRecordlist.get(0);
if(record.getEventSource().equalsIgnoreCase("aws:dynamodb"))
tableName = getTableNameFromARN(record.getEventSourceARN());
}
LaneAnnotation laneAnnotation = new LaneAnnotation();
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = amazonDynamoDBClient.scan(scanRequest);
List<Lines> linesFinalList = new ArrayList<Lines>();
if(result != null) {
for (Map<String, AttributeValue> item : result.getItems()) {
//code for looping through the table items and generating a json object for the elastic search model
}
//Code to post the json below -
RestTemplate restTemplate = new RestTemplate();
SimpleClientHttpRequestFactory clientHttpRequestFactory = (SimpleClientHttpRequestFactory)restTemplate.getRequestFactory();
clientHttpRequestFactory.setConnectTimeout(10000);
clientHttpRequestFactory.setReadTimeout(10000);
HttpEntity<String> entity = new HttpEntity<String>(<json goes here>, headers);
try{
restTemplate.exchange(endpoint, HttpMethod.POST, entity, String.class);
}catch(Exception e){
e.printStackTrace();
}
}
However, I see the following error when I test my AWS lambda function -
org.springframework.web.client.HttpClientErrorException: 403 Forbidden
at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:700)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
at com.here.aws.LambdaApplication.handleRequest(LambdaApplication.java:166)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at lambdainternal.EventHandlerLoader$PojoMethodRequestHandler.handleRequest(EventHandlerLoader.java:456)
at lambdainternal.EventHandlerLoader$PojoHandlerAsStreamHandler.handleRequest(EventHandlerLoader.java:375)
at lambdainternal.EventHandlerLoader$2.call(EventHandlerLoader.java:1139)
at lambdainternal.AWSLambda.startRuntime(AWSLambda.java:285)
at lambdainternal.AWSLambda.<clinit>(AWSLambda.java:57)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at lambdainternal.LambdaRTEntry.main(LambdaRTEntry.java:94)
I even modified the access policy and added my IP address.
Have others faced this too? How did you resolve it?>
Any help will be appreciated.
EDIT1: I am now trying to incorporate signing of the request as is mentioned here - https://aws.amazon.com/blogs/security/how-to-control-access-to-your-amazon-elasticsearch-service-domain/
Will report back if it goes well.
EDIT2:
Here's the second way of sending a request that I tried referring to the link above-
#Override
public Object handleRequest(DynamodbEvent dynamodbEvent, Context context) {
AmazonDynamoDBClient amazonDynamoDBClient = new AmazonDynamoDBClient();
List<DynamodbEvent.DynamodbStreamRecord> dynamodbStreamRecordlist = dynamodbEvent.getRecords();
if (!dynamodbStreamRecordlist.isEmpty()) {
DynamodbEvent.DynamodbStreamRecord record = dynamodbStreamRecordlist.get(0);
if(record.getEventSource().equalsIgnoreCase("aws:dynamodb"))
tableName = getTableNameFromARN(record.getEventSourceARN());
}
LaneAnnotation laneAnnotation = new LaneAnnotation();
ScanRequest scanRequest = new ScanRequest().withTableName(tableName);
ScanResult result = amazonDynamoDBClient.scan(scanRequest);
List<Lines> linesFinalList = new ArrayList<Lines>();
if(result != null) {
for (Map<String, AttributeValue> item : result.getItems()) {
//Generate the json object that needs to be sent in the request
}
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON_UTF8);
Request<?> request = new DefaultRequest<Void>(SERVICE_NAME);
request.setContent(new ByteArrayInputStream(elasticSearchModel.toString().getBytes()));
request.setEndpoint(URI.create(endpoint));
request.setHttpMethod(HttpMethodName.POST);
AWS4Signer signer = new AWS4Signer();
signer.setServiceName(SERVICE_NAME);
signer.setRegionName(Regions.US_EAST_1.getName());
AWSCredentialsProvider credsProvider =
new DefaultAWSCredentialsProviderChain();
AWSCredentials creds = credsProvider.getCredentials();
// Sign request with supplied creds
signer.sign(request, creds);
log.info("Request signed");
ExecutionContext executionContext = new ExecutionContext(true);
ClientConfiguration clientConfiguration = new ClientConfiguration();
AmazonHttpClient client = new AmazonHttpClient(clientConfiguration);
MyHttpResponseHandler<Void> responseHandler = new MyHttpResponseHandler<Void>();
MyErrorHandler errorHandler = new MyErrorHandler();
Response<Void> response =
client.execute(request, responseHandler, errorHandler, executionContext);
return dynamodbEvent;
}
However, I get the following error -
Check the signature you provided. Check your AWS Secret Access Key and signing method. Consult the service documentation for details.
The Canonical String for this request should have been
'GET
/
host:somehostname-XXXXXXXXXXXXXXXX.us-east-1.es.amazonaws.com
x-amz-date:20170130T105736Z
x-amz-security-token:FQoDYXdzEG4aDJJ4ryjXXXXXXXXXXXXXXXX/auMHooYENY6YXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
host;x-amz-date;x-amz-security-token
e3b0c4429XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
The String-to-Sign should have been
'AWS4-HMAC-SHA256
20170130T105736Z
20170130/us-east-1/es/aws4_request
9a5b4c92ec121c333f8cdXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
"}"
10:57:36.818 [main] DEBUG org.apache.http.headers - http-outgoing-1 << HTTP/1.1 403 Forbidden
AWS Elastic Search has a security gateway that provides authentication. The authentication options are configured in the AWS Elastic Search console.
You are receiving a 403 authentication error because your AWS Elastic Search access policy does not allow the IP for the NAT Gateway that your Lambda uses.
http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-createupdatedomains.html#es-createdomain-configure-access-policies
Here is an access policy template that you can use for IP based authentication.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "es:*",
"Resource": "arn:aws:es:us-east-1:YOUR-AWS-ACCOUNT-ID:domain/YOUR-ELASTICSEARCH-DOMAIN-NAME/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"YOUR-NAT-GATEWAY-PUBLIC-IP/32"
]
}
}
}
]
}

Resources