Creating RDS Datasource fails with "AmazonRDSException: The security token included in the request is invalid" - spring

I am unable to start an application with
#EnableRdsInstance(databaseName = "test",
dbInstanceIdentifier = "test",
password = "password",
username = "username",
readReplicaSupport = true
)
The exception I get is:
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'test': Invocation of init method failed; nested exception is com.amazonaws.services.rds.model.AmazonRDSException: The security token included in the request is invalid. (Service: AmazonRDS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 925519ec-582e-11e7-8ca6-8159eafdc3e8)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1628) ~[spring-beans-4.3.9.RELEASE.jar:4.3.9.RELEASE]
...
Caused by: com.amazonaws.services.rds.model.AmazonRDSException: The security token included in the request is invalid. (Service: AmazonRDS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 925519ec-582e-11e7-8ca6-8159eafdc3e8)
at ...
Tried all configurations that are suggested in Spring Cloud AWS Docs including ENV variable, System.setProperties(), and in application.yml as below
cloud:
aws:
credentials:
accessKey: XXXXXXX
secretKey: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
region:
static: us-east-2
also tried even hardcoding in in aws-beans
<beans ...>
<aws-context:context-credentials>
<aws-context:simple-credentials access-key="XXXXXXXXXX" secret-key="XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"/>
</aws-context:context-credentials>
<aws-context:context-resource-loader/>
<aws-context:context-region region="us-east-2" />
</beans>
and nothing works, help is appreciated....

Possible reasons to get this exception:
Make sure public access is enabled for RDS Instance and allow incoming traffic for your machine in Security Group.
Make sure internet gateway is attached with VPC and very importantly you have to use public subnet rather than private subnet.

Related

Wildfly GraphQL Smallrye #Routingcontext

I have setup a Wildfly 27 server with GraphQL Featurepack.
I need to access the Request Headers to fetch a bearertoken.
I find no good doc on how to do this.
My assumption is that i should inject the RoutingContext like this.
#GraphQLApi
#ApplicationScoped
public class FilmResource {
#Inject
GalaxyService service;
#Inject
RoutingContext routingContext;
#Query("allFilms")
#Description("Get all Films from a galaxy far far away")
public List<Film> getAllFilms() {
return service.getAllFilms();
}
}
However this fails runtime with
ERROR [controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("deploy") failed - address: ([("deployment" => "cms-graph-ql-1.1.0-SNAPSHOT.war")]) - failure description:
{"WFLYCTL0080: Failed services" => {"jboss.deployment.unit."cms-graph-ql-1.1.0-SNAPSHOT.war".WeldStartService" => "Failed to start service
Caused by: org.jboss.weld.exceptions.DeploymentException: WELD-001408: Unsatisfied dependencies for type RoutingContext with qualifiers #Default
at injection point [BackedAnnotatedField] #Inject com.scanreco.cms.microprofile.graphql.FilmResource.routingContext
at com.scanreco.cms.microprofile.graphql.FilmResource.routingContext(FilmResource.java:0)
I would be greatful for any help.
So i figured this one out.
Since i am using WF and not Quarkus i am forced to fetch the request headers via #Webfilter.
(Vert.x is not at all used for HTTP in my stack).
This is a minor inconvenince but quite alright.
#WebFilter(servletNames = {"SmallRyeGraphQLExecutionServlet"},
filterName = "CMS Azure AD Filter",
description = "Filter all ServletCalls for Azure AD Bearer token",
dispatcherTypes = {DispatcherType.REQUEST})
#RequestScoped
public class CmsGraphQlServletAzureAdFilter implements Filter {
The name of the executionservlet was found in their gitRepo.

Application won't start when adding function.routing.enabled=true with AWS Kinesis binder

I have a Spring Cloud Stream application where I'm using the AWS Kinesis Binder. I have the following configuration in the application.yml file:
cloud:
stream:
bindings:
customerEvents-out-0:
binder: kinesis
destination: 'customers-service-stream-name'
content-type: application/json
functionRouter-in-0:
destination: 'another-service-stream-name'
content-type: application/json
default:
group: customers-service-group
kinesis:
binder:
checkpoint:
table: 'SpringIntegrationMetadataStore'
locks:
table: 'SpringIntegrationLockRegistry'
headers:
- spring.cloud.function.definition
The application starts without errors.
The moment I add function.routing.enabled=true, the application won't start.
Configuration which stops the app from starting
cloud:
stream:
function:
routing:
enabled: true
bindings:
customerEvents-out-0:
binder: kinesis
destination: 'customers-service-stream-name'
content-type: application/json
functionRouter-in-0:
destination: 'another-service-stream-name'
content-type: application/json
default:
group: customers-service-group
kinesis:
binder:
checkpoint:
table: 'SpringIntegrationMetadataStore'
locks:
table: 'SpringIntegrationLockRegistry'
headers:
- spring.cloud.function.definition
This is part of the logs I see:
Caused by:
org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'kinesisBinderHealthIndicator' defined
in
org.springframework.cloud.stream.binder.kinesis.config.KinesisBinderConfiguration$KinesisBinderHealthIndicatorConfiguration:
Unsatisfied dependency expressed through method
'kinesisBinderHealthIndicator' parameter 0; nested exception is
org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'kinesisMessageChannelBinder' defined in
org.springframework.cloud.stream.binder.kinesis.config.KinesisBinderConfiguration:
Unsatisfied dependency expressed through method
'kinesisMessageChannelBinder' parameter 3; nested exception is
org.springframework.beans.factory.BeanCreationException: Error
creating bean with name 'kinesisCheckpointStore' defined in
org.springframework.cloud.stream.binder.kinesis.config.KinesisBinderConfiguration:
Invocation of init method failed; nested exception is
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: The
security token included in the request is invalid. (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code:
UnrecognizedClientException; Request ID:
BDMVHHRUG4F7DD4GOH9POQGUKBVV4KQNSO5AEMVJF66Q9ASUAAJG; Proxy: null)
Caused by:
com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException: The
security token included in the request is invalid. (Service:
AmazonDynamoDBv2; Status Code: 400; Error Code:
UnrecognizedClientException
Caused by: org.springframework.beans.factory.BeanCreationException:
Error creating bean with name 'kinesisCheckpointStore'
Caused by:
org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'kinesisMessageChannelBinder'
Caused by:
org.springframework.beans.factory.UnsatisfiedDependencyException:
Error creating bean with name 'kinesisBinderHealthIndicator'
org.springframework.context.ApplicationContextException: Failed to
start bean 'inputBindingLifecycle'
In order to make the application start, I have to create the following 2 beans:
#Bean
AmazonKinesisAsync amazonKinesis(AWSCredentialsProvider awsCredentialsProvider) {
return AmazonKinesisAsyncClientBuilder
.standard()
.withCredentials(awsCredentialsProvider)
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(endpointUrl, region))
.build();
}
#Bean
#Primary
AmazonDynamoDBAsync amazonDynamoDBAsync(AWSCredentialsProvider awsCredentialsProvider) {
return AmazonDynamoDBAsyncClientBuilder.standard()
.withCredentials(awsCredentialsProvider)
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(endpointUrl, region))
.build();
}
However, these are beans that I add for local development with localstack, I shouldn't add these beans for prod.
Any ideas?

How to disable oauth2 client registration in #WebFluxTest?

Having configured oauth2:
security:
oauth2:
resourceserver:
jwt:
issuer-uri: ${keycloak.server-url}/realms/${keycloak.realm}
client:
provider:
keycloak:
issuer-uri: ${keycloak.server-url}/realms/${keycloak.realm}
user-name-attribute: email
registration:
keycloak:
client-id: ${keycloak.client-id}
client-secret: ${keycloak.client-secret}
scope: openid
I wonder how to run a #WebFluxTest for a controller without having to run an actual keycloak server. The following test runs fine, when I have the local keycloak server running:
#WebFluxTest(TaskController::class)
class TaskControllerTest {
#Autowired
private lateinit var client: WebTestClient
#MockkBean
private lateinit var taskService: TaskServiceImpl
#Test
#WithMockUser(roles = ["user"])
fun getByIds() {
val ids = setOf<Long>(1, 2)
every { taskService.getByIds(ids, any()) } returns flowOf(
TaskWithRating(
Task(
id = 1,
title = "title",
description = "desc"
),
Rating(0, 0),
)
)
client
.get()
.uri { uriBuilder ->
uriBuilder
.path("/tasks/byIds")
.queryParam("ids", ids)
.build()
}
.exchange()
.expectStatus().isOk
.expectBody()
.jsonPath("$.length()").isEqualTo(1)
.jsonPath("$.1.task.id").isEqualTo(1)
}
}
When I stop the local keycloak server, the test fails, because spring tries to read the oauth2 configuration during startup, while creating the ApplicationContext:
Failed to load ApplicationContext
...
Factory method 'clientRegistrationRepository' threw exception; nested exception is java.lang.IllegalArgumentException: Unable to resolve Configuration with the provided Issuer of "https://keycloak.local.com/auth/realms/myrealm"
...
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "https://keycloak.local.com/auth/realms/myrealm/.well-known/openid-configuration": Connection refused; nested exception is java.net.ConnectException: Connection refused
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:784)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:669)
at org.springframework.security.oauth2.client.registration.ClientRegistrations.lambda$oidc$0(ClientRegistrations.java:156)
at org.springframework.security.oauth2.client.registration.ClientRegistrations.getBuilder(ClientRegistrations.java:209)
... 175 more
Caused by: java.net.ConnectException: Connection refused
The test does not need it, because the user is mocked anyway. So I think of disabling the auth2 configuration in #WebFluxTest, but I could not find out how and if this is a valid approach?
I tried this.
#WebFluxTest(
controllers = [TaskController::class],
excludeAutoConfiguration = [ReactiveSecurityAutoConfiguration::class],
)
and this
#WebFluxTest(
controllers = [TaskController::class],
excludeAutoConfiguration = [ReactiveOAuth2ClientAutoConfiguration::class],
)
But the error is the same.
Turns out I was on the right path. Both, the oauth2 client and resource server auto configurations must be disabled. This works:
#WebFluxTest(
controllers = [TaskController::class],
excludeAutoConfiguration = [
ReactiveOAuth2ClientAutoConfiguration::class,
ReactiveOAuth2ResourceServerAutoConfiguration::class,
],
)
Note that this bypasses potentially disabled features and hence, for example, requires a csrf() mutation on the test client:
client
.mutateWith(SecurityMockServerConfigurers.csrf())
An alternative to this would be to configure a test-configuration for the security setup using #Import(TestSecurityConfiguration::class).

Spring Cloud AWS SQS AccessDenied

I am currently having a connection issue trying to connect to an AWS SQS Queue using Spring Cloud and Spring Boot. I believe I have everything configured fine but am getting:
2015-07-01 18:12:11,926 [WARN][-]
org.springframework.boot.context.embedded.AnnotationConfigEmbeddedWebApplicationContext[487]
- Exception encountered during context initialization - cancelling refresh attempt
org.springframework.context.ApplicationContextException: Failed to
start bean 'simpleMessageListenerContainer'; nested exception is
com.amazonaws.AmazonServiceException: Access to the resource
https://sqs.us-west-2.amazonaws.com/{Number}/{Queue Name} is denied.
(Service: AmazonSQS; Status Code: 403; Error Code: AccessDenied;
Request ID: 87312428-ec0f-5990-9f69-6a269a041b4d)
#Configuration
#EnableSqs
public class CloudConfiguration {
private static final Logger log = Logger.getLogger(CloudConfiguration.class);
#MessageMapping("QUEUE")
public void retrieveProvisionMessages(User user) {
log.warn(user.firstName);
}
}
YML
cloud:
aws:
credentials.accessKey: AccessKey
credentials.secretKey: SecretKey
region.static: us-west-2
credentials.instanceProfile: true
When it attempts to connect I see that a header value of:
AWS4-HMAC-SHA256 Credential=accesskey/20150701/us-west-2/sqs/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=signature
After the request is sent:
HTTP/1.1 403 Forbidden [Server: Server, Date: Wed, 01 Jul 2015 22:51:25 GMT, Content-Type: text/xml, Content-Length: 349, Connection: keep-alive, x-amzn-RequestId: Request Id] org.apache.http.conn.BasicManagedEntity#37e55df6
I have checked all AIM policies and they are correct.
Using:
private AmazonSQS establishQueue(){
AmazonSQS sqs = new AmazonSQSClient(new BasicAWSCredentials(accessKey, secretKey));
sqs.setRegion(RegionUtils.getRegion(region));
return sqs;
}
AmazonSQS sqs = establishQueue();
return sqs.receiveMessage(sqs.getQueueUrl(userProductPurchase).getQueueUrl());
with the same credentials works fine. Any help is greatly appreciated.
Thanks
Do you have GetQueueAttributes calls allowed for your IAM user?
I think it's using also few more operations. Not only ReceiveMessage and GetQueueUrl.
In my case, using Spring Cloud, I had to set the following permissions up:
sqs:DeleteMessage
sqs:GetQueueUrl
sqs:ReceiveMessage
sqs:SendMessage
sqs:GetQueueAttributes

How to deal with AWS security for SNS

I am trying to publish SNS message to AWS with Spring Cloud. But I am newbie with this and cannot understand security settings.
I created a user in AWS. I gave him a "AmazonSNSRole" role.
I have code:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.aws.messaging.core.NotificationMessagingTemplate;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.*;
#RestController
#RequestMapping("/sns")
public class SnsSenderController {
private static final Logger theLogger = LoggerFactory.getLogger(SnsSenderController.class);
private final NotificationMessagingTemplate notificationMessagingTemplate;
#Autowired
public SnsSenderController(NotificationMessagingTemplate notificationMessagingTemplate) {
this.notificationMessagingTemplate = notificationMessagingTemplate;
}
#RequestMapping(value = "/send", method = RequestMethod.POST)
#ResponseStatus(HttpStatus.OK)
public void sendNotification(#RequestBody SnsNotification notification) {
theLogger.debug("Going to send notification {}", notification);
this.notificationMessagingTemplate.sendNotification("SnsTopic", notification.getMessage(), notification.getSubject());
}
}
I have beans.xml where I put the keys for my AMS user:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:aws-context="http://www.springframework.org/schema/cloud/aws/context"
xmlns:aws-messaging="http://www.springframework.org/schema/cloud/aws/messaging"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/cloud/aws/context
http://www.springframework.org/schema/cloud/aws/context/spring-cloud-aws-context-1.0.xsd
http://www.springframework.org/schema/cloud/aws/messaging
http://www.springframework.org/schema/cloud/aws/messaging/spring-cloud-aws-messaging-1.0.xsd">
<!-- Define global credentials for all the AWS clients -->
<aws-context:context-credentials>
<aws-context:instance-profile-credentials/>
<aws-context:simple-credentials access-key="accessKeyForMyUser"
secret-key="secretKeyForMyUser"/>
</aws-context:context-credentials>
<!-- Messaging-->
<aws-messaging:notification-messaging-template id="notificationMessagingTemplate" />
</beans>
But when I am trying to send SNS I see error:
02:08:36.445 [http-nio-8080-exec-4] ERROR o.a.c.c.C.[.[.[.[dispatcherServlet] - Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is com.amazonaws.AmazonServiceException: The security token included in the request is invalid. (Service: AmazonSNS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 65c99dfe-daf2-5f50-a408-d0013c568f48)] with root cause
com.amazonaws.AmazonServiceException: The security token included in the request is invalid. (Service: AmazonSNS; Status Code: 403; Error Code: InvalidClientTokenId; Request ID: 65c99dfe-daf2-5f50-a408-d0013c568f48)
Why so? Please, could somebody explain me how to deal with it? It is really difficult part for newbie like me. Thank you in advance!
You need to provide either instance profile credentials or user credentials.
Since you are using user credentials remove the instance profile credentials from spring xml beans

Resources