I am getting an error trying to connect to an SQS queue in another AWS account using JMS. I have tried to follow the approach taken in this answer, but I am receiving the following error:
com.amazonaws.services.sqs.model.AmazonSQSException: The address https://sqs.us-east-1.amazonaws.com/ is not valid for this endpoint. (Service: AmazonSQS; Status Code: 404; Error Code: InvalidAddress; Request ID: d7f72bd3-6240-5f63-b313-70c2d8978c14; Proxy: null)
Unlike in the post mentioned above (which I believe has the account credentials in the default provider chain?) I am trying to assume a role that has access to this SQS queue. Is this not possible through JMS or am I doing something incorrectly?
import com.amazon.sqs.javamessaging.ProviderConfiguration;
import com.amazon.sqs.javamessaging.SQSConnectionFactory;
import com.amazon.sqs.javamessaging.SQSSession;
import com.amazonaws.auth.AWSCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.securitytoken.AWSSecurityTokenService;
import com.amazonaws.services.securitytoken.AWSSecurityTokenServiceClient;
import com.amazonaws.services.sqs.AmazonSQSClientBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.config.DefaultJmsListenerContainerFactory;
import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.support.destination.DynamicDestinationResolver;
import javax.jms.ConnectionFactory;
import javax.jms.JMSException;
import javax.jms.Queue;
import javax.jms.Session;
/**
* A configuration class for JMS to poll an SQS queue
* in another AWS account
*/
#Configuration
public class TranslationJmsConfig {
private static final Logger LOGGER = LoggerFactory.getLogger(TranslationJmsConfig.class);
#Value("${iam.connection.arn}")
private String connectionRoleArn;
#Value("${account.id}")
private String brokerAccountId;
/**
* JmsListenerContainerFactory bean for translation processing response queue
*
* #param concurrentConsumers number of concurrent consumers
* #param maxConcurrentConsumers max number of concurrent consumers
* #return An instance of JmsListenerContainerFactory
*/
#Bean("translationJmsListenerContainerFactory")
public DefaultJmsListenerContainerFactory translationJmsListenerContainerFactory(
#Value("#{new Integer('${listener.concurrency}')}") int concurrentConsumers,
#Value("#{new Integer('${listener.max.concurrency}')}") int maxConcurrentConsumers) {
DefaultJmsListenerContainerFactory factory =
new DefaultJmsListenerContainerFactory();
factory.setConnectionFactory(getConnectionFactory(connectionRoleArn));
factory.setDestinationResolver(new SqsDynamicDestinationResolver(brokerAccountId));
factory.setSessionTransacted(false); //SQS does not support transaction.
factory.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE); // Automatic message acknowledgment after successful listener execution; best-effort redelivery in case of a user exception thrown as well as in case of other listener execution interruptions (such as the JVM dying).
factory.setConcurrency(String.format("%d-%d", concurrentConsumers, maxConcurrentConsumers));
return factory;
}
/**
* create custom JMS Template
* #return JmsTemplate
*/
#Bean
public JmsTemplate customJmsTemplate() {
JmsTemplate jmsTemplate = new JmsTemplate(getConnectionFactory(connectionRoleArn));
jmsTemplate.setDestinationResolver(new SqsDynamicDestinationResolver(brokerAccountId));
return jmsTemplate;
}
/**
* A dynamic destination resolver for sqs queue
*/
public class SqsDynamicDestinationResolver extends DynamicDestinationResolver {
private final String brokerAccountId;
/**
* Constructor
* #param brokerAccountId broker Account Id
*/
public SqsDynamicDestinationResolver(String brokerAccountId) {
this.brokerAccountId = brokerAccountId;
}
#Override
protected Queue resolveQueue(Session session, String queueName) throws JMSException {
if (session instanceof SQSSession) {
SQSSession sqsSession = (SQSSession) session;
return sqsSession.createQueue(queueName, brokerAccountId); // 404 invalid address -- Something wrong with creds?
}
return super.resolveQueue(session, queueName);
}
}
private ConnectionFactory getConnectionFactory(String connectionRoleArn){
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClient.builder()
.build();
// assume the connector account credentials -> so we can assume customer account using chaining
AWSCredentialsProvider dummyCredentialProviders = IdentityHelpers.assumeInternalRole(stsClient, connectionRoleArn); // A helper that assumes temporary creds
return new SQSConnectionFactory(
new ProviderConfiguration(),
AmazonSQSClientBuilder.standard()
.withRegion(Regions.US_EAST_1)
.withCredentials(dummyCredentialProviders)
);
}
}
I realized that when using the temporary credentials, I didn't need the second parameter (the account id) of the sqsSession.createQueue call. so once i changed
sqsSession.createQueue(queueName, brokerAccountId);
To:
return sqsSession.createQueue(queueName);
it worked fine. I guess i missunderstood the need for the account id. I assume the parameter is used when you have multiple accounts in your providerChain and you want it to search a specific account? Any light on this would still be appreciated!
I've recently tried using the new AWS DocumentDB service as my DB in a Spring application.
The cluster has been created in the same VPC as the EKS on which I deploy my application. Security groups allow connections between all nodes in the VPC.
AWS exposes a mongo URI like this for my DB cluster:
mongodb://<my-user>:<insertYourPassword>#<my-cluster-endpoint>:27017/?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0
My question:
How do I make my Spring code work with this kind of connection?
I have tried adding the followig to my application.properties file:
spring.data.mongodb.uri=mongodb://<my-user>:<insertYourPassword>#<my-cluster-endpoint>:27017/admin?ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs00
spring.data.mongodb.database=admin
server.ssl.key-store=classpath:rds-combined-ca-bundle.pem
And placing the PEM file in /src/main/resources
However the code still fails to connect to the DB cluster.
I get this message as an error: No server chosen by com.mongodb.client.internal.MongoClientDelegate
Followed by a Exception in monitor thread while connecting to server ...
And finally a timeout exception: com.mongodb.MongoSocketReadTimeoutException: Timeout while receiving message
It looks kind of like a security group issue but I have no problem connecting with mongo shell from the same EC2 running the Spring application Pod.
Any ideas?
As mentioned in the documentation,
By design, you access Amazon DocumentDB (with MongoDB compatibility) resources from an Amazon EC2 instance within the same Amazon VPC as the Amazon DocumentDB resources. However, suppose that your use case requires that you or your application access your Amazon DocumentDB resources from outside the cluster's Amazon VPC. In that case, you can use SSH tunneling (also known as "port forwarding") to access your Amazon DocumentDB resources.
Connect from outside VPC
Your Amazon DocumentDB cluster should be running in your default virtual private cloud (VPC). To interact with your Amazon DocumentDB cluster, you must launch an Amazon Elastic Compute Cloud (Amazon EC2) instance into your default VPC, in the same AWS Region where you created your Amazon DocumentDB cluster.
Follow the guide to connect to the cluster
AWS DocumentDB cluster
GitHub Reference: spring-boot-aws-documentdb
Update:
To connect through SSL, use below logic by setting SSL_CERTIFICATE pointing to aws region specific intermediate certificate.
This can be downloaded from SSL certs and copy it to base directory.
Alternatively, you can provide absolute path to the variable SSL_CERTIFICATE.
private static final String SSL_CERTIFICATE = "rds-ca-2015-us-east-1.pem";
private static final String KEY_STORE_TYPE = "JKS";
private static final String KEY_STORE_PROVIDER = "SUN";
private static final String KEY_STORE_FILE_PREFIX = "sys-connect-via-ssl-test-cacerts";
private static final String KEY_STORE_FILE_SUFFIX = ".jks";
private static final String DEFAULT_KEY_STORE_PASSWORD = "changeit";
public static void main(String[] args) {
SSLContextHelper.setSslProperties();
SpringApplication.run(Application.class, args);
}
protected static class SSLContextHelper{
/**
* This method sets the SSL properties which specify the key store file, its type and password:
* #throws Exception
*/
private static void setSslProperties() {
try {
System.setProperty("javax.net.ssl.trustStore", createKeyStoreFile());
} catch (Exception e) {
e.printStackTrace();
}
System.setProperty("javax.net.ssl.trustStoreType", KEY_STORE_TYPE);
System.setProperty("javax.net.ssl.trustStorePassword", DEFAULT_KEY_STORE_PASSWORD);
}
private static String createKeyStoreFile() throws Exception {
return createKeyStoreFile(createCertificate()).getPath();
}
/**
* This method generates the SSL certificate
* #return
* #throws Exception
*/
private static X509Certificate createCertificate() throws Exception {
CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
URL url = new File(SSL_CERTIFICATE).toURI().toURL();
if (url == null) {
throw new Exception();
}
try (InputStream certInputStream = url.openStream()) {
return (X509Certificate) certFactory.generateCertificate(certInputStream);
}
}
/**
* This method creates the Key Store File
* #param rootX509Certificate - the SSL certificate to be stored in the KeyStore
* #return
* #throws Exception
*/
private static File createKeyStoreFile(X509Certificate rootX509Certificate) throws Exception {
File keyStoreFile = File.createTempFile(KEY_STORE_FILE_PREFIX, KEY_STORE_FILE_SUFFIX);
try (FileOutputStream fos = new FileOutputStream(keyStoreFile.getPath())) {
KeyStore ks = KeyStore.getInstance(KEY_STORE_TYPE, KEY_STORE_PROVIDER);
ks.load(null);
ks.setCertificateEntry("rootCaCertificate", rootX509Certificate);
ks.store(fos, DEFAULT_KEY_STORE_PASSWORD.toCharArray());
}
return keyStoreFile;
}
}
connection output:
019-01-17 13:33:22.316 INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster : Canonical address mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017 does not match server address. Removing mongodb.cluster-cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017 from client view of cluster
2019-01-17 13:33:22.401 INFO 3598 --- [onaws.com:27017] org.mongodb.driver.connection : Opened connection [connectionId{localValue:2}] to mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:22.403 INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster : Monitor thread successfully connected to server with description ServerDescription{address=mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017, type=REPLICA_SET_PRIMARY, state=CONNECTED, ok=true, version=ServerVersion{versionList=[3, 6, 0]}, minWireVersion=0, maxWireVersion=6, maxDocumentSize=16777216, logicalSessionTimeoutMinutes=null, roundTripTimeNanos=2132149, setName='rs0', canonicalAddress=mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017, hosts=[mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017], passives=[], arbiters=[], primary='mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017', tagSet=TagSet{[]}, electionId=7fffffff0000000000000001, setVersion=null, lastWriteDate=Thu Jan 17 13:33:21 UTC 2019, lastUpdateTimeNanos=516261208876}
2019-01-17 13:33:22.406 INFO 3598 --- [onaws.com:27017] org.mongodb.driver.cluster : Discovered replica set primary mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:22.595 INFO 3598 --- [ main] com.barath.app.CustomerService : Saving the customer with customer details com.barath.app.Customer#6c130c45
2019-01-17 13:33:22.912 INFO 3598 --- [ main] org.mongodb.driver.connection : Opened connection [connectionId{localValue:3}] to mongodb.cktoiipu3bbd.us-east-1.docdb.amazonaws.com:27017
2019-01-17 13:33:23.936 INFO 3598 --- [ main] pertySourcedRequestMappingHandlerMapping : Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)]
The answer provided by #Sunny Pelletier worked for me with a mashup of #Frank's answer in our Java setup.
So for me, I wanted a solution that worked for our local docker setup and for any of our AWS environments that have active profiles and other env vars set in our environment via the CDK.
I first started with a simple Configuration POJO to setup my properties outside the spring.data.mongo.* paradigm. You don't have to do this and can just let Spring handle it as it normally does to create the MongoClient.
My default local dev application.yml and corresponding config class.
mongo:
user: mongo
password: mongo
host: localhost
port: 27017
database: my-service
#Data
#Configuration
#ConfigurationProperties(prefix = "mongo")
public class MongoConnectConfig {
private int port;
private String host;
private String user;
private String database;
private String password;
}
Then, I created two AbstractMongoClientConfiguration child classes; one for local and one for non-local. The key here is that I didn't create my own MongoClient. The reason is because I want all the good Spring Boot initialization stuff that you get with the framework. For example, the auto-registration of all the converters and such.
Instead, I leveraged the customization hook provided by AbstractMongoClientConfiguration.configureClientSettings(MongoClientSettings.Builder builder) to then aggregate the custom settings like the .pem piece.
The other part is that I leveraged profiles to enable/disable the configurations to make it "seamless" for local developers; we don't use any profiles other than default for local development so it's easier to get setup without having to "know" so much from the start.
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
#Slf4j
#Configuration
#RequiredArgsConstructor
#Profile({"!dev && !qa && !prod"})
#EnableMongoRepositories(basePackages = "co.my.data.repositories")
public class LocalDevMongoConfig extends AbstractMongoClientConfiguration {
private final MongoConnectConfig config;
#Override
public String getDatabaseName() {
return config.getDatabase();
}
#Override
protected void configureClientSettings(MongoClientSettings.Builder builder) {
log.info("Applying Local Dev MongoDB Configuration");
builder.applyConnectionString(new ConnectionString(getConnectionString()));
}
//mongodb://${mongo.user}:${mongo.password}#${mongo.host}:${mongo.port}/${mongo.database}?authSource=admin
private String getConnectionString() {
return String.format("mongodb://%s:%s#%s:%s/%s?authSource=admin",
config.getUser(),
config.getPassword(),
config.getHost(),
config.getPort(),
config.getDatabase()
);
}
}
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import lombok.RequiredArgsConstructor;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManagerFactory;
import java.io.ByteArrayInputStream;
import java.io.File;
import java.nio.file.Files;
import java.security.KeyStore;
import java.security.cert.CertificateFactory;
import java.util.Arrays;
import java.util.stream.Collectors;
#Slf4j
#Configuration
#RequiredArgsConstructor
#Profile({"dev || qa || prod"})
#EnableMongoRepositories(basePackages = "co.my.data.repositories")
public class DocumentDbMongoConfig extends AbstractMongoClientConfiguration {
private final MongoConnectConfig config;
#Override
public String getDatabaseName() {
return config.getDatabase();
}
#SneakyThrows
#Override
protected void configureClientSettings(MongoClientSettings.Builder builder) {
log.info("Applying AWS DocumentDB Configuration");
builder.applyConnectionString(new ConnectionString(getConnectionString()));
var endOfCertificateDelimiter = "-----END CERTIFICATE-----";
File resource = new ClassPathResource("certs/rds-combined-ca-bundle.pem").getFile();
String pemContents = new String(Files.readAllBytes(resource.toPath()));
var allCertificates = Arrays.stream(pemContents
.split(endOfCertificateDelimiter))
.filter(line -> !line.isBlank())
.map(line -> line + endOfCertificateDelimiter)
.collect(Collectors.toUnmodifiableList());
var certificateFactory = CertificateFactory.getInstance("X.509");
var keyStore = KeyStore.getInstance(KeyStore.getDefaultType());
// This allows us to use an in-memory key-store
keyStore.load(null);
for (int i = 0; i < allCertificates.size(); i++) {
var certString = allCertificates.get(i);
var caCert = certificateFactory.generateCertificate(new ByteArrayInputStream(certString.getBytes()));
keyStore.setCertificateEntry(String.format("AWS-certificate-%s", i), caCert);
}
var trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
trustManagerFactory.init(keyStore);
var sslContext = SSLContext.getInstance("TLS");
sslContext.init(null, trustManagerFactory.getTrustManagers(), null);
builder.applyToSslSettings(ssl -> {
ssl.enabled(true).context(sslContext);
});
}
/**
* Partly based on the AWS Console "Connectivity & security " section in the DocumentDB Cluster View.
* Since we register the pem above, we don't need to add the ssl & sslCAFile piece
* mongodb://${user}:${password}#${host}:${port}/?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false
*/
private String getConnectionString() {
return String.format("mongodb://%s:%s#%s:%s/%s?replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false",
config.getUser(),
config.getPassword(),
config.getHost(),
config.getPort(),
config.getDatabase()
);
}
}
Lastly, we place the rds-combined-ca-bundle.pem in the src/main/resources/certs/ folder.
Side Notes:
Again, I believe you should be able to get away with using the default spring.data* properties and your MongoClient should have used them.
Ignore the #SneakyThrows here, I just did that for code brevity purposes, handle your checked exceptions as you see fit.
I guess we can see why Kotlin syntax can be considered "cleaner" huh? :)
I can confirm the solution provided by #Barath allows you to secure the AWS DocumentDB TLS connection inside the Java application itself. This is a much cleaner approach compared to the one described by AWS on https://docs.aws.amazon.com/documentdb/latest/developerguide/connect_programmatically.html which requires you to run a script on your server which is more complicated and difficult for automated deploys etc.
To further set up the connection itself in the Spring application I used the following #Configuration class, which allows you to connect to a local MongoDB for testing during development, and the AWS one once deployed with settings from the properties file.
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.core.convert.converter.Converter;
import org.springframework.data.mongodb.config.AbstractMongoClientConfiguration;
import org.springframework.data.mongodb.core.convert.MongoCustomConversions;
import org.springframework.data.mongodb.repository.config.EnableMongoRepositories;
#Configuration
#EnableMongoRepositories(basePackages = "YOUR.PACKAGE.WITH.repository")
public class MongoDbConfig extends AbstractMongoClientConfiguration {
#Value("${spring.profiles.active}")
private String activeProfile;
#Value("${mongodb.host:localhost}")
private String dbUri;
#Value("${mongodb.port:27017}")
private int dbPort;
#Value("${mongodb.database.name:YOUR_DOCUMENTDB_NAME}")
private String dbName;
#Value("${mongodb.username:}")
private String dbUser;
#Value("${mongodb.password:}")
private String dbPassword;
#Override
public String getDatabaseName() {
return dbName;
}
#Override
public MongoClient mongoClient() {
ConnectionString connectionString = new ConnectionString(getConnectionString());
MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
.applyConnectionString(connectionString)
.build();
return MongoClients.create(mongoClientSettings);
}
private String getConnectionString() {
if (activeProfile.contains("local")) {
return String.format("mongodb://%s:%s/%s", dbUri, dbPort, dbName);
}
return String.format("mongodb://%s:%s#%s:%s/%s?ssl=true&replicaSet=rs0&readpreference=secondaryPreferred&retrywrites=false",
dbUser, dbPassword, dbUri, dbPort, dbName);
}
}
I actually faced the same issue as you did, but now AWS uses rds-combined-ca-bundle.pem which combines together many certificates into one.
If you don't want to create a trust-store using their outdated documentation, you can do it yourself and have the rds-combined-ca-bundle.pem into your application generating the key-store at runtime.
I managed to get this to work with this code sample. This has been tested with spring:2.4, mongo-driver: 4.1.1 and documentDB using mongo 4.0 compatibility.
val endOfCertificateDelimiter = "-----END CERTIFICATE-----"
// rds-combined-ca-bundle.pem contains more than one certificate. We need to add them all to the trust-store independantly.
val allCertificates = ClassPathResource("certificates/rds-combined-ca-bundle.pem").file.readText()
.split(endOfCertificateDelimiter)
.filter { it.isNotBlank() }
.map { it + endOfCertificateDelimiter }
val certificateFactory = CertificateFactory.getInstance("X.509")
val keyStore = KeyStore.getInstance(KeyStore.getDefaultType())
keyStore.load(null) // This allows us to use an in-memory key-store
allCertificates.forEachIndexed { index, certificate ->
val caCert = certificateFactory.generateCertificate(certificate.byteInputStream()) as X509Certificate
keyStore.setCertificateEntry("AWS-certificate-$index", caCert)
}
val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm())
trustManagerFactory.init(keyStore)
val sslContext = SSLContext.getInstance("TLS")
sslContext.init(null, trustManagerFactory.trustManagers, null)
builder.applyToSslSettings {
it.enabled(true)
.context(sslContext)
}
Here is a solution that worked for me just call the setSslProperties method before you connect to your documentdb.
/**
* This method sets the SSL properties which specify the key store file, its type and password.
*
* #throws Exception
*/
private static void setSslProperties() throws Exception {
System.setProperty("javax.net.ssl.trustStore", createKeyStoreFile());
System.setProperty("javax.net.ssl.trustStoreType", KEY_STORE_TYPE);
System.setProperty("javax.net.ssl.trustStorePassword", DEFAULT_KEY_STORE_PASSWORD);
}
/**
* This method returns the path of the Key Store File needed for the SSL verification during the IAM Database Authentication to
* the db instance.
*
* #return
* #throws Exception
*/
private static String createKeyStoreFile() throws Exception {
return createKeyStoreFile(createCertificate()).getPath();
}
/**
* This method generates the SSL certificate.
*
* #return
* #throws Exception
*/
private static X509Certificate createCertificate() throws Exception {
final CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
final ClassLoader classLoader = MyClass.class.getClassLoader();
final InputStream is = classLoader.getResourceAsStream(SSL_CERTIFICATE);
return (X509Certificate) certFactory.generateCertificate(is);
}
/**
* This method creates the Key Store File.
*
* #param rootX509Certificate - the SSL certificate to be stored in the KeyStore
* #return
* #throws Exception
*/
private static File createKeyStoreFile(final X509Certificate rootX509Certificate) throws Exception {
final File keyStoreFile = File.createTempFile(KEY_STORE_FILE_PREFIX, KEY_STORE_FILE_SUFFIX);
try (final FileOutputStream fos = new FileOutputStream(keyStoreFile.getPath())) {
final KeyStore ks = KeyStore.getInstance(KEY_STORE_TYPE, KEY_STORE_PROVIDER);
ks.load(null);
ks.setCertificateEntry("rootCaCertificate", rootX509Certificate);
ks.store(fos, DEFAULT_KEY_STORE_PASSWORD.toCharArray());
}
return keyStoreFile;
}
Here are the constants.
public static final String SSL_CERTIFICATE = "rds-ca-2019-root.pem";
public static final String KEY_STORE_TYPE = "JKS";
public static final String KEY_STORE_PROVIDER = "SUN";
public static final String KEY_STORE_FILE_PREFIX = "sys-connect-via-ssl-test-cacerts";
public static final String KEY_STORE_FILE_SUFFIX = ".jks";
public static final String DEFAULT_KEY_STORE_PASSWORD = "changeit";
Here is the link for rds-ca-2019-root.pem file place that file inder resources folder.
let me know this works for you.
Here is a sample
setSslProperties();
final MongoCredential credential = MongoCredential.createCredential(userName, mongoProps.getDatabaseName(), password.toCharArray());
final MongoClientSettings settings = MongoClientSettings.builder()
.credential(credential)
.readPreference(ReadPreference.secondaryPreferred())
.retryWrites(false)
.applyToSslSettings(builder -> builder.enabled(true))
.applyToConnectionPoolSettings(connPoolBuilder ->
ConnectionPoolSettings.builder().
maxSize(1).build())
.applyToClusterSettings(builder ->
builder.hosts(Arrays.asList(new ServerAddress(clusterEndPoint, 27017))))
.build();
mongoClient = MongoClients.create(settings);
As pointed out by #mmr25 in comments to #Barath answer, The solution only works for when service needs to only connect to documentDB. You start getting "Gettting PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested" for other http requests.
To address this issue we need to only enable sslcontext for documentdb connections. To do we can use Netty as HttpClient for mongodb connections. To enable netty we need to add following maven dependency to your spring boot project:
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-tcnative-boringssl-static</artifactId>
<version>2.0.53.Final</version>
</dependency>
and put your pem file in your resources folder and define following beans in one of the class annotated with #Configutation annotations.
#Slf4j
#Configuration
public class MongoDbConfiguration {
private static final String AWS_PUBLIC_KEY_NAME = "rds-ca-2019-root.pem";
private final String mongoConnectionUri;
private final String databaseName;
public MongoDbConfiguration(#Value("${spring.data.mongodb.uri}") String mongoConnectionUri, #Value("${spring.data.mongodb.database}") String databaseName) {
this.mongoConnectionUri = mongoConnectionUri;
this.databaseName = databaseName;
}
#Bean
#Primary
#SneakyThrows
#Profile("!default")
public MongoClient mongoClient() {
SslContext sslContext = SslContextBuilder.forClient()
.sslProvider(SslProvider.OPENSSL)
.trustManager(new ClassPathResource(AWS_PUBLIC_KEY_NAME).getInputStream())
.build();
ConnectionString connectionString = new ConnectionString(mongoConnectionUri);
return MongoClients.create(
MongoClientSettings.builder()
.applyConnectionString(connectionString)
.applyToSslSettings(builder -> {
builder.enabled((null == connectionString.getSslEnabled()) ? false : connectionString.getSslEnabled());
builder.invalidHostNameAllowed((null == connectionString.getSslInvalidHostnameAllowed()) ? false : connectionString.getSslInvalidHostnameAllowed());
})
.streamFactoryFactory(NettyStreamFactoryFactory.builder()
.sslContext(sslContext)
.build())
.build());
}
}
Import Statements:
import com.mongodb.ConnectionString;
import com.mongodb.MongoClientSettings;
import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.connection.netty.NettyStreamFactoryFactory;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.SslProvider;
import lombok.SneakyThrows;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Primary;
import org.springframework.context.annotation.Profile;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.mongodb.MongoDatabaseFactory;
import org.springframework.data.mongodb.MongoTransactionManager;
Now you should be able to connect to your documentdb and other http connection should also work as expected.
Reference: https://www.mongodb.com/docs/drivers/java/sync/current/fundamentals/connection/tls/#customize-tls-ssl-configuration-through-the-netty-sslcontext
The Simple solution is you can remove the TLS (SSL) option in AWS, then you can remove the "ssl_ca_certs=rds-combined-ca-bundle.pem" from your connection string. But if the application required the SSL DB connectivity, then you can use the
AWS Guide
I'm using spring-security-core 2.0 in my grails 2.4.4 application. I limited my concurrent session access of a user to 1.
So the problem is after session timeout of 30 minutes I couldn't login with that username and password, it will throw the concurrent session maximum exceeded exception.
I'm doubtful that, on the session timeout my logout is not working properly so that session can still be active.
I'm a newbi to spring-security so can anyone tell me what to do?
Here I'm giving What are changes I have made in my code to limit the concurrent access.
resources.groovy
// Place your Spring DSL code here
/*beans = {
messageSource(org.springframework.context.support.ReloadableResourceBundleMessageSource) { basename = "classpath:grails-app/src/resource_bundle" }
}*/
import org.springframework.security.core.session.SessionRegistryImpl;
import org.springframework.security.web.authentication.session.ConcurrentSessionControlAuthenticationStrategy;
import org.springframework.security.web.authentication.session.SessionFixationProtectionStrategy;
import org.springframework.security.web.authentication.session.RegisterSessionAuthenticationStrategy;
import org.springframework.security.web.authentication.session.CompositeSessionAuthenticationStrategy;
import com.custom.sessiontime.CustomSessionLogoutHandler
beans = {
sessionRegistry(SessionRegistryImpl)
customSessionLogoutHandler(CustomSessionLogoutHandler,ref('sessionRegistry'))
concurrentSessionControlAuthenticationStrategy(ConcurrentSessionControlAuthenticationStrategy,ref('sessionRegistry')){
exceptionIfMaximumExceeded = true
maximumSessions = 1
}
sessionFixationProtectionStrategy(SessionFixationProtectionStrategy){
migrateSessionAttributes = true
alwaysCreateSession = true
}
registerSessionAuthenticationStrategy(RegisterSessionAuthenticationStrategy,ref('sessionRegistry'))
sessionAuthenticationStrategy(CompositeSessionAuthenticationStrategy,[ref('concurrentSessionControlAuthenticationStrategy'),ref('sessionFixationProtectionStrategy'),ref('registerSessionAuthenticationStrategy')])
}
CustomSessionLogoutHandler
package com.custom.sessiontime
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.springframework.security.core.Authentication;
import org.springframework.security.web.authentication.logout.LogoutHandler;
import org.springframework.util.Assert;
import org.springframework.security.core.session.SessionRegistry;
/**
* {#link CustomSessionLogoutHandler} is in charge of removing the {#link SessionRegistry} upon logout. A
* new {#link SessionRegistry} will then be generated by the framework upon the next request.
*
* #author Mohd Qusyairi
* #since 0.1
*/
public final class CustomSessionLogoutHandler implements LogoutHandler {
private final SessionRegistry sessionRegistry;
/**
* Creates a new instance
* #param sessionRegistry the {#link SessionRegistry} to use
*/
public CustomSessionLogoutHandler(SessionRegistry sessionRegistry) {
Assert.notNull(sessionRegistry, "sessionRegistry cannot be null");
this.sessionRegistry = sessionRegistry;
}
/**
* Clears the {#link SessionRegistry}
*
* #see org.springframework.security.web.authentication.logout.LogoutHandler#logout(javax.servlet.http.HttpServletRequest,
* javax.servlet.http.HttpServletResponse,
* org.springframework.security.core.Authentication)
*/
public void logout(HttpServletRequest request, HttpServletResponse response,
Authentication authentication) {
this.sessionRegistry.removeSessionInformation(request.getSession().getId());
}
}
Finally added below line to Config.groovy
grails.plugin.springsecurity.logout.handlerNames = ['customSessionLogoutHandler','securityContextLogoutHandler','rememberMeServices']
I want to cache a List of Category when level == 0, but keeping getting IllegalArgumentException. What am I missing?
In Service class:
#Override
#Transactional(readOnly = true)
#Cacheable(value="categories", condition="#level == 0")
public List<Category> findCategoryByLevel(int level) throws DataAccessException {
return categoryRepository.findCategoryByLevel(level);
}
Error:
java.lang.IllegalArgumentException: Cannot find cache named 'categories' for CacheableOperation[public java.util.List com.mySite.service.DidicityServiceImpl.findCategoryByLevel(int) throws org.springframework.dao.DataAccessException] caches=[categories] | key='' | condition='#level == 0' | unless=''
What caching provider are you using in Spring's Cache Abstraction? (I.e. ehcache, Guava, Hazelcast, etc)
It appears you are missing an explicit "Cache" definition and instance in your actual caching provider. For example, when using Pivotal GemFire as a caching provider in Spring's Cache Abstraction, you need to define a Region (a.k.a. Cache in the Spring Cache Abstraction), using your example above, like so...
<gfe:cache ...>
<gfe:replicated-region id="categories" persistent="false"...>
...
</gfe:replicated-region>
Spring Data GemFire goes onto lookup the "Cache" when the cached application service|repository method is invoked, and so the actual backing "Cache" (i.e. the GemFire Region) must exist, otherwise the Spring Cache Abstraction throws an IllegalArgumentException.
So, by way of a more explicit example, I wrote the following test...
/*
* Copyright 2014-present the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.spring.cache;
import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.*;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.spring.cache.CachingWithConcurrentMapUsingExplicitlyNamedCachesTest.ApplicationConfiguration;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.CacheManager;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.cache.concurrent.ConcurrentMapCacheManager;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Service;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
/**
* The CachingWithConcurrentMapUsingExplicitlyNamedCachesTest class is a test suite of test cases testing the contract
* and functionality of Spring Cache Abstraction using the ConcurrentMap-based Cache Management Strategy
* with explicitly named "Caches".
*
* NOTE: when the Cache(s) [is|are] explicitly named using the ConcurrentMapCacheManager, then "dynamic" is disabled
* and corresponding the named Cache in the #Cacheable annotation of the cached service method must exist
* (or be declared). If no explicitly named Caches are provided to the ConcurrentMapManager constructor, then dynamic
* is enabled and the Cache will be created at runtime, on the fly.
*
* #author John Blum
* #see org.junit.Test
* #see org.junit.runner.RunWith
* #see org.springframework.cache.Cache
* #see org.springframework.cache.CacheManager
* #see org.springframework.cache.annotation.Cacheable
* #see org.springframework.cache.annotation.EnableCaching
* #see org.springframework.cache.concurrent.ConcurrentMapCacheManager
* #see org.springframework.context.annotation.Bean
* #see org.springframework.context.annotation.Configuration
* #see org.springframework.test.context.ContextConfiguration
* #see org.springframework.test.context.junit4.SpringJUnit4ClassRunner
* #since 1.0.0
*/
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = ApplicationConfiguration.class)
#SuppressWarnings("unused")
public class CachingWithConcurrentMapUsingExplicitlyNamedCachesTest {
#Autowired
private NumberCategoryService numberCategoryService;
#Test
public void numberCategoryCaching() {
assertThat(numberCategoryService.isCacheMiss(), is(false));
List<NumberCategory> twoCategories = numberCategoryService.classify(2.0);
assertThat(twoCategories, is(notNullValue()));
assertThat(twoCategories.size(), is(equalTo(3)));
assertThat(twoCategories.containsAll(Arrays.asList(
NumberCategory.EVEN, NumberCategory.POSITIVE, NumberCategory.WHOLE)), is(true));
assertThat(numberCategoryService.isCacheMiss(), is(true));
List<NumberCategory> twoCategoriesAgain = numberCategoryService.classify(2.0);
assertThat(twoCategoriesAgain, is(sameInstance(twoCategories)));
assertThat(numberCategoryService.isCacheMiss(), is(false));
List<NumberCategory> negativeThreePointFiveCategories = numberCategoryService.classify(-3.5);
assertThat(negativeThreePointFiveCategories, is(notNullValue()));
assertThat(negativeThreePointFiveCategories.size(), is(equalTo(3)));
assertThat(negativeThreePointFiveCategories.containsAll(Arrays.asList(
NumberCategory.ODD, NumberCategory.NEGATIVE, NumberCategory.FLOATING)), is(true));
assertThat(numberCategoryService.isCacheMiss(), is(true));
}
#Configuration
#EnableCaching
public static class ApplicationConfiguration {
#Bean
public CacheManager cacheManager() {
//return new ConcurrentMapCacheManager("Categories");
return new ConcurrentMapCacheManager("Temporary");
}
#Bean
public NumberCategoryService numberCategoryService() {
return new NumberCategoryService();
}
}
#Service
public static class NumberCategoryService {
private volatile boolean cacheMiss;
public boolean isCacheMiss() {
boolean localCacheMiss = this.cacheMiss;
this.cacheMiss = false;
return localCacheMiss;
}
protected void setCacheMiss() {
this.cacheMiss = true;
}
#Cacheable("Categories")
public List<NumberCategory> classify(double number) {
setCacheMiss();
List<NumberCategory> categories = new ArrayList<>(3);
categories.add(isEven(number) ? NumberCategory.EVEN : NumberCategory.ODD);
categories.add(isPositive(number) ? NumberCategory.POSITIVE : NumberCategory.NEGATIVE);
categories.add(isWhole(number) ? NumberCategory.WHOLE : NumberCategory.FLOATING);
return categories;
}
protected boolean isEven(double number) {
return (isWhole(number) && Math.abs(number) % 2 == 0);
}
protected boolean isFloating(double number) {
return !isWhole(number);
}
protected boolean isNegative(double number) {
return (number < 0);
}
protected boolean isOdd(double number) {
return !isEven(number);
}
protected boolean isPositive(double number) {
return (number > 0);
}
protected boolean isWhole(double number) {
return (number == Math.floor(number));
}
}
public enum NumberCategory {
EVEN,
FLOATING,
NEGATIVE,
ODD,
POSITIVE,
WHOLE
}
}
This test example is currently setup to throw the IllegalArgumentException. If you change this...
return new ConcurrentMapCacheManager("Temporary");
To this...
return new ConcurrentMapCacheManager("Categories");
Then all is well.
Hopefully this adequately illustrates the problem you are having and how to fix it.
Cheers,
John
Looks like you want to use the key as a static hardcoded string.
Try the following "'categories'"
#Override
#Transactional(readOnly = true)
#Cacheable(value="'categories'", condition="#level == 0")
public List<Category> findCategoryByLevel(int level) throws DataAccessException {
return categoryRepository.findCategoryByLevel(level);
}
I am trying to use jetty7 to build a transparent proxy setup. Idea is to hide origin servers behind the jetty server so that the incoming request can be forwarded in a transparent manner to the origin servers.
I want to know if I can use jetty's ProxyServlet.Transparent implementation to do so. If yes, can anyone give me some examples.
This example is based on Jetty-9. If you want to implement this with Jetty 8, implement the proxyHttpURI method (See Jetty 8 javadocs.). Here is some sample code.
import java.io.IOException;
import java.net.InetAddress;
import java.net.URI;
import java.util.List;
import java.util.Map;
import java.util.Random;
import javax.servlet.ServletConfig;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.http.HttpServletRequest;
import org.eclipse.jetty.servlets.ProxyServlet;
/**
* When a request cannot be satisfied on the local machine, it asynchronously
* proxied to the destination box. Define the rule
*/
public class ContentBasedProxyServlet extends ProxyServlet {
private int remotePort = 8080;
public void setPort(int port) {
this.remotePort = port;
}
public void init(ServletConfig config) throws ServletException {
super.init(config);
}
public void service(ServletRequest request, ServletResponse response) throws IOException, ServletException {
super.service(request, response);
}
/**
* Applicable to Jetty 9+ only.
*/
#Override
protected URI rewriteURI(HttpServletRequest request) {
String proxyTo = getProxyTo(request);
if (proxyTo == null)
return null;
String path = request.getRequestURI();
String query = request.getQueryString();
if (query != null)
path += "?" + query;
return URI.create(proxyTo + "/" + path).normalize();
}
private String getProxyTo(HttpServletRequest request) {
/*
* Implement this method: All the magic happens here. Use this method to figure out your destination machine address. You can maintain
* a static list of addresses, and depending on the URI or request content you can route your request transparently.
*/
}
}
Further more, you can implement a Filter that determines whether the request needs to terminate on the local machine or on the destination machine. If the request is meant for the remote machine, forward the request to this servlet.
// Declare this method call in the filter.
request.getServletContext()
.getNamedDispatcher("ContentBasedProxyServlet")
.forward(request, response);