I have 2 issues trying to connect to dynamodb in aws. It's working locally:
#Configuration
class DynamoDbConfig {
#Value("${amazon.access.key}")
private String awsAccessKey;
#Value("${amazon.access.secret.key}")
private String awsSecretKey;
#Value("${amazon.dynamodb.endpoint}")
private String awsDynamoDBEndPoint;
#Value("${amazon.dynamodb.region}")
private String awsDynamoDBRegion;
#Bean
public AWSCredentials amazonAWSCredentials() {
return new BasicAWSCredentials(awsAccessKey, awsSecretKey);
}
public AWSCredentialsProvider amazonAWSCredentialsProvider() {
return new AWSStaticCredentialsProvider(amazonAWSCredentials());
}
#Bean
public DynamoDB dynamoDB() {
AmazonDynamoDB amazonDynamoDB = AmazonDynamoDBClientBuilder.standard()
.withEndpointConfiguration(new AwsClientBuilder.EndpointConfiguration(awsDynamoDBEndPoint, awsDynamoDBRegion))
.withCredentials(amazonAWSCredentialsProvider())
.build();
return new DynamoDB(amazonDynamoDB);
}
}
application-local.properties
amazon.access.key=key1
amazon.access.secret.key=key2
amazon.dynamodb.endpoint=http://localhost:8000
amazon.dynamodb.region=us-east-1
application-prod.properties
amazon.access.key=${AWS_ACCESS_KEY_ID}
amazon.access.secret.key=${AWS_SECRET_ACCESS_KEY}
amazon.dynamodb.endpoint=dynamodb.us-east-1.amazonaws.com
amazon.dynamodb.region=${AWS_DEFAULT_REGION}
I already got credentials and my .aws/credentials looks good:
[default]
aws_access_key_id = MyKeyId
aws_secret_access_key = MySecretKey
aws_session_token = blablabla
disney_session_expiration = This is also ok
1 Issue) It looks like always take the application-local.properties profile, if I show the awsAccessKey and awsSecretKey in the class DynamoDbConfig, I get key1 and key2. I tried with these 2 commands:
mvn spring-boot:run -Dspring.profiles.active=prod
mvn spring-boot:run -Pprod
2 Issue) I renamed application-prod.properties as application.properties to make spring takes that config file and I get this error message:
Could not resolve placeholder 'AWS_SECRET_ACCESS_KEY' in value "${AWS_SECRET_ACCESS_KEY}"
I guess the profile is not an issue, the values are not set/defined for the following keys
${AWS_ACCESS_KEY_ID}
${AWS_SECRET_ACCESS_KEY}
${AWS_DEFAULT_REGION}
Related
I have an env variable $MONGODB_URI
echo $MONGODB_URI
mongodb://localhost:27017
My application.properties
mongoUri=${MONGODB_URI}
My Config.java
#Configuration
public class Config {
#Value("${mongoUri}")
private String mongoUri;
..
}
When I try to start up the app in IntelliJ Idea, I get
Could not resolve placeholder 'MONGODB_URI' in value "${MONGODB_URI}"
The app starts up fine with
./gradlew bootRun
How can I properly configure IntelliJ to read from the environment? I'll need to swap out the db url depending on if it's prod, local, etc.
You need to run to your application with environment configuration as shown below in image:
sample code to verify
#RestController
public class StatusController {
private final Environment environment;
public StatusController(Environment environment) {
this.environment = environment;
}
#GetMapping("/env")
public String envValue() {
return environment.getProperty("MONGODB_URI");
}
}
I cannot connect to Amazon S3 through the IAM role. I have been told that I cannot use the secret-key or the access-key, but I can't find any way to do it without this.
Actually what I have is this:
public class S3Config {
#Autowired
Environment env;
#Value("${amazon.aws.accesskey}")
private String awsId;
#Value("${amazon.aws.secretkey}")
private String awsKey;
#Value("${amazon.aws.role}")
private String roleArn;
#Value("${amazon.aws.region}")
private String region;
#Bean
#Scope("prototype")
#Primary
public AmazonS3Client s3client() {
BasicAWSCredentials awsCreds = new BasicAWSCredentials("", "");
AWSSecurityTokenService stsClient = AWSSecurityTokenServiceClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCreds))
.withRegion(Regions.fromName(region))
.build();
AssumeRoleRequest assumeRequest = new AssumeRoleRequest().withRoleArn(roleArn).withDurationSeconds(3600)
.withRoleSessionName("Test");
AssumeRoleResult roleResponse = stsClient.assumeRole(assumeRequest);
Credentials sessionCredentials = roleResponse.getCredentials();
BasicSessionCredentials awsCredentials = new BasicSessionCredentials(
sessionCredentials.getAccessKeyId(),
sessionCredentials.getSecretAccessKey(),
sessionCredentials.getSessionToken());
AmazonS3Client s3ClientRole = (AmazonS3Client) AmazonS3ClientBuilder.standard()
.withCredentials(new AWSStaticCredentialsProvider(awsCredentials))
.withRegion(Regions.fromName(region))
.build();
AmazonS3Client s3Client = s3ClientRole;
return s3Client;
}
}
If on the line where I create the awsCreds object, I fill it correctly with the values I get from the application.properties, if it does everything correctly. But if I leave it empty I never get to execute the line stsClient.assumeRole(assumeRequest); and it gives me connection time out error.
I think you should follow this doc: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/java-dg-roles.html#default-provider-chain
It should follow you step bu step on resolving you problem.
Basically an IAM Role allows a specific "resource", such as an EC2 instance, to connect to the service and thus you configure your client to get temporary credentials just to "authenticate" and "authorize" it.
I've created a personal repository on Git where I have kept my application.properties file.
I've created a cloud config server ('my-config-server') and used the git repository url.
I have bound my spring-boot application that is supposed to access the external properties file with Git repository.
#javax.jws.WebService(
serviceName = "myService",
portName = "my_service",
targetNamespace = "urn://vdc.com/xmlmessaging/SD",
wsdlLocation = "classpath:myService.wsdl",
endpointInterface = "com.my.service.SDType")
#ConfigurationProperties
public class SDTypeImpl implements SDType {
/*It has various services implementation that use following method**/
private SDObj getObj (BigDecimal value) {
AnnotationConfigApplicationContext context =
new AnnotationConfigApplicationContext(
SDTypeImpl.class);
SDObj obj = context.getBean(SDPropertiesUtil.class).getObj(value);
context.close();
return obj;
}
}
Another Class:
public class SDPropertiesUtil {
#Autowired
public Environment env;
public SDObj getObj(BigDecimal value) {
String valueStr = env.getProperty(value.toString());
/*do logic*/
}
But, I get 'valueStr' as null.
My application starts and the config server on the cloud loads properties file from my git repository.
From the cloud logs, after i pushed my application:
PropertySourceBootstrapConfiguration : Located property source: CompositePropertySource [name='configService', propertySources=[MapPropertySource [name='configClient'], MapPropertySource [name='ssh://blah blah blah.git/application.properties']]]
My application is unable to access an external properties file through spring cloud config server.
I'm using Spring Cloud Config in my spring-boot application and I need to write some custom code to handle properties to be read from my corporate password vault when property is flagged as such. I know spring cloud supports Hashicorp Vault, but that's not the one in case.
I don't want to hard-code specific properties to be retrieved from a different source, for example, I would have a properties file for application app1 with profile dev with values:
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=dbuser
spring.datasource.password=dbpass
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
but for some other profiles such as prod, I would have:
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=prod-user
spring.datasource.password=[[vault]]
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
So I need the custom property vault to intercept the property loaded whenever it finds a returned value equals to [[vault]] (or some other type of flag), and query from the corporate vault instead. In this case, my custom property loader would find the value of property spring.datasource.password from the corporate password vault. All other properties would still be returned as-is from values loaded by standard spring cloud config client.
I would like to do that using annotated code only, no XML configuration.
You can implement your own PropertySourceLocator and add entry to
spring.factories in directory META-INF.
#spring.factories
org.springframework.cloud.bootstrap.BootstrapConfiguration=/
foo.bar.MyPropertySourceLocator
Then you can you can refer to keys in your corporate password vault like a normal properties in spring.
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=prod-user
spring.datasource.password=${lodaded.password.from.corporate.vault}
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
Implementation by HasiCorp: VaultPropertySourceLocatorSupport
While trying to solve the identical problem, I believe that I have come to work-around that may be acceptable.
Here is my solution below.
public class JBossVaultEnvironmentPostProcessor implements EnvironmentPostProcessor {
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
MutablePropertySources propertySources = environment.getPropertySources();
Map<String, String> sensitiveProperties = propertySources.stream()
.filter(propertySource -> propertySource instanceof EnumerablePropertySource)
.map(propertySource -> (EnumerablePropertySource<?>) propertySource)
.map(propertySource -> {
Map<String, String> vaultProperties = new HashMap<>();
String[] propertyNames = propertySource.getPropertyNames();
for (String propertyName : propertyNames) {
String propertyValue = propertySource.getProperty(propertyName).toString();
if (propertyValue.startsWith("VAULT::")) {
vaultProperties.put(propertyName, propertyValue);
}
}
return vaultProperties;
})
.reduce(new HashMap<>(), (m1, m2) -> {
m1.putAll(m2);
return m1;
});
Map<String, Object> vaultProperties = new HashMap<>();
sensitiveProperties.keySet().stream()
.forEach(key -> {
vaultProperties.put(key, VaultReader.readAttributeValue(sensitiveProperties.get(key)));
});
propertySources.addFirst(new MapPropertySource("vaultProperties", vaultProperties));
}
Being new to Spring Boot I am wondering on how I can configure connection details for MongoDB.
I have tried the normal examples but none covers the connection details.
I want to specify the database that is going to be used and the url/port of the host that runs MongoDB.
Any hints or tips?
Just to quote Boot Docs:
You can set spring.data.mongodb.uri property to change the url, or alternatively specify a host/port. For example, you might declare the following in your application.properties:
spring.data.mongodb.host=mongoserver
spring.data.mongodb.port=27017
All available options for spring.data.mongodb prefix are fields of MongoProperties:
private String host;
private int port = DBPort.PORT;
private String uri = "mongodb://localhost/test";
private String database;
private String gridFsDatabase;
private String username;
private char[] password;
It's also important to note that MongoDB has the concept of "authentication database", which can be different than the database you are connecting to. For example, if you use the official Docker image for Mongo and specify the environment variables MONGO_INITDB_ROOT_USERNAME and MONGO_INITDB_ROOT_PASSWORD, a user will be created on 'admin' database, which is probably not the database you want to use. In this case, you should specify parameters accordingly on your application.properties file using:
spring.data.mongodb.host=127.0.0.1
spring.data.mongodb.port=27017
spring.data.mongodb.authentication-database=admin
spring.data.mongodb.username=<username specified on MONGO_INITDB_ROOT_USERNAME>
spring.data.mongodb.password=<password specified on MONGO_INITDB_ROOT_PASSWORD>
spring.data.mongodb.database=<the db you want to use>
spring.data.mongodb.host and spring.data.mongodb.port are not supported if you’re using the Mongo 3.0 Java driver. In such cases, spring.data.mongodb.uri should be used to provide all of the configuration, like this:
spring.data.mongodb.uri=mongodb://user:secret#mongo1.example.com:12345
In a maven project create a file src/main/resources/application.yml with the following content:
spring.profiles: integration
# use local or embedded mongodb at localhost:27017
---
spring.profiles: production
spring.data.mongodb.uri: mongodb://<user>:<passwd>#<host>:<port>/<dbname>
Spring Boot will automatically use this file to configure your application. Then you can start your spring boot application either with the integration profile (and use your local MongoDB)
java -jar -Dspring.profiles.active=integration your-app.jar
or with the production profile (and use your production MongoDB)
java -jar -Dspring.profiles.active=production your-app.jar
You can define more details by extending AbstractMongoConfiguration.
#Configuration
#EnableMongoRepositories("demo.mongo.model")
public class SpringMongoConfig extends AbstractMongoConfiguration {
#Value("${spring.profiles.active}")
private String profileActive;
#Value("${spring.application.name}")
private String proAppName;
#Value("${spring.data.mongodb.host}")
private String mongoHost;
#Value("${spring.data.mongodb.port}")
private String mongoPort;
#Value("${spring.data.mongodb.database}")
private String mongoDB;
#Override
public MongoMappingContext mongoMappingContext()
throws ClassNotFoundException {
// TODO Auto-generated method stub
return super.mongoMappingContext();
}
#Override
#Bean
public Mongo mongo() throws Exception {
return new MongoClient(mongoHost + ":" + mongoPort);
}
#Override
protected String getDatabaseName() {
// TODO Auto-generated method stub
return mongoDB;
}
}
In case that somebody is trying to connect to a Atlas MongoDB Cluster in application.properties has to have a config like:
spring.data.mongodb.uri=mongodb+srv://databaseUsername:usernamePassword#cluster0.j4koa.mongodb.net/databaseUsername?retryWrites=true&w=majority
In my case I needed to set up MongoDB for integration tests using Testcontainers. Using properites file was not an option since Mongo port had to be specified during runtime. I wanted to preseve original MongoDB autoconfiguration provided by SpringBoot but override some of the properties. This can be achieved by defining a bean of type MongoClientSettingsBuilderCustomizer which can be used to customize mongo settings :
#Bean
public MongoClientSettingsBuilderCustomizer clientSettingsBuilderCustomizer(final GenericContainer<?> mongoDBContainer) {
String database = environment.getProperty("spring.data.mongodb.database");
ConnectionString connectionString = new ConnectionString(String.format("mongodb://localhost:%s/%s", mongoDBContainer.getFirstMappedPort(), database));
return settings -> settings.applyConnectionString(connectionString);
}
If you simply want to read a MongoDB connection string from an environment variable, one way is to set the following environment variable:
SPRING_DATA_MONGODB_URI=mongodb://localhost:27017/trying-mongo
This doesn't require any changes in the application.properties as the spring data mongo will read the value from the above environment variable by default.
Here is How you can do in Spring Boot 2.0 by creating custom MongoClient adding Providing more control for Connection ,
Please follow github Link for Full Source Code
#Configuration
#EnableMongoRepositories(basePackages = { "com.frugalis.repository" })
#ComponentScan(basePackages = { "com.frugalis.*" })
#PropertySource("classpath:application.properties")
public class MongoJPAConfig extends AbstractMongoConfiguration {
#Value("${com.frugalis.mongo.database}")
private String database;
#Value("${com.frugalis.mongo.server}")
private String host;
#Value("${com.frugalis.mongo.port}")
private String port;
#Value("${com.frugalis.mongo.username}")
private String username;
#Value("${com.frugalis.mongo.password}")
private String password;
#Override
protected String getDatabaseName() {
return database;
}
#Override
protected String getMappingBasePackage() {
return "com.frugalis.entity.mongo";
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongoClient(), getDatabaseName());
}
#Override
#Bean
public MongoClient mongoClient() {
List<MongoCredential> allCred = new ArrayList<MongoCredential>();
System.out.println("???????????????????"+username+" "+database+" "+password+" "+host+" "+port);
allCred.add(MongoCredential.createCredential(username, database, password.toCharArray()));
MongoClient client = new MongoClient((new ServerAddress(host, Integer.parseInt(port))), allCred);
client.setWriteConcern(WriteConcern.ACKNOWLEDGED);
return client;
}}