List final list of properties - Spring Cloud Config Server - spring-boot

Spring Cloud Config Server accepts multiple profile and returns the properties for all the profiles when I access the /env endpoint of the application. The response lists the properties specific to each profile. If same property is present in 2 different property files , the one that is defined last takes precedence. Is there a way to get the final list of property key and values that will be used by the application?

For Cloud Config Client Application
I've tried different ways and found the following (accidentally):
GET /env/.* returns full list of configuration properties
For Cloud Config Server Application
It turns out this is already implemented, but not documented well. All you need is to request json, yml or properties according to the patterns:
/{application}-{profile}.{ext}
/{label}/{application}-{profile}.{ext}

import java.util.properties;
import org.springframework.core.env.AbstractEnvironment;
import org.springframework.core.env.CompositePropertySource;
import org.springframework.core.env.Environment;
public class MyClass {
#Autowired
private Environment env;
Properties getProperties() {
Properties props = new Properties();
CompositePropertySource bootstrapProperties = (CompositePropertySource) ((AbstractEnvironment) env).getPropertySources().get("bootstrapProperties");
for (String propertyName : bootstrapProperties.getPropertyNames()) {
props.put(propertyName, bootstrapProperties.getProperty(propertyName));
}
return props;
}
}
Sorry... this is my first time answering a question here. I created an account specifically to
answer this question because I came upon it while researching the same issue. I found a
solution that worked for me and decided to share it.
Here goes my explanation of what was done:
I initialize a new "Properties" object (could be a HashMap or whatever else you want)
I lookup the property source for the "bootstrapProperties" which is a CompositePropertySource object.
This property source contains all of the application properties that were loaded.
I loop through all the property names returned from the "getPropertyNames" method on the CompositePropertySource object
and create a new property entry.
I return the properties object.

This seems to be an intentional limitation of the Spring Framework.
See here
You could hack it and inject the PropertySources interface, then loop over all the individual PropertySource objects, but you'd have to know what properties you're looking for.

Externalized Configuration
Spring Boot allows you to externalize your configuration so you can work with the same application code in different environments. You can use properties files, YAML files, environment variables and command-line arguments to externalize configuration. Property values can be injected directly into your beans using the #Value annotation, accessed via Spring’s Environment abstraction or bound to structured objects via #ConfigurationProperties.
Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:
Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
#TestPropertySource annotations on your tests.
#SpringBootTest#properties annotation attribute on your tests.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that only has properties in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
#PropertySource annotations on your #Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).
The below program prints properties from spring boot environment.
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ApplicationObjectSupport;
import org.springframework.core.env.Environment;
import org.springframework.core.env.MapPropertySource;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.core.env.PropertySource;
import org.springframework.stereotype.Component;
import org.springframework.web.context.support.StandardServletEnvironment;
#Component
public class EnvironmentLogger extends ApplicationObjectSupport {
#Override
protected void initApplicationContext(ApplicationContext context) throws BeansException {
Environment environment = context.getEnvironment();
String[] profiles = environment.getActiveProfiles();
if(profiles != null && profiles.length > 0) {
for (String profile : profiles) {
System.out.print(profile);
}
} else {
System.out.println("Setting default profile");
}
//Print the profile properties
if(environment != null && environment instanceof StandardServletEnvironment) {
StandardServletEnvironment env = (StandardServletEnvironment)environment;
MutablePropertySources mutablePropertySources = env.getPropertySources();
if(mutablePropertySources != null) {
for (PropertySource<?> propertySource : mutablePropertySources) {
if(propertySource instanceof MapPropertySource) {
MapPropertySource mapPropertySource = (MapPropertySource)propertySource;
if(mapPropertySource.getPropertyNames() != null) {
System.out.println(propertySource.getName());
String[] propertyNames = mapPropertySource.getPropertyNames();
for (String propertyName : propertyNames) {
Object val = mapPropertySource.getProperty(propertyName);
System.out.print(propertyName);
System.out.print(" = " + val);
}
}
}
}
}
}
}
}

Related

Is it possible to have application.properties depend on multiple profiles?

Is it possible to have a Spring Boot properties file depend on two or more profiles? Something like application-profile1-profile2.properties?
Spring Boot does not support this out of the box. It only supports a single profile as described here.
However, it does provide enough flexibility to add your own property sources using EnvironmentPostProcessor.
Here is an example of how to implement this:
public class MultiProfileEnvironmentPostProcessor implements EnvironmentPostProcessor, Ordered {
private final ResourceLoader resourceLoader = new DefaultResourceLoader();
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
String[] activeProfiles = environment.getActiveProfiles();
for (int i = 2; i <= activeProfiles.length; i++) {
Generator.combination(activeProfiles).simple(i)
.forEach(profileCombination -> {
String propertySourceName = String.join("-", profileCombination);
String location = "classpath:/application-" + propertySourceName + ".properties";
if (resourceLoader.getResource(location).exists()) {
try {
environment.getPropertySources().addFirst(new ResourcePropertySource(propertySourceName, location));
} catch (IOException e) {
throw new RuntimeException("could not add property source '" + propertySourceName + "'", e);
}
}
});
}
}
#Override
public int getOrder() {
return Ordered.LOWEST_PRECEDENCE;
}
}
Couple of things to note:
This implementation only supports .properties files but can easily be extended to .yml files as well.
getActiveProfiles already returns the profiles in an order where the last one wins. This implementation relies on this order and builds the different file names leveraging this order. i.e. if active profiles are: profile1,profile2,profile3 then application-profile1-profile3.properties is supported but application-profile3-profile1.properties isn't, and application-profile1-profile3.properties will override properties defined in application-profile1.properties or application-profile3.properties.
This implementation uses a third party library com.github.dpaukov:combinatoricslib3 to create the different sets of profiles.
The property sources are added to the front of the property source list to override existing sources. But if you have custom property sources that should take precedence you need to modify this a bit to consider them in the order, i.e. by leveraging methods like Environment.addAfter.
Registering an EnvironmentPostProcessor is done using the spring.factories file.
There are 4 ways I know.
insert .yaml or .properties programmatically like Asi Bross Said. Use ResourceLoader or YamlPropertySourceLoader to insert.
Use .yaml. but it will be replace when you have a another spring project to dependent it.
Use properties instead of profiles. (For api project)
Use one #PropertySource to define properties file A.
Get the variables from properties file A and assign them to the parameters in another #PropertySource file path expression.
For example:
resources
/-application.properties <-- remove or empty,because it will be override by application project
/-moduleA
/-application.properties <-- Intellij can identify properties files started with application-
/-application-mysql-dev.properties
/-application-psql-dev.properties
/-application-psql-prod.properties
The content of resources/moduleA/application.properties :
moduleA.custom.profile1=mysql
moduleA.custom.profile2=dev
Content of Java Config file:
#SpringBootApplication
#PropertySources({
#PropertySource("/moduleA/application.properties"),
#PropertySource("/moduleA/application-${moduleA.custom.profile1}-${moduleA.custom.profile2}.properties"),
})
public class ModuleConfig {}
Use properties instead of profiles. (For application project)
resources
/-application.properties
/-application-mysql-dev.properties
/-application-psql-dev.properties
/-application-psql-prod.properties
The content of resources/application.properties :
moduleA.custom.profile1=mysql
moduleA.custom.profile2=dev
The content of SpringMvcApplication.java:
#SpringBootApplication
#PropertySource("/application-${moduleA.custom.profile1}-${moduleA.custom.profile2}.properties")
public class SpringMvcApplication {...}

Custom PropertyPlaceholderConfigurer not resolving

Hi I am new to Spring Boot (but have been using Spring in my apps for a while). I am trying to use a custom SSM PropertyPlaceholderConfigurer, based on my SSM Client, that reads my properties from AWS SSM, in addition to properties in my normal application.properties.
This code works fine in my pre-spring-boot application. However, in the new application, I see that it overrides the application.properties. And this seems to be a well documented problem.
So i decided to include the application.properties file in my custom PropertyPlaceholderConfigurer class and load all the properties together and still it does not resolve any properties in application.properties that are marked with "${}" and resolve by my custom location. What more do i need to do?
As an alternative, I tried to have the properties i need to load from SSM to be loaded via an EnvironmentPostProcessor but it was unable to connect to the AWS SSM server at this point in the loading process (not sure why)
The answer is to use the EnvironmentPostProcessor. Works perfectly. See code below:
import com.amazonaws.auth.DefaultAWSCredentialsProviderChain;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MapPropertySource;
import java.util.HashMap;
import java.util.Map;
/**
* This class loads SSM parameters base on region and environment
* Needs to be added to spring.factories class so that it will be invoked. as follow:
* org.springframework.boot.env.EnvironmentPostProcessor=<full package>.SSMEnvironmentPostProcessor
* Add the SSM propeties to other properties already set
*/
public class SSMEnvironmentPostProcessor implements EnvironmentPostProcessor {
private static final String QUOTE = "\"";
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
SSMClient ssmClient = new SSMClient(DefaultAWSCredentialsProviderChain.getInstance(), System.getProperty("env" +
".region"), new ClientConfiguration());
ssmClient.init();
Map<String, Object> parameters = new HashMap<>();
ssmClient.getParametersByPath("/" + System.getProperty("env"), true).entrySet().stream()
.forEach(entry -> parameters.put(entry.getKey(), entry.getValue()));
MapPropertySource mapPropertySource = new MapPropertySource("ssm", parameters);
environment.getPropertySources().addLast(mapPropertySource);
}
}

Reading applicatiion.properties once in a singleton class

I have a singleton configuration class where I want to store all the properties for our web application.
How do we read in the application.properies file like any other properties file without using annotations?
What is the the fully qualified filename for application.properies i.e. /application.properies?
We only want to read application.properties once.
Spring boot already reads all the properties stored in application.properties and much more, read Externalized Configuration documentation.
If you want to map one property named server.port you can just use #Value("${server.port}") Integer port.
If you want to access to all the properties loaded by Spring Boot, you can use the Environment object and access to all loaded PropertySources and retrieve all values from each property source.
In this this answer shows how. However, to avoid losing the precedence order of loaded properties, you have to reverse the property source list. Here you can find the code to load all the properties without losing the spring precedence order:
#Configuration
public class AppConfiguration {
#Autowired
Environment env;
public void loadProperties() {
Map<String, Object> map = new HashMap();
for (Iterator it = ((AbstractEnvironment) env).getPropertySources().iterator().reverse(); it.hasNext(); ) {
PropertySource propertySource = (PropertySource) it.next();
if (propertySource instanceof MapPropertySource) {
map.putAll(((MapPropertySource) propertySource).getSource());
}
}
}
}

application.properties spring boot value injection

I'm working on the REST API with spring boot. I want to use git in my project. in the file application.properties I have the database Url, username and password that I do not want to push on git. I don't know how can I create a file which contains my database configuration and how to inject those configurations in the application.properties .
application.properties
## Server Properties
server.port= 5000
## Spring DATASOURCE (DataSourceAutoConfiguration & DataSourceProperties)
spring.datasource.url= jdbc:mysql://localhost:3306/MyApp?useSSL=false&serverTimezone=UTC&useLegacyDatetimeCode=false
spring.datasource.username= user
spring.datasource.password= pass
Spring picks up configuration properties not only from the application.properties but also from command line arguments, JAVA System-properties or from environmental-variables.
See complete list here: Spring Externalized Configuration.
So - for reference - you can keep the properties in the application.properties file with some default values (like in your example) in order to let other users know what kind of properties they can set for your application.
But instead of setting your real values there, you can either pass the variable to your application as arguments, like
-Dspring.datasource.username=user -Dspring.datasource.password= pass
or you can set them as environmental variables.
You can even create multiple configuration with different settings. If Spring cannot find a variable in the current configuration, then it will pick it up from application.properties (or from the other sources - see above)
first you should add application.properties to .ignore file like this
application.properties
if you will just connect to database you won't need to inject values by hand you just write it in application.properties
but if you want to put values in properties file and use it in Application
package com.microservice.test.limitservice;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
#Component
#ConfigurationProperties("limit-service")
public class Configuration {
private int minimum;
private int maximum;
public int getMinimum() {
return minimum;
}
public void setMinimum(int minimum) {
this.minimum = minimum;
}
public int getMaximum() {
return maximum;
}
public void setMaximum(int maximum) {
this.maximum = maximum;
}
}
and how to inject it simply
#Autowired
private Configuration configuration;
the application.properties file could be like this
limit-service.minimum=56333445
limit-service.maximum=6500
you should notice that it start with as example limit-service
and #ConfigurationProperties("**limit-service**")
And if you want to store your configuration in application.properties secure
you can see this link Spring Boot how to hide passwords in properties file

Spring cloud config server. Environment variables in properties

I configured Spring Cloud Config server like this:
#SpringBootApplication
#EnableAutoConfiguration
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
I'm using 'native' profile so properties are picked up from the file system:
server.port=8888
spring.profiles.active=native
spring.cloud.config.server.native.search-locations: classpath:/global
Now the tricky part is that some properties contain environmental variables. Properties in 'global/application-production.properties' are configured like this:
test=${DOCKER_HOST}
When I start up Config Server - everything works fine. However when I access http://localhost:8888/testapp/production I see this:
{
name: "testapp",
profiles: [
"production"
],
label: null,
version: null,
propertySources: [
{
name: "classpath:/global/application-production.properties",
source: {
test: "${DOCKER_HOST}"
}
}
]
}
So value from ENV variable is not replacing ${DOCKER_HOST} put rather returned as is.
But if I access http://localhost:8888/application-production.properties then result is non JSON but rather plain text:
test: tcp://192.168.99.100:2376
Spring documentation says:
The YAML and properties representations have an additional flag (provided as a boolean query parameter resolvePlaceholders) to signal that placeholders in the source documents, in the standard Spring ${…​} form, should be resolved in the output where possible before rendering. This is a useful feature for consumers that don’t know about the Spring placeholder conventions.
For some reason resolvePlaceholders is not working for JSON representation thus server config clients need to be aware of all ENV variables configured on server.
Is it possible to force JSON representation resolvePlaceholders same way as plain text (properties) representation?
I faced the same issue. After looking into Spring Cloud Config Repository I have found the following commit:
Omit system properties and env vars from placeholders in config
It looks like such behavior is not supported.
You can try the Property Overrides feature to override properties from git Environment Repository.
To override property foo at runtime, just set a system property or an environment variable spring.cloud.config.server.overrides.foo before starting the config server.
There was an update in order to accomplish this, in the following merge. 1 I found an implementation for resolvePlaceholders. Which gave me the idea of just creating a new controller which uses the EnvironmentController. This will allow you to resolve configuration, this is a good bootstrap.
import com.fasterxml.jackson.databind.ObjectMapper;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cloud.config.server.environment.EnvironmentController;
import org.springframework.cloud.config.server.environment.EnvironmentRepository;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
#RestController
#RequestMapping(method = RequestMethod.GET, path = "resolved/${spring.cloud.config.server.prefix:}")
public class ReplacedEnvironmentController {
private EnvironmentController environmentController;
#Autowired
public ReplacedEnvironmentController(EnvironmentRepository repository) {
environmentController = new EnvironmentController(repository, new ObjectMapper());
}
public ReplacedEnvironmentController(EnvironmentRepository repository,
ObjectMapper objectMapper) {
environmentController = new EnvironmentController(repository, objectMapper);
}
#RequestMapping("/{name}/{profiles:.*[^-].*}")
public ResponseEntity<String> resolvedDefaultLabel(#PathVariable String name,
#PathVariable String profiles) throws Exception {
return resolvedLabelled(name, profiles, null);
}
#RequestMapping("/{name}/{profiles}/{label:.*}")
public ResponseEntity<String> resolvedLabelled(#PathVariable String name, #PathVariable String profiles,
#PathVariable String label) throws Exception {
return environmentController.labelledJsonProperties(name, profiles, label, true);
}
}

Resources