In our company we have 100+ services built using Spring Boot. Most of these services share configuration values that allow us to configure how these applications register on Eureka, consume from Kafka and so on. This configuration is specified on the application.yml file in most of the services, and we'd like to avoid repeating these values, so we want to move this common configuration to a dependency that, when added to the classpath, will make that you no longer have to configure those values.
I achieved it by creating a new library which contains different #Configuration classes that configure the different beans with the right values. So now applications just need to add this library to the build.gradle file, and things are configured.
The problem is that, if one of the applications need to override any of the values for those beans, specifying the configuration in their application.yml file won't work, because I'm directly setting the right values to the bean.
For example, if my library declares this Bean
#Bean
#Profile("aws")
public EurekaInstanceConfigBean eurekaInstanceConfigAWS(InetUtils inetUtils) {
EurekaInstanceConfigBean eurekaInstanceConfigBean = new EurekaInstanceConfigBean(inetUtils);
AmazonInfo info = AmazonInfo.Builder.newBuilder().autoBuild("eureka");
eurekaInstanceConfigBean.setDataCenterInfo(info);
eurekaInstanceConfigBean.setInstanceId(info.get(AmazonInfo.MetaDataKey.instanceId));
eurekaInstanceConfigBean.setHostname(info.get(AmazonInfo.MetaDataKey.localHostname));
eurekaInstanceConfigBean.setIpAddress(info.get(AmazonInfo.MetaDataKey.localIpv4));
return eurekaInstanceConfigBean;
}
And someone specifies on the application.yml that they want a different hostname like
eureka:
instance:
hostname: some-custom-value.example.com
It will be ignored: the configuration set in the bean will be used, not the local application.yml values.
Is there a way to achieve an order of precedence here where values specified on application.yml are more important than the values on the bean coming from the library? I'd like a precedence like:
local application.yml > bean config in library > bean defaults coming from spring-boot/spring-cloud
Related
I use WAS Liberty Server container which provides server.xml and server.env file for configuring many things in addition to configuring DataSource properties such as url, username, password etc.
For security reasons, these properties are not known to developers for production Liberty servers. However, developers use same server.xml/server.evn files but with different DataSource properties so they can do their work.
So, I might have two server.env files like:
PRODUCTION server.env: dataSourceUrl="server-A-URL" (this URL is not known to developers)
DEVELOPMENT server.env: dataSourceUrl="server-B-URL" (this URL is known to developers)
, then the dataSourceUrl is used in server.xml files in production and development to set the url accordingly.
So, the structure of server.xml/server.env file is same for developers and production, only the DataSource url, username, password are different. This way developers can work using their own DataSource properties and once ready to deploy they app, it is handed to other team which then just changes the DataSource properties to the production values and deploys the application to production server.
With Springboot JPA, I know we can use application.properties file to set these DataSource values. But, we would like to be able to set these to the values located in server.env file. Basically to have something like this in application.properties file:
spring.datasource.url=dataSourceUrl //dataSourceUrl is set in server.env
, then be able to use this value in Java code using #Value injection like:
public class MyClass {
#Value("${spring.datasource.url}")
String dsUrl;
...
}
I have been reading about externalizing properties but I am not able to figure out how to do this
You can use Liberty's jndiEntry elements to make configured values available in JNDI. You will need the jndi-1.0 feature, after which you can configure,
<jndiEntry jndiName="spring/datasource/url" value="${dataSourceUrl}"/>
and access it in your application as:
String dsUrl = InitialContext.doLookup("spring/datasource/url");
Or, from a web or ejb component as:
#Resource(lookup = "spring/datasource/url")
String dsUrl;
We are using spring boot 2.0.0. We have three environments dev, staging, production. Our current config structure
dev
application-dev.yml
application-dev.properties
Likewise, we have a yml and properties file for each environment. After a year of development now the single yml file for a profile become a large monolithic config.
is it possible to have a multiple config files for a profile like below?
application-dev.yml
application-dev-sqs.yml
application-dev-redis.yml
I think there are 2 ways you can achieve this requirement.
spring.profiles.active accepts a comma-separated list of active profiles, so you can always provide dev,dev-sqs,dev-redis as the value.
Another approach is by making use of #PropertySource and a custom PropertySourceFactory to achieve this requirement. You can find an implementation which takes the value from spring.profiles.active to load one corresponding YAML file in the article below. It should be super easy to adapt the implementation to load multiple files by looking for the profile id in the name of the YAML files.
[How-to] Read profile-based YAML configurations with #PropertySource
I was dealing with a similar problem and I'd recommend using yaml configuration.
Let's describe .properties file:
Initital approach
One can use it like this:
#Component
#PropertySources({
#PropertySource("classpath:application.properties"),
#PropertySource("classpath:application-${spring.profiles.active}.properties")
})
public class AppProperties {
}
This is very easy to configure. Limitation is, that you cannot combine profiles. I mean, that when you want to use profile as dev,local where local just alters some config properties for dev profile, Spring will try to load application-dev,local.properties file, which is very likely not what you want.
Btw, this is what Spring will do for you automatically, this is useful for topics as you described.
There is no way to configure it per profile (and not for whole list). Other possibility would be, that one can specify the list in spring.config.name which is not the case at the moment.
Better approach
In short, use:
#Profile("dev")
#Configuration
#PropertySources({
#PropertySource("classpath:topic1-dev.properties"),
#PropertySource("classpath:topic2-dev.properties")
})
public class AppPropertiesDev {
}
Disadvantage is, you have to have several such config classes (dev, staging), but know you have the topics. Also you can use mutliple profiles, which are (as of my testing) loaded in order you specified. That way, your developer can easily use dev configuration and alter just what's needed for his/her testing.
Yaml approach
You can see the approach with yaml in question I asked earlier - Property resolving for multiple Spring profiles (yaml configuration), benefit is smaller amount of files - yaml has all the profiles in one file, which may or may not be what you want.
Yes, it's possible. spring.config.location is used to externalize the config file location in Spring boot applications. This can be used to provide a location of the file in the filesystem or even in the classpath. Based on how you want to provide your application access to the files, you can choose the URI.
Doing it programmatically:
#SpringBootApplication
public class Application {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = new SpringApplicationBuilder(Application.class)
.properties("spring.config.location:classpath:/application-dev.yml,classpath:/application-dev-sqs.yml,classpath:/application-dev-redis.yml")
.build()
.run(args);
}
}
Doing it via environment variables:
set SPRING_CONFIG_LOCATION=classpath:/application-dev.yml, \
classpath:/application-dev-sqs.yml, \
classpath:/application-dev-redis.yml
So, you can provide your files as comma-separated values.
I've used classpath here, it can also be a location in the file system:
/home/springboot-app/properties/application-dev.yml,/home/springboot-app/properties/application-sqs.yml,/home/springboot-app/properties/application-redis.yml
Have you tried including profiles yet ?
Example with profile default, you want to load additional properties for redis and db. Within application.properties file, add:
spring.profiles.include=redis, db
This will load files application-redis.properties and application-db.properties respectively
I want to configure the properties of the Tomcat JDBC Pool with custom parameter values. The pool is bootstrapped by the spring-cloud (Spring Cloud Connector) environment (Cloud Foundry) and connected to a PostgreSQL database. In particular, I want to set the minIdle, maxIdle and initialSize properties for the given pool.
In a "spring-vanilla" environment (non-cloud) the properties can be easily set by using
application.properties / .yaml files with environment properties,
#ConfigurationProperties annotation.
However, this approach doesn't transfer to my Cloud environment, where the URL (and other parameters) are injected from the environment variable VCAP_SERVICES (via the ServiceInfo instances). I don't want to re-implement the logic which Spring Cloud already did with its connectors.
After some searching I also stumbled over some tutorials / guides, which suggest to make use of the PoolConfig object (e.g. http://cloud.spring.io/spring-cloud-connectors/spring-cloud-spring-service-connector.html#_relational_database_db2_mysql_oracle_postgresql_sql_server). However, that way one cannot set the properties I need but merely the following three:
minPoolSize,
maxPoolSize,
maxWaitTime.
Note that I don't want to set connection-related properties (such as charset), but the properties are associated with the pool itself.
In essence, I would like to do the configuration similarly to https://www.baeldung.com/spring-boot-tomcat-connection-pool (using spring.datasource.tomcat.* properties). The problem with that approach is that the properties are not considered if the datasource was created by Spring Cloud. The article https://dzone.com/articles/binding-data-services-spring, section "Using a CloudFactory to create a DataSource", claims that the following code snippet makes it so that the configuration "can be tweaked using application.properties via spring.datasource.* properties":
#Bean
#ConfigurationProperties(DataSourceProperties.PREFIX)
public DataSource dataSource() {
return cloud().getSingletonServiceConnector(DataSource.class, null);
}
However, my own local test (with spring-cloud:Greenwich.RELEASE and spring-boot-starter-parent:2.1.3.RELEASE) showed that those property values are simply ignored.
I found an ugly way to solve my problem but I think it's not appropriate:
Let spring-cloud create the DataSource, which is not the pooled DataSource directly,
check that the reference is a descendant of a DelegatingDataSource,
resolve the delegate, which is then the pool itself,
change the properties programmatically directly at the pool itself.
I do not believe that this is the right way as I am using internal knowledge (on the layering of datasources). Additionally, this approach does not work for the property initialSize, which is only considered when the pool is created.
At the project with Spring Boot we use application.properties but need to configure some of these properties (like port number of logging level) based on an external configuration. We access the configuration via API so it is known only at runtime.
Is there a way to override or set some Spring properties at runtime (for example using a bean) and if yes how can this be achieved?
You could do this with Spring Cloud Config
Just for the purpose of illustration, here's a relatively quick way to see dynamic property overrides at runtime:
First, for your bean to be able to pick up changed properties, you need to annotate it with
#RefreshScope
Add the spring cloud dependency to your spring boot app, eg for gradle
compile group: 'org.springframework.cloud', name: 'spring-cloud-starter', version: '1.1.1.RELEASE'
( NB You also need the spring boot actuator dependency.)
With the app running, you can view your current config at eg
http://localhost:8080/env
eg if you have a property 'my.property' in application.properties, you'll see something like:
"applicationConfig: [classpath:/application.properties]": {
"my.property": "value1",
etc
To change the value, POST my.property=value2 to /env as application/x-www-form-urlencoded
eg
curl -X POST http://localhost:8080 -d my.property=value2
GET /env again and you'll see the new value appears under the "manager" section
To apply the changed properties, do an empty POST to /refresh. Now your bean will have the new value.
Could you use system properties to pass in the variable? If you configure the PropertyPlaceholderConfigurer you can set the precedence of system properties vs file properties.
For example, something like:
#Bean public PropertyPlaceholderConfigurer placeHolderConfigurer() {
PropertyPlaceholderConfigurer props = new PropertyPlaceholderConfigurer()
props.setSystemPropertiesMode( PropertyPlaceholderConfigurer.SYSTEM_PROPERTIES_MODE_OVERRIDE )
props.setLocations(new
PathMatchingResourcePatternResolver().getResources("classpath:/**.properties"));
props
}
The above would load your .properties file, but we set the priority to be system variables first, so if you set a system variable that will override the same variable in the config.
Alternatively, looking at the docs, Spring recommends defining a search order in your Environment:
[PropertyPlaceholderConfigurer is still appropriate for use when]
existing configuration makes use of the "systemPropertiesMode" and/or "systemPropertiesModeName" properties. Users are encouraged to
move away from using these settings, and rather configure property
source search order through the container's Environment; however,
exact preservation of functionality may be maintained by continuing to
use PropertyPlaceholderConfigurer.
Hopefully one of the above should sort out what you need?
I'm using Sprint Boot, and would like to have multiple profile specific property files. The docs state:
In addition to application.properties files, profile specific
properties can also be defined using the naming convention
application-{profile}.properties.
http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-external-config-profile-specific-properties
However I have multiple properties files (e.g. db.properties). I'm loading currently load this non-profile specific file as:
#Configuration
#PropertySource( {"classpath:db.properties"} )
class DataSourceConfig {
#Value("db.server") String server;
...
}
How can I combine these two things together, so it loads db-dev.properties like Spring Boot does for application.properties
It sounds like it should be easy, but I can't work out how to do it?!
Java -jar my-spring-boot.jar --spring.profiles.active=test you can set profile.active=your environment via commandline
I just saw that you use #PropertySource. The docs say:
Profile specific variants of both application.properties (or application.yml) and files referenced via #ConfigurationProperties are considered as files are loaded.