My intention is to have two profiles in a spring boot application - development and production one. Development profile is meant just to override some variables of production profile (like in-memory database instead of database in the cloud). As I expect some changes to be done to production profile in the future, duplicating variables in development profile doesn't seem to be a solution.
So, in Spring Reference I read that spring.profiles.include is supposed to only add properties from referenced profile.
Sometimes, it is useful to have profile-specific properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to unconditionally add active profiles.
However, from what I've checked it rather overrides it. So, when having two profiles foo and bar, in separate yaml files:
application-foo.yaml:
myproperty: 44
application-bar.yaml:
spring:
profiles:
include: foo
active: bar,foo
myproperty: 55
And setting -Dspring.profiles.active=bar variable in IDE, the runtime value of myproperty is 44. That means that bar, is overriden with foo which was supposed to only add properties, but not to override them. When starting the application, I get:
The following profiles are active: foo,bar
I added spring.profiles.active=bar to application-bar.yaml as suggested by this answer, in another question, but it has no effect - there is no difference when property is there or not (I also tried using dash listing instead of comma separated values).
My question is, is it how it is supposed to work (then Spring Reference is misleading)? If so, are there any solutions for that?
Adding a link to the application source code on a github.
We implemented the Spring active profiles in a slightly different way. Let's say the default properties file, application.yml, contains all default values which is same in both production and development environments.
Create separate properties for production and development files named application-prd.yml and application-dev.yml respectively. These files may contain extra properties or override some of the default properties.
During application startup, we pass the spring.profiles.active as an environment variable. For example,
-Dspring.profiles.active=prd
will pick up application-prd.yml along with application.yml
or
-Dspring.profiles.active=dev
will pick up application-dev.yml along with application.yml
According to the spring boot documentation here, spring.profiles.include is used to add the properties from other profiles. It will add the property from other profiles if the property is not present in active one. But if it is present, then it will overwrite and the last one to be applied wins
Sometimes, it is useful to have profile-specific properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to unconditionally add active profiles.
Spring Boot 2.4 changes the mechanism for including multiple profiles to use a new profile groups feature, rather than using spring.profiles.include within the profile-specific document. This means that your configuration is no longer valid for new versions of Spring Boot, and would need to be changed.
That said, your use case does not seem to be a great fit for profile groups, as it's not really combining two profiles so much as overriding defaults. Therefore, I recommend using the approach suggested in another answer of putting the common and default properties in a shared application.yaml file, and only including the environment-specific values & overrides in the profile-specific documents.
application.yaml
spring:
myproperty: 44 # Default value
application-bar.yaml
spring:
myproperty: 55 # Override default
Note that Spring Boot supports multi-document files, so these can be combined into a single application.yaml file if desired:
spring:
myproperty: 44 # Default value
---
spring.config.activate.on-profile: bar # These configs apply to the bar profile
spring:
myproperty: 55 # Override default
Relevant 2.4 Changes
It is no longer possible to use spring.profiles.include within a profile-specific document as of Spring Boot 2.4, unless legacy mode is enabled using spring.config.use-legacy-processing=true. Per the 2.4 Spring Boot Config Data Migration Guide:
you can still use the spring.profiles.include property, but only in non profile-specific documents.
This approach has been replaced by the profile groups feature. Per the migration guide:
As discussed above, it’s no longer possible to use spring.profiles.include in a profile-specific document so this file isn’t valid.
Since this use-case is quite common, we’ve tried to provide another way to support it. In Spring Boot 2.4 you can use the “profile groups” feature.
This feature is documented in the Profile Groups section of the Spring Boot reference guide:
A profile group allows you to define a logical name for a related group of profiles.
For example, we can create a production group that consists of our proddb and prodmq profiles.
spring:
profiles:
group:
production:
- "proddb"
- "prodmq"
Our application can now be started using --spring.profiles.active=production to active the production, proddb and prodmq profiles in one hit.
The migration guide points out that the spring.profile.group property cannot be used in profile-specific documents.
The spring.profile.group property cannot be used in profile-specific documents.
You could add a new profile in the application-bar.yaml:
spring.profiles.include: foo,foo-override
myproperty: 33
---
spring.profiles: foo-override
myproperty: 55
The order is: 33 in bar overridden by 44 in foo overridden by 55 in foo-override.
Given:
The files: application-default.yml, application-foo.yml, application-bar.yml
myproperty: default in application-default.yml
myproperty: foo in application-foo.yml
myproperty: bar in application-bar.yml
I think these 2 use cases of using profiles are a little bit opposite in the meaning:
In the most common case (-Dspring.profiles.active but no spring.profiles.include):
When profile foo or boo are activated the properties from
application-foo.yml (or application-bar.yml) will add/override the
ones from application-default.yml.
When profiles foo,bar are activated then properties from bar will add/override those from application-foo.yml and then those from application-default.yml.
E.g: -Dspring.profiles.active=foo,bar the property from application-bar.yml wins (overrides) -> myproperty: bar
In the second case (spring.profiles.include is used)
The properties from the include statement add/overrides the properties from the application-*.yml files which uses spring.profiles.include
I.e.: If application-boo.yml contains the spring.profiles.include=foo then properties from application-foo.bar adds/override properties from from application-bar.yml which add/override those from application-default.yml.
On the other hand (I suppose) if application-boo.yml includes the spring.profiles.include=default,foo then properties from application-foo.yml will add/override those from application-default.yml which add/overrides those from application-bar.yml. So myproperty: bar. I wouldn't recommend the usage of default in combination with spring.profiles.include because this way it mixes the two cases and the override strategy is counterintuitive considering application-default.yml has a special treatment in springboot.
I also admit I am not at all a fan of the spring.profiles.active usage in application-*.yml files. I prefer to activate the profiles with system properties (maven included) or env variables. IMO it makes the whole profiles thing clearer to me.
If with my (herein above)reasoning I am on the wrong path please let me know.
Related
Let say my application.properties file of my microservice looks like this.
server.port=8023
spring.profiles.active=staging
spring.application.name=service-ws
spring.config.import=optional:configserver:http://my.cloud.config.server:8012/
What I try to achieve is, when I change "spring.profiles.active" to local, I need to use
spring.config.import=optional:configserver:http://localhost:8012/
As Config Server. Instead of
spring.config.import=optional:configserver:http://my.cloud.config.server:8012/
You need to use this format: application-{env}.properties
For example, for local environment: application-local.properties file will do the work.
Create profile application.properties files.
An example:
application.properties - defaults for all profiles.
application-local.properties - property values for the local profile
application-dev.properties - property values for the dev profile
application-kapow.properties - property values for the kapow profile
Note,
the profile name is your decision;
common values include: local, dev, qa, uat, and prod.
"kapow" is a valid profile name.
I've seen profiles defined by hostname.
Remember that you can have multiple profiles active at once.
Here is a link to a
Baeldung article that discusses Spring profiles
I have seen projects where a standalone application.properties is defined under src/test/resources containing all the properties from the default application.properties defined under src/main/resources while overriding properties for testing purposes. The disadvantage of this approach is that every change (adding/removing/updating of properties) must be done in both application.properties, otherwise tests may fail.
On the other hand I have seen projects where only a context based property file is defined under src/test/resources, say application-test.properties containing only the overriden properties for testing purposes. This approach merges the default application.properties under src/main/resources and the application-test.properties under src/test/resources. The respective context must be activated in test classes via #ActiveProfiles, say #ActiveProfiles("test")
What is the approach you follow and why? Is there any best practice or recommendation for this?
Thanks for your replies in advance.
Profiles are generally used to "map" different Spring beans accordingly. I've seen that people usually stick to three of them for most of the workflows: development, test, and production — mainly in tutorials and blogs.
I use profiles aligned with the actual environments the application is going to be deployed at since it's more realistic. So I always run the tests with the local profile. If I need any specific settings for running the tests, I just redefine an application.properties file under src/test/resources and overwrite what's needed.
In this way, you would always effectively have what's going to be used to run the application in each environment within src/main/resources, and if anything needs to be overwritten only for testing, the src/test/resources/application.properties could take care of that.
For instance, let's say your src/main/resources/application.yaml file looks like:
application:
key: value-common
another-key: value-common-another
spring:
profiles:
active: default
---
spring:
config:
activate:
on-profile: local
main:
banner-mode: console
---
spring:
config:
activate:
on-profile: test
This defines effectively two profiles: local and test — because default would have what's common to both. Now, if you place an application.properties file with the following content in src/test/resources:
application.key: value-common-test
...everything will stay the same, but application.key will be bound to value-common-test instead when you run the tests (for any profile). If you happen to just run the application, it will be bound to value-common instead.
I am trying to revamp my microservice to maintain a single application yaml rather maintaining multiple profiles. Initially I was maintaining different profiles and the common configurations were repeated across the helm environment specific values yaml. Now the strategy is to move everything to values.yml and maintain environment specific values in config map. The problem I face is my application yml now looks very generic with placeholders and for the same reason the test runs fails as I cannot give a default value for each of the configurations in application.yml. The reason being, For Eg: mongodb cluster endpoint format is different in local to other environment. I managed to place a local specific yaml file under test/resources, but not sure it's the right approach. I need to anyway maintain a local specific yaml under main/resources for running locally. So essentially I am duplicating it under test resources as well. Is there any better way of pointing test to load the application-local.yml under main resources so that I can avoid the duplication or is there any better way of doing this as a whole?
1. Working with multiple configs in One File
You can add all your configurations in one property file as illustrated below
spring.application.name: test. ## Used for all profiles
---
spring.config.active.on-profile:dev
spring.database.host: localhost
spring.database.name: testing
---
spring.config.active.on-profile:prod. ##You can use spring.profiles:prod
spring.database.host: localhost
spring.database.name: testing
---
--- marks where yml document splits, while #--- marks where properties file splits.
Multi-document property files are often used in conjunction with the following activation properties
spring.config.activate.on-profile
spring.config.activate.on.on-cloud-platform
All property definitions defined without specifying the profile name are used on all profiles. In the above case spring.application.name will be used on all profiles dev or prod.
When running the application you can manually specify profile or you can set in within the yml or properties on properties that are used throughout the application.
spring.application.name: test
spring.profiles.active: prod
2. Testing your application
when running tests that need to access properties in yml(property)file there is
no need to redefine your configurations.Just add #ActiveProfile("profile-name")
on your tests.
for example:
#ActiveProfiles("dev")
#SpringBootTest(webEnvironmentSpringBootTest.WebEnvironment.RANDOM_PORT)
I understand there's multiple ways of handling application property files and profiles in Spring Boot and I've seen multiple questions and answers on how to handle each but I'm trying to find the "best" way of handling it for a new project, if there is one.
The application is hosted in Weblogic 12c on production/pre-prod (with a jndi database connection) and ran locally in tomcat (with hardcoded database details) for development. I'd like it so that when the project is built via gradle and deployed to production it uses the jndi properties file and when ran locally it defaults to the hardcoded datasource with minimal changes required.
src/main/resources/application.properties
# DEV
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# DEV
# PROD
# spring.datasource.jndi-name=
# spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# PROD
From my understanding the recommended way is to externalize the property files and place the required one in a config directory alongside the WAR file for any differing config which is then automatically picked up and used?
You should consider creating multiple profiles. This means: Either multiple properties-Files, or multiple profiles in one file:
See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html
I would recommend to use multiple application-ENV.properties, e.g.
application-prod.properties and application-preprod.properties.
There is always one active profile and settings from the application.properties (without any profile suffix) are used as default values if not overwritten in a specific profile-file.
Depdending on your environment (local, prod etc.) you should set an environment variable (start the java-process/application server with that environment variable), e.g.:
SPRING_PROFILES_ACTIVE=prod
On your local machine you would set:
SPRING_PROFILES_ACTIVE=dev
With this variable you can control, which profile is currently active.
Also consider integrating the active profile into you Continious Integration/Deployment settings.
Please note that putting plain text passwords hardcoded into committed files is not a good idea. Consider using jasypt or a spring cloud config server for your prod database configuraiton or use any mechanism that your cloud provider provides to you if you use any. Microsoft Azure for example provides a KeyVault for sensitive data.
https://cloud.spring.io/spring-cloud-config/multi/multi_spring-cloud-config.html
http://www.jasypt.org/
If you use gradle good solution is to set application.properties and test.properties files and put into them properties for prod and preprod respectively.
Then run application with different vm arguments: -Dspring.profiles.active=test for test.properties and without arguments for application.properties
Use gradle instruments and configure them once for test and prod. For example: gradle bootWar configure without vm arguments, gradle bootWarTest with vm arguments -Dspring.profiles.active=test. Save once you configs and you will create war for different environments only selecting between two buttons in gradle.
I currently have the following config setup in spring boot:
application.properties
app.database.host=${DB_HOST}
app.database.port=${DB_PORT}
app.database.name=${DB_NAME}
app.database.user=${DB_USER}
app.database.password=${DB_PASSWORD}
app.database.schema=${DB_SCHEMA:public}
spring.datasource.url=jdbc:postgresql://${app.database.host}:${app.database.port}/${app.database.name}
spring.datasource.username=${app.database.user}
spring.datasource.password=${app.database.password}
spring.datasource.driver-class-name=org.postgresql.Driver
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.PostgreSQLDialect
application-local-dev.properties:
app.database.host=${DB_HOST:localhost}
app.database.port=${DB_PORT:5432}
app.database.name=${DB_NAME:db_name}
app.database.user=${DB_USER:root}
app.database.password=${DB_PASSWORD:root}
app.database.schema=${DB_SCHEMA:public}
application-load-fixtures.properties:
spring.profiles.include=local-dev
spring.profiles.active=load-fixtures,local-dev
app.database.name=${DB_NAME:db_name}_fixtures
The idea here is that when starting the app in default mode, it will fail to boot when critical properties like database name are missing.
They should be passed via environment variables.
For development purposes, this is unnecessary overhead when setting up the project because we have a docker container with static credentials and I'd like to provide them as defaults. Therefore, I created a profile local-dev that will use default values to be able to connect to our docker database and still have the ability to override them via environment variables in case someone needs to.
Until here, everything works fine.
But now, we also have a profile that is used to load fixtures into the database (drop all tables, recreate and fill them with data).
For obvious reasons, I want to ensure that this cannot be done on an arbitrary database, so I created a profile load-fixtures that should inherit all properties from local-dev and override the database name. However, this approach seems to be wrong. I can see in the spring log that the profiles are loaded properly:
2017-11-16 13:32:11.508 INFO 23943 --- [ main] Main:
The following profiles are active: load-fixtures,local-dev
But it still uses the database name provided by the local-dev profile.
When I remove the line
app.database.name=${DB_NAME:db_name}
from the local-dev config file, it works.
However, what I want to avoid is having to add new properties to both, local-dev and load-fixtures, whenever we add a new configuration property to the project.
I understand that profile specific properties take precedence over non-profile specific ones. And also that non-default location properties take precedence over properties from the default locations. But here, both profiles (local-dev and load-fixtures) are in the same location, and they are also both profile specific.
What are proper ways to go about this problem?
Thanks in advance!
I recently came across quite the same problem and had to figure out which precedence Spring applies to several profile specific property files. Unfortunately this is not well documented and I did not find the location of the code that is responsible for that.
However after some tests and tries I'm pretty sure it works like this (or at least in a similar way):
Probably some kind of map is used to gather up all properties of all the different places and possibilites where you could define them like documented here. So for example a property my.value is defined in application.properties and so stored in the mentioned map. Then the same property is found as Java system property. Since this way of defining a property is higher in the PropertySource-order it will override the value found before in the map. Until here it is clear according to the documentation that the Java system property will win.
But as we come to two different sources on the same precedence level like two different profile specific property files the documentation is not a 100% clear in my opinion. However it says in 24.4:
If several profiles are specified, a last-wins strategy applies. For example, profiles specified by the spring.profiles.active property are added after those configured through the SpringApplication API and therefore take precedence.
Maybe it is just the example that is not optimal here or I just do not understand it correctly. But I guess the "last-wins" strategy also applies to all profiles defined for example in spring.profiles.active. That means if you run java -jar -Dspring.profiles.active=dev,fix application.jar, the properties in application-fix.properties will overwrite the values of properties having the same key in application-dev.properties.
So in your case considering the output of your application I guess you specified something like java -jar -Dspring.profiles.active=load-fixtures,local-dev application.jar. If I was correct, you would just have to change that into java -jar -Dspring.profiles.active=local-dev,load-fixtures application.jar.