I am trying to revamp my microservice to maintain a single application yaml rather maintaining multiple profiles. Initially I was maintaining different profiles and the common configurations were repeated across the helm environment specific values yaml. Now the strategy is to move everything to values.yml and maintain environment specific values in config map. The problem I face is my application yml now looks very generic with placeholders and for the same reason the test runs fails as I cannot give a default value for each of the configurations in application.yml. The reason being, For Eg: mongodb cluster endpoint format is different in local to other environment. I managed to place a local specific yaml file under test/resources, but not sure it's the right approach. I need to anyway maintain a local specific yaml under main/resources for running locally. So essentially I am duplicating it under test resources as well. Is there any better way of pointing test to load the application-local.yml under main resources so that I can avoid the duplication or is there any better way of doing this as a whole?
1. Working with multiple configs in One File
You can add all your configurations in one property file as illustrated below
spring.application.name: test. ## Used for all profiles
---
spring.config.active.on-profile:dev
spring.database.host: localhost
spring.database.name: testing
---
spring.config.active.on-profile:prod. ##You can use spring.profiles:prod
spring.database.host: localhost
spring.database.name: testing
---
--- marks where yml document splits, while #--- marks where properties file splits.
Multi-document property files are often used in conjunction with the following activation properties
spring.config.activate.on-profile
spring.config.activate.on.on-cloud-platform
All property definitions defined without specifying the profile name are used on all profiles. In the above case spring.application.name will be used on all profiles dev or prod.
When running the application you can manually specify profile or you can set in within the yml or properties on properties that are used throughout the application.
spring.application.name: test
spring.profiles.active: prod
2. Testing your application
when running tests that need to access properties in yml(property)file there is
no need to redefine your configurations.Just add #ActiveProfile("profile-name")
on your tests.
for example:
#ActiveProfiles("dev")
#SpringBootTest(webEnvironmentSpringBootTest.WebEnvironment.RANDOM_PORT)
I have application on spring-boot with any profiles for different servers(qa, dev, etc.)So, this is application.properties with different params.
Do I need addition application.properties for every server in test/resources for every servers or I need to use only src/resouces profiles?
You would want one application.properties that holds default values and then override properties files for each profile.
I am working on microservices using spring boot. i have around 5 microservices. so I configured spring cloud config server to centralized configuration. its perfectly working fine.
config server configuretion
spring.profiles.active=native
spring.cloud.config.server.native.search-locations=classpath:/common-config
server.port=8888
now the problem is I have two environment dev and test and i created application-test.properties and application-dev.properties as per the spring documented i need to set profile like
Spring profile
spring.profiles.active=test
but its already set as a native then how can i load multiple profile.
please help me ..
You can pass in the active profile name as an environment variable at run time of each of the apps. This will take priority over the active profile listed in your properties file. You can do this with any property actually. If you're using docker to launch your apps, you can pass it in from your Dockerfile or from your docker-compose.yml
You can set spring.profiles.active=$ACTIVE_SPRING_PROFILE and set OS environment ACTIVE_SPRING_PROFILE = test
You can use multiple profiles in Spring. Try:
spring.profiles.active=dev,native
See more: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-profiles.html#boot-features-profiles
I understand there's multiple ways of handling application property files and profiles in Spring Boot and I've seen multiple questions and answers on how to handle each but I'm trying to find the "best" way of handling it for a new project, if there is one.
The application is hosted in Weblogic 12c on production/pre-prod (with a jndi database connection) and ran locally in tomcat (with hardcoded database details) for development. I'd like it so that when the project is built via gradle and deployed to production it uses the jndi properties file and when ran locally it defaults to the hardcoded datasource with minimal changes required.
src/main/resources/application.properties
# DEV
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# DEV
# PROD
# spring.datasource.jndi-name=
# spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# PROD
From my understanding the recommended way is to externalize the property files and place the required one in a config directory alongside the WAR file for any differing config which is then automatically picked up and used?
You should consider creating multiple profiles. This means: Either multiple properties-Files, or multiple profiles in one file:
See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html
I would recommend to use multiple application-ENV.properties, e.g.
application-prod.properties and application-preprod.properties.
There is always one active profile and settings from the application.properties (without any profile suffix) are used as default values if not overwritten in a specific profile-file.
Depdending on your environment (local, prod etc.) you should set an environment variable (start the java-process/application server with that environment variable), e.g.:
SPRING_PROFILES_ACTIVE=prod
On your local machine you would set:
SPRING_PROFILES_ACTIVE=dev
With this variable you can control, which profile is currently active.
Also consider integrating the active profile into you Continious Integration/Deployment settings.
Please note that putting plain text passwords hardcoded into committed files is not a good idea. Consider using jasypt or a spring cloud config server for your prod database configuraiton or use any mechanism that your cloud provider provides to you if you use any. Microsoft Azure for example provides a KeyVault for sensitive data.
https://cloud.spring.io/spring-cloud-config/multi/multi_spring-cloud-config.html
http://www.jasypt.org/
If you use gradle good solution is to set application.properties and test.properties files and put into them properties for prod and preprod respectively.
Then run application with different vm arguments: -Dspring.profiles.active=test for test.properties and without arguments for application.properties
Use gradle instruments and configure them once for test and prod. For example: gradle bootWar configure without vm arguments, gradle bootWarTest with vm arguments -Dspring.profiles.active=test. Save once you configs and you will create war for different environments only selecting between two buttons in gradle.
My intention is to have two profiles in a spring boot application - development and production one. Development profile is meant just to override some variables of production profile (like in-memory database instead of database in the cloud). As I expect some changes to be done to production profile in the future, duplicating variables in development profile doesn't seem to be a solution.
So, in Spring Reference I read that spring.profiles.include is supposed to only add properties from referenced profile.
Sometimes, it is useful to have profile-specific properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to unconditionally add active profiles.
However, from what I've checked it rather overrides it. So, when having two profiles foo and bar, in separate yaml files:
application-foo.yaml:
myproperty: 44
application-bar.yaml:
spring:
profiles:
include: foo
active: bar,foo
myproperty: 55
And setting -Dspring.profiles.active=bar variable in IDE, the runtime value of myproperty is 44. That means that bar, is overriden with foo which was supposed to only add properties, but not to override them. When starting the application, I get:
The following profiles are active: foo,bar
I added spring.profiles.active=bar to application-bar.yaml as suggested by this answer, in another question, but it has no effect - there is no difference when property is there or not (I also tried using dash listing instead of comma separated values).
My question is, is it how it is supposed to work (then Spring Reference is misleading)? If so, are there any solutions for that?
Adding a link to the application source code on a github.
We implemented the Spring active profiles in a slightly different way. Let's say the default properties file, application.yml, contains all default values which is same in both production and development environments.
Create separate properties for production and development files named application-prd.yml and application-dev.yml respectively. These files may contain extra properties or override some of the default properties.
During application startup, we pass the spring.profiles.active as an environment variable. For example,
-Dspring.profiles.active=prd
will pick up application-prd.yml along with application.yml
or
-Dspring.profiles.active=dev
will pick up application-dev.yml along with application.yml
According to the spring boot documentation here, spring.profiles.include is used to add the properties from other profiles. It will add the property from other profiles if the property is not present in active one. But if it is present, then it will overwrite and the last one to be applied wins
Sometimes, it is useful to have profile-specific properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to unconditionally add active profiles.
Spring Boot 2.4 changes the mechanism for including multiple profiles to use a new profile groups feature, rather than using spring.profiles.include within the profile-specific document. This means that your configuration is no longer valid for new versions of Spring Boot, and would need to be changed.
That said, your use case does not seem to be a great fit for profile groups, as it's not really combining two profiles so much as overriding defaults. Therefore, I recommend using the approach suggested in another answer of putting the common and default properties in a shared application.yaml file, and only including the environment-specific values & overrides in the profile-specific documents.
application.yaml
spring:
myproperty: 44 # Default value
application-bar.yaml
spring:
myproperty: 55 # Override default
Note that Spring Boot supports multi-document files, so these can be combined into a single application.yaml file if desired:
spring:
myproperty: 44 # Default value
---
spring.config.activate.on-profile: bar # These configs apply to the bar profile
spring:
myproperty: 55 # Override default
Relevant 2.4 Changes
It is no longer possible to use spring.profiles.include within a profile-specific document as of Spring Boot 2.4, unless legacy mode is enabled using spring.config.use-legacy-processing=true. Per the 2.4 Spring Boot Config Data Migration Guide:
you can still use the spring.profiles.include property, but only in non profile-specific documents.
This approach has been replaced by the profile groups feature. Per the migration guide:
As discussed above, it’s no longer possible to use spring.profiles.include in a profile-specific document so this file isn’t valid.
Since this use-case is quite common, we’ve tried to provide another way to support it. In Spring Boot 2.4 you can use the “profile groups” feature.
This feature is documented in the Profile Groups section of the Spring Boot reference guide:
A profile group allows you to define a logical name for a related group of profiles.
For example, we can create a production group that consists of our proddb and prodmq profiles.
spring:
profiles:
group:
production:
- "proddb"
- "prodmq"
Our application can now be started using --spring.profiles.active=production to active the production, proddb and prodmq profiles in one hit.
The migration guide points out that the spring.profile.group property cannot be used in profile-specific documents.
The spring.profile.group property cannot be used in profile-specific documents.
You could add a new profile in the application-bar.yaml:
spring.profiles.include: foo,foo-override
myproperty: 33
---
spring.profiles: foo-override
myproperty: 55
The order is: 33 in bar overridden by 44 in foo overridden by 55 in foo-override.
Given:
The files: application-default.yml, application-foo.yml, application-bar.yml
myproperty: default in application-default.yml
myproperty: foo in application-foo.yml
myproperty: bar in application-bar.yml
I think these 2 use cases of using profiles are a little bit opposite in the meaning:
In the most common case (-Dspring.profiles.active but no spring.profiles.include):
When profile foo or boo are activated the properties from
application-foo.yml (or application-bar.yml) will add/override the
ones from application-default.yml.
When profiles foo,bar are activated then properties from bar will add/override those from application-foo.yml and then those from application-default.yml.
E.g: -Dspring.profiles.active=foo,bar the property from application-bar.yml wins (overrides) -> myproperty: bar
In the second case (spring.profiles.include is used)
The properties from the include statement add/overrides the properties from the application-*.yml files which uses spring.profiles.include
I.e.: If application-boo.yml contains the spring.profiles.include=foo then properties from application-foo.bar adds/override properties from from application-bar.yml which add/override those from application-default.yml.
On the other hand (I suppose) if application-boo.yml includes the spring.profiles.include=default,foo then properties from application-foo.yml will add/override those from application-default.yml which add/overrides those from application-bar.yml. So myproperty: bar. I wouldn't recommend the usage of default in combination with spring.profiles.include because this way it mixes the two cases and the override strategy is counterintuitive considering application-default.yml has a special treatment in springboot.
I also admit I am not at all a fan of the spring.profiles.active usage in application-*.yml files. I prefer to activate the profiles with system properties (maven included) or env variables. IMO it makes the whole profiles thing clearer to me.
If with my (herein above)reasoning I am on the wrong path please let me know.