I understand there's multiple ways of handling application property files and profiles in Spring Boot and I've seen multiple questions and answers on how to handle each but I'm trying to find the "best" way of handling it for a new project, if there is one.
The application is hosted in Weblogic 12c on production/pre-prod (with a jndi database connection) and ran locally in tomcat (with hardcoded database details) for development. I'd like it so that when the project is built via gradle and deployed to production it uses the jndi properties file and when ran locally it defaults to the hardcoded datasource with minimal changes required.
src/main/resources/application.properties
# DEV
spring.datasource.url=
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# DEV
# PROD
# spring.datasource.jndi-name=
# spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
# PROD
From my understanding the recommended way is to externalize the property files and place the required one in a config directory alongside the WAR file for any differing config which is then automatically picked up and used?
You should consider creating multiple profiles. This means: Either multiple properties-Files, or multiple profiles in one file:
See https://docs.spring.io/spring-boot/docs/current/reference/html/howto-properties-and-configuration.html
I would recommend to use multiple application-ENV.properties, e.g.
application-prod.properties and application-preprod.properties.
There is always one active profile and settings from the application.properties (without any profile suffix) are used as default values if not overwritten in a specific profile-file.
Depdending on your environment (local, prod etc.) you should set an environment variable (start the java-process/application server with that environment variable), e.g.:
SPRING_PROFILES_ACTIVE=prod
On your local machine you would set:
SPRING_PROFILES_ACTIVE=dev
With this variable you can control, which profile is currently active.
Also consider integrating the active profile into you Continious Integration/Deployment settings.
Please note that putting plain text passwords hardcoded into committed files is not a good idea. Consider using jasypt or a spring cloud config server for your prod database configuraiton or use any mechanism that your cloud provider provides to you if you use any. Microsoft Azure for example provides a KeyVault for sensitive data.
https://cloud.spring.io/spring-cloud-config/multi/multi_spring-cloud-config.html
http://www.jasypt.org/
If you use gradle good solution is to set application.properties and test.properties files and put into them properties for prod and preprod respectively.
Then run application with different vm arguments: -Dspring.profiles.active=test for test.properties and without arguments for application.properties
Use gradle instruments and configure them once for test and prod. For example: gradle bootWar configure without vm arguments, gradle bootWarTest with vm arguments -Dspring.profiles.active=test. Save once you configs and you will create war for different environments only selecting between two buttons in gradle.
Related
I am trying to revamp my microservice to maintain a single application yaml rather maintaining multiple profiles. Initially I was maintaining different profiles and the common configurations were repeated across the helm environment specific values yaml. Now the strategy is to move everything to values.yml and maintain environment specific values in config map. The problem I face is my application yml now looks very generic with placeholders and for the same reason the test runs fails as I cannot give a default value for each of the configurations in application.yml. The reason being, For Eg: mongodb cluster endpoint format is different in local to other environment. I managed to place a local specific yaml file under test/resources, but not sure it's the right approach. I need to anyway maintain a local specific yaml under main/resources for running locally. So essentially I am duplicating it under test resources as well. Is there any better way of pointing test to load the application-local.yml under main resources so that I can avoid the duplication or is there any better way of doing this as a whole?
1. Working with multiple configs in One File
You can add all your configurations in one property file as illustrated below
spring.application.name: test. ## Used for all profiles
---
spring.config.active.on-profile:dev
spring.database.host: localhost
spring.database.name: testing
---
spring.config.active.on-profile:prod. ##You can use spring.profiles:prod
spring.database.host: localhost
spring.database.name: testing
---
--- marks where yml document splits, while #--- marks where properties file splits.
Multi-document property files are often used in conjunction with the following activation properties
spring.config.activate.on-profile
spring.config.activate.on.on-cloud-platform
All property definitions defined without specifying the profile name are used on all profiles. In the above case spring.application.name will be used on all profiles dev or prod.
When running the application you can manually specify profile or you can set in within the yml or properties on properties that are used throughout the application.
spring.application.name: test
spring.profiles.active: prod
2. Testing your application
when running tests that need to access properties in yml(property)file there is
no need to redefine your configurations.Just add #ActiveProfile("profile-name")
on your tests.
for example:
#ActiveProfiles("dev")
#SpringBootTest(webEnvironmentSpringBootTest.WebEnvironment.RANDOM_PORT)
My Spring Boot app will have 4 different environments:
Local; running locally on my machine
Dev
UAT
Prod
When a new user registers for my app, the backend sends them an email to verify their address/account. In this email there will be a link; the user clicks the link which verifies them in the database and allows them to now login and use the app.
These links of course have to have the environment built into them:
Locally, the link might be http://localhost:9200/v1/data/accounts/verify?vt=12345
In dev the link might be: http://dev.myapp.example.com/v1/data/accounts/verify?vt=12345
In UAT the link might be: http://uat.myapp.example.com/v1/data/accounts/verify?vt=12345
In Prod the link might be: http://myapp.example.com/v1/data/accounts/verify?vt=12345
In all three environments, the hostname + port are different. Locally I specify localhost:9200 (both localhost + port). Non-locally I don't need to specify port because the app will be running on nodes behind a load balancer. In production I don't need an environment-specific subdomain like dev or uat.
In order for my emails to work correctly in all 4 environments, I need to set an environment variable that is then used by my email generator to generate links correctly.
I could use something homegrown, such as a custom property in application.properties like emailDomain, and set the property different in each environment's properties file.
But I feel like the smart folks that make up the Spring Boot community have probably already solved this problem for me...have they? Is there already a Spring Boot app property I can set that will take care of all this for me?
In the comments, I read that your main concern is being able to update the property without having to modify your .JAR/.WAR or changing some code.
This is possible since you can externalize your Spring boot configuration. If you check the Externalized Configuration docs, you can see that it looks for properties within:
OS environment variables.
...
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar
(application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
So, ideally, you can put an application.properties file next to your JAR, and update the properties in that file depending on the environment you run on.
I am trying to find the best way to configure my Spring Boot Web application to easily switching between the following data sources for both local testing and deployment.
H2 in memory db. Local testing only.
Dev oracle. Local testing and deployment.
Prod oracle. Deployment only.
By local testing, I mean to test in IDE environment (Eclipse). Dev and prod oracle databases are set up on two remote servers.
After some research, there are different ways to switch from one data source to another.
Use Spring profile. Using H2 and Oracle with Spring Boot. Set up the following files in classpath, application.properties, application-h2. properties and application-dev.properties. While connections for h2 and dev are defined in corresponding properties files, spring.profiles.active is set in application.properties. My understanding is this property can be overridden during build process by specifying spring.profiles.active. However, it seems to be a JVM variable, how do I set it running maven?
Maven profile. Create multiple profiles in pom and a filter pointing to application properties files. The profile specified by -P option during maven build will determine which application properties file to look. However, according to maven application with multi environment configuration can't deploy on tomcat, this will generate multiple wars for different deployment. So method 1 is preferred. Plus, it does not apply to switching datasources while testing locally.
Persistence units. Define different persistence units for different data sources in persistence.xml. Use EntityManager by choosing a specific unit. Variation of this method include having a variable in unit names which is determined in application.properties.
JNDI lookup. Set up a jndi name in application.properties with spring.datasource.jndi-name. The actual database information including url and credentials will be specified in context.xml in the tomcat folder where the war will be deployed.
My mind is set on local testing environment. Gonna go with method 1. Switching between H2 in memory and oracle is so easy just by changing the property in application.properties. Since the testing is usually done in IDE, war does not need to be generated, although answers are welcome for run maven install with spring.profiles.active.
As far as deployment, JNDI is definitely the way to go. However, I am concerned that the two properties in application.properties: spring.profiles.active and spring.datasource.jndi-name may be conflicting with each other. If I have spring.profiles.active=h2 and then tried to deploy the war to prod server, does it try to connect to h2 based on the spring profile or to prod db based on jdni-name? What is the best practice to accommodate all scenarios with enough flexibility?
Also is a explicit configuration class for DataSource required such as Configure Mutiple DataSource in Spring Boot with JNDI? My understanding is application.properties and spring profile should be enough to handle it, right?
Definitely use Spring profiles.
You don't want to use Maven profiles as it creates different artifacts. Ask your QA/Release engineers how they feel about having different artifacts for different environments :). They wouldn't be happy.
H2 is what you want to use in CI server integration testing as well. Such integration testing is fast and easy.
Instead of changing profile in application.properties, consider defining profile via command line parameter. So that configuration file changes are not required to run your application in different profiles.
Having something like
security.user.password = plainTextPassword
inside Spring Boot’s application.properties is obviously an anti-pattern as it prevents the code from being pushed to a public SCM. In my non-Spring Boot projects I use
security.user.password = ${myPasswordFromMavenSettingsXML}
and put a corresponding <properties/> reference inside my pom.xml.
Using Maven’s resource filter plugin the passwords are replaced at build time so the application have access to actual plain text passwords after it has been build and deployed.
For some reason Maven’s resource filter plugin does not work in this case. Is there a way to not commit plain text passwords to an SCM and let Spring Boot to insert them at build time?
Spring boot has multiple mechanisms to provided externalized configuration. Some examples are command line arguments, environment variables and also application properties outside of your packaged JAR.
What I usually do:
Locally we configured several environment variables. Most (if not all) IDE's allow you to configure environment variables from within the run configuration.
For example if you don't want to expose the spring.datasource.password property you could set an environment variable called SPRING_DATASOURCE_PASSWORD.
When we deploy on another environment, we usually choose to add another application.properties or application.yml file within the same folder as the application JAR/WAR, since Spring boot picks that up as well.
Another solution is to use Spring cloud since it has a config service which can be used as a microservice to provide configuration. The configuration can be versioned using SCM as well, but you can put it on a separate system that is not connected to your source code.
I am wondering what is the best way to configure an WAS Liberty installation, allowing to to switch from a DEV environment configuration to an UAT(testing) environment configuration dynamically.
To elaborate, we have a similar setup with our glassfish servers, we simply configure system properties for both in the Glassfish console. For example
hostname.uat="some uat value"
hostname.dev="some dev value"
Dropping the ".uat" or ".dev" in the system property configuration in Glassfish makes that property active. In Glassfish, this can be done dynamically and while the application is running (no need to reboot).
Is there or can someone elaborate how I could achieve a similar setup in WAS Liberty?
Thank-you kindly
You can create a server.env file in two possible places:
${wlp.install.dir}/etc/server.env (properties are applied to all servers) or
${server.config.dir}/server.env (properties applied only to one server)
and specify any environment variables in that file.
For example:
# Specify properties and values
admin.email=dev.admin#domain.com
admin.email.uat=uat.admin#domain.com
To access these properties in an application environment (such as a Servlet) do the following:
System.getenv("admin.email"); // returns "dev.admin#domain.com"
Other useful properties can be specified in the server.env file as well such as JAVA_HOME, WLP_USER_DIR, WLP_OUTPUT_DIR, and WLP_DEBUG_ADDRESS.
For IBM's full doc on this, see: Customizing the Liberty Environment.
What we do is generate the Liberty server with Ansible, where the variables can be added to an ansible inventory based on environment.
So, our deployments essentially drop and recreate the Liberty server by using ansible templates and roles to stamp it out as needed.
Lastly, we make use of Hasicorp Vault (you can also use ansible-vault) for credentials or secrets at deploy time to fetch credentials. This is then injected into Ansible as JSON and used to stamp out the server.xml and other related configuration files.