I have seen many questions in regards to Spring Boot external configuration, in specific, externalizing a application.properties file, but didn't find a specific answer on whether property values from externalized application.properties files should be merged or the whole .properties file on the classpath should be overridden with the external one.
After recent app upgrade to Spring Boot 2.2.1-RELEASE version, there was a need to enable bean overriding with spring.main.allow-bean-definition-overriding=true property, which is set in the application.properties file on the classpath. Since we want to use an external application.properties file, I created one(same name) without the aforementioned property and loaded it with the following property in IntelliJ:
java -jar appName.jar --spring.config.location="C:\Users\User\Downloads\application.properties"
The error came up as though the external props file completely overrode the classpath file, without looking for the mentioned property in the classpath file and using the missing property from the classpath application.properties:
***************************
APPLICATION FAILED TO START
***************************
Description:
The bean 'userRepository' could not be registered. A bean with that name has already been defined and overriding is disabled.
Action:
Consider renaming one of the beans or enabling overriding by setting spring.main.allow-bean-definition-overriding=true
After adding the bean overriding property to the external application.properties file, but removing it from the classpath one, error is gone.
In Spring docs, it is clearly stated that :
Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:
...
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
After testing it out with IntelliJ and a packaged jar, both method exhibit the same: seems like the whole file is being overridden and values are not merged, but only loaded from the externalized properties file. My questions is, is this the expected behavior or am I missing out on something ?
EDIT: tested the behavior with jar
server.port = ?
spring.datasource.url = ?
spring.datasource.username = ?
spring.datasource.password = ?
I want to externalize all the "?" values outside the application.properties, and have them in a text file or something.
I already have a configuration.txt file which holds other values, used in the services, but I just don't know how it works for the application.properties.
Solved, just have the property file in the same path where the jar file is, then spring boot will replace the values for you.
Spring.io Externalized Configuration
Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values. Properties are considered in the following order:
Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
#TestPropertySource annotations on your tests.
properties attribute on your tests. Available on #SpringBootTest and the test annotations for testing a particular slice of your application.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that has properties only in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
What this means is that what you want to do is supported without needing to do anything fancy inside your application.properties file.
Spring Boot will look at the application.properties file outside of your jar and consider any values in there and use them instead of any values in the application.properties file inside of your jar.
So, wherever your jar is, put the application.properties file you want for that environment. See the link for more details on just how much you can customize this (profiles, YAML, system properties, environment variables, etc.)
You might also consider moving to a Spring Cloud Config implementation, but that's a bit more work.
How we can programmatically configure Spring Boot to define new values to the spring.config.name and spring.config.location properties when running JUnit tests?
For example, if we would like to define these properties when running the application itself we could use something like (in Kotlin):
fun main(args: Array<String>) {
// SpringApplication.run(Application::class.java, *args)
val applicationContext = SpringApplicationBuilder(Application::class.java)
.properties(
"""spring.config.name:
${getSpringConfigNames()}
""",
"""spring.config.location:
${getSpringConfigLocationPaths()}
"""
)
.build()
.run(*args)
// val environment = applicationContext.getEnvironment()
}
But I wasn't able to find a way to configure this to use in the JUnit tests.
Edit
There is a complication here because of an spring boot limitation.
I would like to use an entire folder and its subfolders as valid locations to search for configuration files (so, for example, we could have folders for specific environments, databases, third-parties, and so on).
When running the application this was possible creating a method, in this case getSpringConfigLocationPaths(). And this method create a comma separated list with all folder inside the "main" folder.
For example, for the main folder src/main/resources/configuration it will output:
src/main/resources/configuration,
src/main/resources/configuration/environments,
src/main/resources/configuration/environments/development,
src/main/resources/configuration/environments/staging,
src/main/resources/configuration/environments/testing,
src/main/resources/configuration/environments/production,
src/main/resources/configuration/environments/common
How could we solve this situation when using JUnit tests and Spring Boot?
Unfortunately Spring Boot doesn't allow something like src/main/resources/configuration/**/*.
Because we have organized the system with several properties files on different subfolders we need to find a way to dinamically consider them.
I am using latest Spring Boot 2.2.0 and from my experience both #TestPropertySource and #SpringBootTest annotations can do the job because they have properties attribute.
So, you can do something like this:
#TestPropertySource(properties = ["spring.config.location=classpath:dev/", "spring.config.name=custom-app-name"]
#TestConfiguration
class DevTestCfg {} // this will make tests to look for configs in resources/dev/custom-app-name.properties
Also notice that there is a spring.config.additional-location property if you want your properties to be loaded from multiple locations.
The only problem here is that values in properties attribute must be constant.
But you can create multiple configurations for each environment and put corresponding #Profile("envName") on each configuration class. Then run your tests with different -Dspring.profiles.active and corresponding test configuration should be automatically picked up.
The tests that run spring boot should be carefully designed,
There is a whole testing framework for spring boot tests, so obviously consider using this framework.
When it comes to configuration management, I suggest considering the following:
There are two types of tests basically:
Tests that load a concrete specific configuration (set of beans), for example if you want to test only a DAO, you load a configuration for this dao.
In this case, the configuration is something that should be "tailored" to the needs of a specific test, and no "full" configuration is required.
For example, if the microservice contains a configuration for a database (user, password, schema, etc) and for, say, messaging management, there is no need to specify a configuration of a messaging system when testing a DAO, messaging beans won't be loaded anyway.
Usually, the test of this "type" will look like this:
#SpringBootTest(classes = {RelationalDbDaoConfiguration.class})
public class MyDaoTest {
}
If you don't have a configuration for your needs you can use #MockBean to mock unnecessary beans or even create a custom configuration in src/test/java so that it will be only in test classpath. It makes sense to use #TestConfiguration but it's beyond the scope of the question.
Now in order to load the configuration for db only, the are many options, to name a few:
#ActiveProfiles("dao") on a test class + putting "application-dao.properties/yaml" into the src/test/resources or src/test/resources/config
Use #TestPropertySource(locations = "classpath:whatever.properties") on test
Create a special "DbProperties" bean and initialize it programmatically in spring, it can make sense when you know some details about the context in which the test runs only during the actual test execution (for example, if you start a database before the test and the port is created dynamically, but its really a fairly advanced setup and is beyond the scope of this question) + the data source bean can read these properties
Use #SpringBootTest's properties attribute to provide 'fine-grained' properties definitions
Kind of obvious, but I'll mention it anyway: put application.properties in src/test/resources it will override regular configurations
The second type of tests is when you load the "entire" microservice, usually, these are tests that do not have "classes" parameter in #SpringBootTest annotation
#SpringBootTest // note, no actual configurations specified
public class MyMicroserviceTest {
...
}
Now, this definitely requires to specify a whole set of configurations, although the techniques for actually specifying these configurations are still applicable (just the content of configuration files will be different).
I do not suggest the usage of spring.config.location during the test, because this means that the test depends on some external resource, which makes the whole setup even more complicated.
If it's XML driven configuration,
#ContextConfiguration(locations = "/app-context.xml")
If it's annotation driven by configuration classes,
#ContextConfiguration(classes = {AppCOnfig::class, AnotherCOnfig::class}
These would be defined on the class level on the unit test class you run.
Further, if you have profiles for Junit to consider,
#ActiveProfiles("myProfile") would be added to the test class.
I have my config class as below
#Configuration
#PropertySource(value= "classpath:conf/agent.properties")
public class AppConfig{
While running the application using
java -jar app.jar spring.config.location=path-to-external-property-file
the application runs fine and takes the properties from the external properties file, but if the properties file in classpath is missing FileNotFoundException is thrown, is this expected behavior? For me it should work with the external properties, it should not care about classpath resource and stop running.
To overcome this I am currently using ignoreResourceNotFound = true
You seem to mis-understand the meaning of spring-config-location.
From the docs, by default spring searches for a file named application.properties in certain locations in your project(/config, ..) to setup application context.
You can tell spring that you don't want to use this packaged file in the project, and you will specify the file needed to setup application context at run time using command line arguments. This is facilitated using
spring-config-location option.
Having said that, in your project you might have several configuration (#Configuration) files. And some files need some properties to be loaded from some file. This can be specified using #PropertySource, which is what you are doing. This is not like a replacement for application.properties. #PropertySource is just an indication to tell spring that it needs to read a file to inject properties specified in a particular class.
Hence even if you specify external config location spring before setting up the AppConfig class it looks for the Source mentioned in #PropertySource
I have some common properties that every projects should set, such as
feign.hystrix.enabled=false
feign.httpclient.enabled=true
I don't want to repeatedly add these props in every project so I'm going to create an extra jar file containing #Configruation class. How to add properties in #Configuration class? Thanks!
PropertySources
You may load an application.properties from another jar this way:
#PropertySources({
#PropertySource("classpath:common.properties")
})
#Configuration
public class SomeJavaConfig {
}
You can find the reference in Spring's documentation:
Spring Boot uses a very particular PropertySource order that is
designed to allow sensible overriding of values. Properties are
considered in the following order:
...
#PropertySource annotations on your #Configuration classes.
Spring-cloud-config
I won't go in all the details, but another option is to use spring-cloud-config to define these properties in a git (using spring-cloud-config-server). Then, have your spring-boot application load the application.properties using spring-cloud-config-client directly from git.
Check this:
https://cloud.spring.io/spring-cloud-config/spring-cloud-config.html
https://spring.io/guides/gs/centralized-configuration/