Binding multiple application.yml from working directory to ConfigurationProperties - spring

Starting with Spring Boot 2.4.0, from what I gather from the reference documentation, I should be able to bind multiple application.yaml files from the working directory to ConfigurationProperties:
Wildcard locations are particularly useful in an environment such as Kubernetes when there are multiple sources of config properties.
For example, if you have some Redis configuration and some MySQL configuration, you might want to keep those two pieces of configuration separate, while requiring that both those are present in an application.properties file. This might result in two separate application.properties files mounted at different locations such as /config/redis/application.properties and /config/mysql/application.properties. In such a case, having a wildcard location of config/*/, will result in both files being processed.
By default, Spring Boot includes config/*/ in the default search locations. The means that all subdirectories of the /config directory outside of your jar will be searched.
It is my understanding that the following test case shows that this does not work as described:
Project hierarchy:
config/
first/
application.yaml
second/
application.yaml
src/
main/
kotlin/
test/
Application.kt
test/
kotlin/
test/
TestCase.kt
config/first/application.yaml
composite:
from-first-file:
it: works
config/second/application.yaml
composite:
from-second-file:
it: works
src/main/kotlin/test/Application.kt
#SpringBootApplication(proxyBeanMethods = false)
#EnableConfigurationProperties(CompositeProperties::class)
class Application
fun main(vararg args: String) {
runApplication<Application>(*args)
}
#ConstructorBinding
#ConfigurationProperties("composite")
data class CompositeProperties(
val fromFirstFile: Map<String, String> = mapOf(),
val fromSecondFile: Map<String, String> = mapOf()
)
src/test/kotlin/test/TestCase.kt
#SpringBootTest
class TestCase {
#Autowired
private lateinit var compositeProperties: CompositeProperties
#Test
fun `first file is bound to configuration properties`() {
assertThat(compositeProperties.fromFirstFile).containsEntry("it", "works")
}
#Test
fun `second file is bound to configuration properties`() {
assertThat(compositeProperties.fromSecondFile).containsEntry("it", "works")
}
}
The first test passes, the second one fails. I tried several variations to rule out edge cases and it looks like the second file is completely overlooked. For instance, I tried setting the spring.application.name in the second file and it is not taken into account either.
If this is indeed a bug, is there a workaround I could use until this is fixed?

As I suspected, this is a bug in 2.4.0. Until the problem is fixed, it can be worked around by setting spring.config.use-legacy-processing=true.

Related

jar with embedded tomcat, when using another spring project, is not working with just yml - it's needing a blank application.properties file as well

Been searching for others that have run into this issue, and not finding much out there, so it can't be that common.
I have a spring-boot project that I want to convert into a jar project, running with embedded tomcat. It's using yml files (application.yml and then the profile versions - eg appplication-dev.yml.) It ran fine as war with the yml files, however, when I convert it to a jar, and kick off the jar, the embedded tomcat never starts UNLESSS I add an empty application.properties file as well. (No errors just no Tomcat startup unless the empty application.properties file is added.)
I believe it's somehow related to one of our internal jar dependencies (also spring), since if I remove that dependency from the pom (and any of the code referencing it) I can get the jar to startup the embedded tomcat just fine (without providing the empty application.properties file.)
I could also, of course, forgo using yml files and just use .properties files, but I'd like to use yml files if possible. Why adding an empty applcation.properties file causes things to work has me stumped.
If it helps, the config in the dependency project that causes the issue we're seeing is set up as:
#Configuration
#EnableConfigurationProperties(OracleDataSourceProperties.class)
#EnableTransactionManagement
#ComponentScan(basePackages = {"com.foo.data.services","com.foo.data.domain", "com.foo.utility", "com.foo.cipher.utility"})
#MapperScan(value = {"com.foo.data.services.mapper","com.foo.data.services.batchmapper"})
public class DataServicesPersistenceConfig { ... }
and the OracleDataSourceProperties class:
#ConfigurationProperties(prefix="oradb", ignoreUnknownFields = true)
public class OracleDataSourceProperties extends BaseVO implements InitializingBean{

How to use #Value in Junit tests

I'm trying to get a property value from my application-test.yml file like that :
#SpringBootTest()
#ActiveProfiles("test")
#ExtendWith(MockitoExtension.class)
public class GoogleServiceUtilsTest {
#Value("${google.service.account.user}")
private String serviceAccountUser;
#Value("${google.service.account.path}")
private String pathFile;
Set<String> scopesSet = DirectoryScopes.all();
List<String> scopesList = new ArrayList<String>(scopesSet);
#Test
void getCredentialTest() throws GoogleCredentialException {
// Given
GoogleCredentials credentials;
// When
credentials = GoogleServiceUtils.getCredential(serviceAccountUser, scopesList, pathFile);
// Then
assertThat(credentials != null);
assertThat(new HttpCredentialsAdapter(credentials));
}
}
but when I use it in my test method, serviceAccountUser and pathFile variables are always null.
My application-test.yml file is located in 'src/test/resources', and tests are in 'src/test/java/' and content :
google:
service:
account:
user: ...
path: ...
# FED credentials
fed:
url: ...
token: ...
grantType: ...
# Logging
logging:
level:
root: ...
org.springframework: ...
When I use the #Value tag in my app code, all is working. Variables are getting good values from 'src/main/resources/application.yml' file.
After reading comments, I add that the profiles are not useful in my case, but as when I don't use them, it doesn't work, I thought that maybe it came from there and that it is necessary to use them.
I also tried to write the value of the variables hard in the file, and then there is no problem, tests are passing well.
Is someone understanding this problem? Many people seem to have had the same, but I can't find an answer working for me.
Thank you for your answer!
Sorry, cannot reproduce!
Having:
Simplest starter
src/main/resources/application.yml
foo:
bar:
baz: normal
...and
src/test/resources/application-test.yml
foo:
bar:
baz: test
This test passes:
#SpringBootTest
#ActiveProfiles("test") // !
class SomeTest {
#Value("${foo.bar.baz}")
String foo;
#Test
void testProp() {
Assertions.assertEquals("test", foo); //!#
}
}
Either #ExtendWith(MockitoExtension.class) has no effect/does no harm.
Whenever you wonder, where your properties come from
Consult:https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config
And check (current version):
...considered in the following order:
Default properties (specified by setting SpringApplication.setDefaultProperties).
#PropertySource annotations on your #Configuration classes. Please note ...
Config data (such as application.properties files).
A RandomValuePropertySource that has properties only in random.*.
OS environment variables.
Java System properties (System.getProperties()).
JNDI attributes from java:comp/env.
ServletContext init parameters.
ServletConfig init parameters.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).
Command line arguments.
properties attribute on your tests. Available on #SpringBootTest and the test annotations for testing a particular slice of your application.
#TestPropertySource annotations on your tests.
Devtools global settings properties in the $HOME/.config/spring-boot directory when devtools is active.
Config data files (3.) are considered in the following order:
Application properties packaged inside your jar (application.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).

Give external path in #Value Spring annotation and Resource

In spring boot application how do I give an external windows path using #Value Spring annotation and Resource
The below example works fine that look into resources folder but I want to give the path outside of application like c:\data\sample2.csv
#Value("classPath:/sample2.csv")
private Resource inputResource;
...
#Bean
public FlatFileItemReader<Employee> reader() {
FlatFileItemReader<Employee> itemReader = new FlatFileItemReader<Employee>();
itemReader.setLineMapper(lineMapper());
itemReader.setLinesToSkip(1);
itemReader.setResource(inputResource);
and if I want to get the value from properties file in annotaion, whats the format to put the path in windows?
i tried these, none of them worked:
in code
#Value("${inputfile}")
in properties file:
inputfile="C:\Users\termine\dev\sample2.csv"
inputfile="\\C:\\Users\\termine\\dev\\sample2.csv"
inputfile="C:/Users/termine/dev/sample2.csv"
inputfile="file:\\C:\Users\termine\dev\sample2.csv"
inputfile="file://C://Users//termine///dev//sample2.csv"
When you use classpath spring will try to search with the classpath even if you provide the outside file path.
so instead of using classpath: you can use file:
Ex.
#Value("file:/sample2.csv") //provide full file path if any
Use the key spring.config.location in properties to set the config location. Spring-boot will by default load properties from the locations, with precedence like below :
A /config subdir of the current directory.
The current directory
A classpath /config package
The classpath root
and apart from this when you start the jar or in application.properties you can provide the location of the config file like :
$ java -jar myproject.jar --spring.config.location=classpath:/default.properties,classpath:/override.properties
You can serve static files from the local disk, by making the resource(s) "sample2.csv" as a static resource. An easy way to do this is by adding spring.resources.static-locations configuration to your applicaiton.properties file. Example:
spring.resources.static-locations=file:///C:/Temp/whatever/path/sample2.csv",classpath:/static-files, classpath:/more-static-resource
When I did this in one of the projects, I was able to access the file form the browser using localhost:8080/sample2.csv.

Spring boot: populate h2 db from schema in test/resources

On my local machine I load an in-memory h2 database to start my spring boot application in a safe environment, here's the properties:
spring.datasource.url: jdbc:h2:mem:DB_TEST;Mode=Oracle
spring.datasource.platform: h2
spring.jpa.hibernate.ddl-auto: none
spring.datasource.continue-on-error: false
spring.jpa.database-platform: org.hibernate.dialect.Oracle10gDialect
Then, in my src/main/resources I have the file schema-h2.sql containing my local db initiations.
That's fine, but then I also have some junit tests I want to execute:
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyTest {
#Autowired
private MyController controller;
#Test
public void myTest(){
controller.doSomething();
}
This is also fine, as the schema-h2.sql is seen.
Anyway according to me it would be better to put the schema-h2.sql in src/test/resources as it has to be used only on my local environment. Doing so also allows maven to exclude it from the final build and that is also pretty fine.
Anyway if I put it there the test keeps working...but the main application breaks as the schema-h2.sql is not found!
How to modify the above properties to specify that the shema-h2.sql has to be searched inside of the test/resources folder?
Thanks
For normal mode, the properties file is put in src/main/resources,
and for the testing method, the properties file in the src/test/resources folder.
By Trying to run a test-class, eclipse runs EACH file ending with .sql (and thus containing a script to create tables or to insert data) it finds under src/main/resources and src/test/resources.
So if you put a script-file schema.sql (containing a script that creates a table: create table ..) in both folders, you'll get an "table already exits" error, if you let jut one, the test will run smoothly.
If you put a script-file (that insert data in a table) in both folder, both scripts will be run.
You can also use the #PropertySource("..") in your repository to tell spring where to find the properties-file to use.

Spring boot on Tomcat with external configuration

I can't find an answer to this question on stackoverflow hence im asking here so I could get some ideas.
I have a Spring Boot application that I have deployed as a war package on Tomcat 8. I followed this guide Create a deployable war file which seems to work just fine.
However the issue I am currently having is being able to externalize the configuration so I can manage the configuration as puppet templates.
In the project what I have is,
src/main/resources
-- config/application.yml
-- config/application.dev.yml
-- config/application.prod.yml
-- logback-spring.yml
So how can I possibly load config/application.dev.yml and config/application.prod.yml externally and still keep config/application.yml ? (contains default properties including spring.application.name)
I have read that the configuration is load in this order,
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
Hence I tried to load the configuration files from /opt/apache-tomcat/lib to no avail.
What worked so far
Loading via export CATALINA_OPTS="-Dspring.config.location=/opt/apache-tomcat/lib/application.dev.yml"
however what I would like to know is,
Find out why loading via /opt/apache-tomcat/lib classpath doesn't work.
And is there a better method to achieve this ?
You are correct about load order. According to Spring boot documentation
SpringApplication will load properties from application.properties files in the following locations and add them to the Spring Environment:
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
The list is ordered by precedence (properties defined in locations higher in the list override those defined in lower locations).
[Note]
You can also use YAML ('.yml') files as an alternative to '.properties'.
This means that if you place your application.yml file to /opt/apache-tomcat/lib or /opt/apache-tomcat/lib/config it will get loaded.
Find out why loading via /opt/apache-tomcat/lib classpath doesn't work.
However, if you place application.dev.yml to that path, it will not be loaded because application.dev.yml is not filename Spring is looking for. If you want Spring to read that file as well, you need to give it as option
--spring.config.name=application.dev or -Dspring.config.name=application.dev.
But I do not suggest this method.
And is there a better method to achieve this ?
Yes. Use Spring profile-specific properties. You can rename your files from application.dev.yml to application-dev.yml, and give -Dspring.profiles.active=dev option. Spring will read both application-dev.yml and application.yml files, and profile specific configuration will overwrite default configuration.
I would suggest adding -Dspring.profiles.active=dev (or prod) to CATALINA_OPTS on each corresponding server/tomcat instance.
I have finally simplified solution for reading custom properties from external location i.e outside of the spring boot project. Please refer to below steps.
Note: This Solution created and executed windows.Few commands and folders naming convention may vary if you are deploying application on other operating system like Linux..etc.
1. Create a folder in suitable drive.
eg: D:/boot-ext-config
2. Create a .properties file in above created folder with relevant property key/values and name it as you wish.I created dev.properties for testing purpose.
eg :D:/boot-ext-config/dev.properties
sample values:
dev.hostname=www.example.com
3. Create a java class in your application as below
------------------------------------------------------
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.PropertySource;
#PropertySource("classpath:dev.properties")
#ConfigurationProperties("dev")
public class ConfigProperties {
private String hostname;
//setters and getters
}
--------------------------------------------
4. Add #EnableConfigurationProperties(ConfigProperties.class) to SpringBootApplication as below
--------------------------------------------
#SpringBootApplication
#EnableConfigurationProperties(ConfigProperties.class)
public class RestClientApplication {
public static void main(String[] args) {
SpringApplication.run(RestClientApplication.class, args);
}
}
---------------------------------------------------------
5. In Controller classes we can inject the instance using #Autowired and fetch properties
#Autowired
private ConfigProperties configProperties;
and access properties using getter method
System.out.println("**********hostName******+configProperties.getHostName());
Build your spring boot maven project and run the below command to start application.
-> set SPRING_CONFIG_LOCATION=<path to your properties file>
->java -jar app-name.jar

Resources