Cannot load config from custom yml file in spring boot application - spring-boot

I am loading the custom configuration from application.yml in my spring boot service.
I have annotated by bean class as below,
#Component
#ConfigurationProperties("app")
public class ContinentConfig {
private Map<String, List<Country>> continents = new HashMap<String, List<Country>>();
//get/set/tostring methods
}
My custom class Country includes 2 fields,
public class Country {
String name;
String capital;
//get/set/tostring methods
}
In application.yml I have as below,
app:
continents:
Europe:
- name: France
capital: Paris
Asia:
- name: China
capital: Beijing
With the above setup, I am able to load the config from application.yml.
I now want to extract the config to a separate continentconfig.yml in the same src/main/resources folder. So, I moved the custom config to continentconfig.yml leaving other properties like server.port in application.yml.
The continentconfig.yml has the same content as I had earlier in application.yml.
I also added the below annotations to the ContinentConfig Class,
#Component
#ConfigurationProperties("app")
#EnableConfigurationProperties
#PropertySource(value="classpath:continentconfig.yml")
public class ContinentConfig {
}
After this change, I see the config is not getting loaded from continentconfig.yml to ContinentConfig bean.
Can someone please help in resolving the issue.

The short answer you can't do that and you should use the property file.
24.6.4 YAML shortcomings
YAML files can’t be loaded via the #PropertySource annotation. So in
the case that you need to load values that way, you need to use a
properties file.
You can create your initializer and use the YamlPropertySourceLoader.

I believe by externalizing you mean using a property file to load the configuration from a file configured in github/or other such hosting repos? You could very well do that by using bootstrap.yml. This loads all the configurations from an external file and also allows the provision to override them using the local application.properties/application.yml
spring:
application:
name:
cloud :
config :
uri :
Also make sure that you have spring cloud in your pom to resolve this just in case
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter</artifactId>
</dependency>
Just incase if your local yml property files are not loaded in your classpath, then add the following
<resources>
<resource>
<directory>src/main/resources</directory>
<includes>
<include>**/*.properties</include>
<include>**/*.yml</include>
<include>**/*.jks</include>
</includes>
</resource>
</resources>
Note: It is preferred to have the YamlPropertySourceLoader to load your config file to your classpath and on top of that you could use the above config

Related

How to use #Value in Junit tests

I'm trying to get a property value from my application-test.yml file like that :
#SpringBootTest()
#ActiveProfiles("test")
#ExtendWith(MockitoExtension.class)
public class GoogleServiceUtilsTest {
#Value("${google.service.account.user}")
private String serviceAccountUser;
#Value("${google.service.account.path}")
private String pathFile;
Set<String> scopesSet = DirectoryScopes.all();
List<String> scopesList = new ArrayList<String>(scopesSet);
#Test
void getCredentialTest() throws GoogleCredentialException {
// Given
GoogleCredentials credentials;
// When
credentials = GoogleServiceUtils.getCredential(serviceAccountUser, scopesList, pathFile);
// Then
assertThat(credentials != null);
assertThat(new HttpCredentialsAdapter(credentials));
}
}
but when I use it in my test method, serviceAccountUser and pathFile variables are always null.
My application-test.yml file is located in 'src/test/resources', and tests are in 'src/test/java/' and content :
google:
service:
account:
user: ...
path: ...
# FED credentials
fed:
url: ...
token: ...
grantType: ...
# Logging
logging:
level:
root: ...
org.springframework: ...
When I use the #Value tag in my app code, all is working. Variables are getting good values from 'src/main/resources/application.yml' file.
After reading comments, I add that the profiles are not useful in my case, but as when I don't use them, it doesn't work, I thought that maybe it came from there and that it is necessary to use them.
I also tried to write the value of the variables hard in the file, and then there is no problem, tests are passing well.
Is someone understanding this problem? Many people seem to have had the same, but I can't find an answer working for me.
Thank you for your answer!
Sorry, cannot reproduce!
Having:
Simplest starter
src/main/resources/application.yml
foo:
bar:
baz: normal
...and
src/test/resources/application-test.yml
foo:
bar:
baz: test
This test passes:
#SpringBootTest
#ActiveProfiles("test") // !
class SomeTest {
#Value("${foo.bar.baz}")
String foo;
#Test
void testProp() {
Assertions.assertEquals("test", foo); //!#
}
}
Either #ExtendWith(MockitoExtension.class) has no effect/does no harm.
Whenever you wonder, where your properties come from
Consult:https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.external-config
And check (current version):
...considered in the following order:
Default properties (specified by setting SpringApplication.setDefaultProperties).
#PropertySource annotations on your #Configuration classes. Please note ...
Config data (such as application.properties files).
A RandomValuePropertySource that has properties only in random.*.
OS environment variables.
Java System properties (System.getProperties()).
JNDI attributes from java:comp/env.
ServletContext init parameters.
ServletConfig init parameters.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property).
Command line arguments.
properties attribute on your tests. Available on #SpringBootTest and the test annotations for testing a particular slice of your application.
#TestPropertySource annotations on your tests.
Devtools global settings properties in the $HOME/.config/spring-boot directory when devtools is active.
Config data files (3.) are considered in the following order:
Application properties packaged inside your jar (application.properties and YAML variants).
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants).
Application properties outside of your packaged jar (application.properties and YAML variants).
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants).

How can I load additional properties file in a spring boot application packaged as a war?

I have a standard springboot web app. I want to load properties file that's not in the classpath. application.properties is in the classpath and is being read correctly.
I don't have an issue when I'm building a jar. I just put the .properties alongside the jar and it works. But when I package a war, I couldn't get the application to read the properties file .
You put the properties file parallel to application.properties file and load it in a class like this:
#PropertySource("classpath:foo.properties")
public class My class {
#Value( "${some.property}" )
String myProp;
}
You can using ClassPathResource. given Class for loading resources.
This is sample code for you
ClassPathResource resource = new ClassPathResource("/application/context/blabla.yml");
InputStream inputStream = resource.getInputStream();
File file = resource.getFile();
// using inputStream or file
ClassPathResource
You can use spring application.properties to have spring profiles and use the spring profiles to define separate properties for each environment as you like.You can even separate out the spring profiles in to different files like appication-dev.properties etc so that you can have each spring profile in different files.
You can read the properties by using #Configuration annotation :
#Configuration
#EnableConfigurationProperties(TestProperties.class)
public class MySampleConfiguration {
}
Here TestProperties.class is used to map the values from the property file or yaml .
Reference in detail : https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html

SpringBoot: Separate property files

I've created my application.properties file:
spring.config.additional-location=C:\Users\user\
spring.datasource.url=jdbc:postgresql://<db>:<port>/<db>
I need to feed Spring with an additional file located on C:\Users\user\application.properties:
spring.datasource.username=user
spring.datasource.password=password
As you can see I've tried to use spring.config.additional-location property into my application.properties file.
However, bootstrap tells me that no authentication has been provided.
you can use another file name by specifying a spring.config.name environment property. You can also refer to an explicit location by using the spring.config.location environment property (which is a comma-separated list of directory locations or file paths). The following example shows how to specify a different file name:
$ java -jar myproject.jar --spring.config.name=myproject
Reference URL:
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
You can specify your alternative properties using #PropertySources like this:
#PropertySources({
#PropertySource({"classpath:application.properties"}),
#PropertySource(value = {"file:${conf-dir}}/application-override.properties" },ignoreResourceNotFound = true)
})
public class AppConfig {
...
The properties in the bottom PropertySource will override properties from first one, if the file exists.
The documentation says to use file:C:/Users/user/
the "file:" or "classpath:" part is important.
There are numerous other ways to do this, too.
- profiles (and per profile application-<profile>.properties)
- #Configuration + #PropertySource
- ...

Spring boot on Tomcat with external configuration

I can't find an answer to this question on stackoverflow hence im asking here so I could get some ideas.
I have a Spring Boot application that I have deployed as a war package on Tomcat 8. I followed this guide Create a deployable war file which seems to work just fine.
However the issue I am currently having is being able to externalize the configuration so I can manage the configuration as puppet templates.
In the project what I have is,
src/main/resources
-- config/application.yml
-- config/application.dev.yml
-- config/application.prod.yml
-- logback-spring.yml
So how can I possibly load config/application.dev.yml and config/application.prod.yml externally and still keep config/application.yml ? (contains default properties including spring.application.name)
I have read that the configuration is load in this order,
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
Hence I tried to load the configuration files from /opt/apache-tomcat/lib to no avail.
What worked so far
Loading via export CATALINA_OPTS="-Dspring.config.location=/opt/apache-tomcat/lib/application.dev.yml"
however what I would like to know is,
Find out why loading via /opt/apache-tomcat/lib classpath doesn't work.
And is there a better method to achieve this ?
You are correct about load order. According to Spring boot documentation
SpringApplication will load properties from application.properties files in the following locations and add them to the Spring Environment:
A /config subdirectory of the current directory.
The current directory
A classpath /config package
The classpath root
The list is ordered by precedence (properties defined in locations higher in the list override those defined in lower locations).
[Note]
You can also use YAML ('.yml') files as an alternative to '.properties'.
This means that if you place your application.yml file to /opt/apache-tomcat/lib or /opt/apache-tomcat/lib/config it will get loaded.
Find out why loading via /opt/apache-tomcat/lib classpath doesn't work.
However, if you place application.dev.yml to that path, it will not be loaded because application.dev.yml is not filename Spring is looking for. If you want Spring to read that file as well, you need to give it as option
--spring.config.name=application.dev or -Dspring.config.name=application.dev.
But I do not suggest this method.
And is there a better method to achieve this ?
Yes. Use Spring profile-specific properties. You can rename your files from application.dev.yml to application-dev.yml, and give -Dspring.profiles.active=dev option. Spring will read both application-dev.yml and application.yml files, and profile specific configuration will overwrite default configuration.
I would suggest adding -Dspring.profiles.active=dev (or prod) to CATALINA_OPTS on each corresponding server/tomcat instance.
I have finally simplified solution for reading custom properties from external location i.e outside of the spring boot project. Please refer to below steps.
Note: This Solution created and executed windows.Few commands and folders naming convention may vary if you are deploying application on other operating system like Linux..etc.
1. Create a folder in suitable drive.
eg: D:/boot-ext-config
2. Create a .properties file in above created folder with relevant property key/values and name it as you wish.I created dev.properties for testing purpose.
eg :D:/boot-ext-config/dev.properties
sample values:
dev.hostname=www.example.com
3. Create a java class in your application as below
------------------------------------------------------
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.PropertySource;
#PropertySource("classpath:dev.properties")
#ConfigurationProperties("dev")
public class ConfigProperties {
private String hostname;
//setters and getters
}
--------------------------------------------
4. Add #EnableConfigurationProperties(ConfigProperties.class) to SpringBootApplication as below
--------------------------------------------
#SpringBootApplication
#EnableConfigurationProperties(ConfigProperties.class)
public class RestClientApplication {
public static void main(String[] args) {
SpringApplication.run(RestClientApplication.class, args);
}
}
---------------------------------------------------------
5. In Controller classes we can inject the instance using #Autowired and fetch properties
#Autowired
private ConfigProperties configProperties;
and access properties using getter method
System.out.println("**********hostName******+configProperties.getHostName());
Build your spring boot maven project and run the below command to start application.
-> set SPRING_CONFIG_LOCATION=<path to your properties file>
->java -jar app-name.jar

Can I start a Spring Boot WAR with PropertiesLauncher?

I have a Spring Boot 1.2 app packaged as a WAR because I need to be able to deploy the app in an app server.
I also want to configure an external path which will contain jars to be added to the classpath. After reading the Launcher documentation, I configured the build to use PropertiesLauncher to this end :
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
...
<layout>ZIP</layout>
</configuration>
</plugin>
I tried to start the app with various combinations of this additional system property : -Dloader.path=lib/,lib-provided/,WEB-INF/classes,<my additional path>
But I always end up with this error :
java.lang.IllegalArgumentException: Invalid source folder C:\<path to my war>\<my war>.war
at org.springframework.boot.loader.archive.ExplodedArchive.<init> ExplodedArchive.java:78)
at org.springframework.boot.loader.archive.ExplodedArchive.<init>(ExplodedArchive.java:66)
at org.springframework.boot.loader.PropertiesLauncher.addParentClassLoaderEntries(PropertiesLauncher.java:530)
at org.springframework.boot.loader.PropertiesLauncher.getClassPathArchives(PropertiesLauncher.java:451)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:60)
at org.springframework.boot.loader.PropertiesLauncher.main(PropertiesLauncher.java:609)
I looked at the source code and it seems that PropertiesLauncher can only handle jar archives (ending with ".jar" or ".zip") and "exploded archives" (not ending with the former)
Is it possible to do achieve what I want ? Am I doing it wrong ?
If it's not possible, which alternative is there ?
If somebody end up here this might be useful:
java -cp yourSpringBootWebApp.war -Dloader.path=yourSpringBootWebApp.war!/WEB-INF/classes/,yourSpringBootWebApp.war!/WEB-INF/,externalLib.jar org.springframework.boot.loader.PropertiesLauncher
(Spring-Boot 1.5.9)
https://docs.spring.io/spring-boot/docs/1.5.x/reference/html/executable-jar.html#executable-jar-launching
In Spring Boot 1.2, PropertiesLauncher handles .jar and .zip files as "jar archives" and everything else as "exploded archives" (unzipped jars). It does not properly handles .war
Here's the alternative I found :
I eventually switched back to the regular war launcher and I managed to configure a folder which jar contents are added to the classpath using a SpringApplicationRunListener such as this (pseudo-code for concision) :
public class ClasspathExtender implements SpringApplicationRunListener {
public void contextPrepared(ConfigurableApplicationContext context) {
// read jars folder path from environment
String path = context.getEnvironment().getProperty("my.jars-folder");
// enumerate jars in the folder
File[] files = new File(path).listFiles(new FilenameFilter() {
public boolean accept(File dir, String name) { return name.endsWith(".jar"); }
});
URL[] urls = // convert files array to urls array
// create a new classloader which contains the jars...
ClassLoader extendedClassloader = new URLClassLoader(urls, context.getClassLoader());
// and replace the context's classloader
((DefaultResourceLoader) context).setClassLoader(extendedClassloader);
}
// other methods are empty
}
This listener is instanciated by declaring it in a META-INF/spring.factories file :
org.springframework.boot.SpringApplicationRunListener=my.ClasspathExtender
This worked for me (Spring Boot 1.3.2)
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
...
<layout>WAR</layout>
</configuration>
</plugin>

Resources