What's the best way to add a new property source in Spring? - spring

I'd like to add a new property source that could be used to read property values in an application. I'd like to do this using Spring. I have a piece of code like this in a #Configuration class:
#Bean
public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
PropertySourcesPlaceholderConfigurer properties = new PropertySourcesPlaceholderConfigurer();
MutablePropertySources sources = new MutablePropertySources();
MyCustomPropertySource propertySource = new MyCustomPropertySource("my custom property source");
sources.addFirst(propertySource);
properties.setPropertySources(sources);
return properties;
}
This seems to work pretty well. However, what it is also doing is overriding other property values (e.g. server.port property in application.properties file used by spring boot) which I don't want to overwrite. So the basic question is what's the best way to add this propertysource but not have it override other properties. Any way to grab the existing propertysources and simply add on to it?

I got this working by adding a custom initiailizer to my spring boot app:
#SpringBootApplication
public class MyApp {
public static void main(String[] args) {
new SpringApplicationBuilder(MyApp.class)
.initializers(new MyContextInitializer()) // <---- here
.run(args);
}
}
Where MyContextInitializer contains: -
public class MyContextInitializer implements
ApplicationContextInitializer<ConfigurableApplicationContext> {
public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
ConfigurableEnvironment environment = configurableApplicationContext.getEnvironment();
// Create map for properites and add first (important)
Map<String, Object> myProperties = new HashMap<>();
myProperties.put("some-prop", "custom-value");
environment.getPropertySources().addFirst(
new MapPropertySource("my-props", myProperties));
}
}
Note, if your application.yaml contains: -
some-prop: some-value
another-prop: this is ${some-prop} property
Then the initialize method will update the some-prop to custom-value and when the app loads it will have the following values at run-time:
some-prop: custom-value
another-prop: this is custom-value property
Note, if the initialize method did a simple System.setProperty call i.e.
public void initialize(ConfigurableApplicationContext configurableApplicationContext) {
ConfigurableEnvironment environment = configurableApplicationContext.getEnvironment();
System.setProperty("some-prop", "custom-value");
}
... then the another-prop would be equal to this is some-value property which is not what we generally want (and we lose the power of Spring config property resolution).

Try setting IgnoreUnresolvablePlaceholders to TRUE. I had a similar problem which I was able to resolve in this way. In my case, I had another placeholderconfigurer, which was working - but properties in the second one were not being resolved unless I set this property to TRUE.
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer = new PropertySourcesPlaceholderConfigurer();
propertySourcesPlaceholderConfigurer.setIgnoreUnresolvablePlaceholders(Boolean.TRUE);
propertySourcesPlaceholderConfigurer.setIgnoreResourceNotFound(Boolean.TRUE);
return propertySourcesPlaceholderConfigurer;
}

Yet another possibility (after lots of experimentation, it's what worked for me) would be to declare your PropertySource inside a ApplicationContextInitializer and then inject that one in your SpringBootServletInitializer:
public class MyPropertyInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final Logger logger = LoggerFactory.getLogger(ApplicationPropertyInitializer.class);
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
MyPropertySource ps = new MyPropertySource();
applicationContext.getEnvironment().getPropertySources().addFirst(ps);
}
}
public class MyInitializer extends SpringBootServletInitializer{
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return super.configure(builder.initializers(new MyPropertyInitializer()));
}
}

You can perhaps add your propertySource straight into environment once it is initialized.
EDIT: As this is done AFTER the class is processed, you cannot expect the #Value annotations to pick up anything from this particular PropertySource in the same #Configuration class - or any other that is loaded before.
#Configuration
public class YourPropertyConfigClass{
#Value("${fromCustomSource}")
String prop; // failing - property source not yet existing
#Autowired
ConfigurableEnvironment env;
#PostConstruct
public void init() throws Exception {
env.getPropertySources().addFirst(
new MyCustomPropertySource("my custom property source"));
}
}
#Configuration
#DependsOn("YourPropertyConfigClass")
public class PropertyUser {
#Value("${fromCustomSource}")
String prop; // not failing
}
You could move the #PostConstruct to a separate #Configuration class and mark other classes using those properties #DependOn("YourPropertyConfigClass") (this works - but perhaps there is a better way to force configuration order?).
Anyway - it is only worth it, if MyCustomPropertySource cannot be simply added using #PropertySource("file.properties") annotation - which would solve the problem for simple property files.

If you implement PropertySourceFactory as such:
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.support.EncodedResource;
import org.springframework.core.io.support.PropertySourceFactory;
public class CustomPropertySourceFactory implements PropertySourceFactory {
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource resource) {
...
}
}
You can use the following property source:
#PropertySource(name="custom-prop-source", value="", factory=CustomPropertySourceFactory.class)
Kind of hack-ish, but it works.

Related

Is there a possibility to group properties using Spring Boot?

Can something like this be accomplished using Spring Boot?
The idea is to group properties and assign the same value to all of them, so instead of all of the properties ending with 'test*' i would like to change just one property 'my.flag'. I know that such functionality works in case of loggers, but can I define my own group?
I am not sure whether your problem has been solved or not, but I want to provide a solution to achieve what you want by using spring.factories and implementing ApplicationListener as following steps.
STEP 1
Create a class MyPropertiesListener which implements ApplicationListener and read the value of my.flag in application.properties first, then set it to all the properties whose key starts with my.flag..
public class MyPropertiesListener implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
ConfigurableEnvironment env = event.getEnvironment();
String myFlag = env.getProperty("my.flag");
Properties props = new Properties();
MutablePropertySources propSrcs = env.getPropertySources();
StreamSupport.stream(propSrcs.spliterator(), false)
.filter(ps -> ps instanceof EnumerablePropertySource)
.map(ps -> ((MapPropertySource) ps).getPropertyNames())
.flatMap(Arrays::<String>stream)
.forEach(propName -> {
if (propName.toString().startsWith("my.flag.")) {
props.put(propName, myFlag);
}
});
env.getPropertySources().addFirst(new PropertiesPropertySource("myProps", props));
}
}
STEP 2
Create a file named spring.factories under src/main/resource/META-INF and configure MyPropertiesListener into it.
org.springframework.context.ApplicationListener=xxx.xxx.xxx.MyPropertiesListener
TEST
The value of my.flag.test3 is false in application.properties originally, but it is going to be overwritten as true while application starts.
#SpringBootApplication
public class Application implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
#Value("${my.flag.test3}")
private String test3;
#Override
public void run(String... args) throws Exception {
System.out.println(test3); //true
}
}
See also
Creating a Custom Starter with Spring Boot

How can i externalize datasource configuration with spring-boot?

I'm currently trying to move an existing spring-application to spring-boot and therefore recreate things that worked without boot.
I want to configure some properties (like spring.datasource.*) from an external source. specificly a folder with several properties files.
I set up a configuration class that creates propertyPlaceholder configurers like this:
#Configuration
public class PropertySourceConfiguration {
#Bean
public static PropertySourcesPlaceholderConfigurer defaultsPlaceHolderConfigurer() throws IOException {
PropertySourcesPlaceholderConfigurer propertyConfigurer = new PropertySourcesPlaceholderConfigurer();
propertyConfigurer.setLocations(new PathMatchingResourcePatternResolver().getResources("classpath*:/*-defaults.properties"));
propertyConfigurer.setIgnoreUnresolvablePlaceholders(true);
return propertyConfigurer;
}
#Bean
public static PropertySourcesPlaceholderConfigurer externalPlaceHolderConfigurer() throws IOException {
PropertySourcesPlaceholderConfigurer propertyConfigurer = new PropertySourcesPlaceholderConfigurer();
propertyConfigurer.setLocations(new
PathMatchingResourcePatternResolver().getResources("file:/my-config-path/*.properties"));
propertyConfigurer.setOrder(1);
propertyConfigurer.setIgnoreUnresolvablePlaceholders(true);
return propertyConfigurer;
}
This seems to work for most things (like amqp or my own config properties) but when i try to use spring-data-jpa they are ignored. basicly setting spring.datasource.url (and other things used for auto-config) in those files has no effect.
looking through the logs of the PropertySourcesPropertyResolver i figured out that these configurer fall under the localProperties group which is not used when looking for spring.datasource.*.
is there a way to fix this or a better way to add external properties files to my context?
I know i could set spring.config.location to do something similar but i can not pass command-line properties to my application and need to do this config from within my application. afaik this is not possible with this property.
EDIT: setting spring.config.location:
Attempt 1:
public class ServletInitializer extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(CampaignServiceStarter.class);
}
#Override
public void onStartup(ServletContext servletContext) throws ServletException {
super.onStartup(servletContext);
servletContext.setInitParameter("spring.config.location", "file:/my-config-path/*.properties");
}
}
Attempt 2:
public class ServletInitializer extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(CampaignServiceStarter.class).properties("spring.config.location=file:/my-config-path/*.properties");
}
}
in both cases the external properties were not picked up at all (even in places where it worked before, like the amqp config)
How to use external configuration is explained in this section of the Spring Boot Reference Guide.
The spring.config.location is a path to the directory which contains your application.properties file. It takes a comma separated list of values so you could specify multiple paths. It doesn't take the wildcard. It is a path so not an expression to match multiple property files. If you want to change the default application then use the spring.config.name to make it something else.
The defaults of Spring Boot are opinionated as the rest of Spring Boot (with the default configuration etc.).
If you want to do more extensive (pre) configuration you should use an ApplicationContextInitializer to manually add the PropertySources to the Environment. This is mentioned here in the Spring Boot Reference Guide.
An example of how the initializer might look.
public class ConfigurationInitializer implements ApplicationContextInitializer {
private static final String DEFAULT_PROPS = "classpath*:/*-defaults.properties";
private static final String EXTERNAL_PROPS = "file:/my-config-path/*.properties";
public void initialize(ConfigurableApplicationContext applicationContext) {
final Resource[] defaultConfigs = applicationContext.getResources(DEFAULT_PROPS);
final Resource[] externalConfigs = applicationContext.getResources(EXTERNAL_PROPS);
final ConfigurableEnvironment env = applicationContext.getEnvironment();
final MutablePropertySources mps = env.getPropertySources();
for (Resource r : externalConfigs) {
mps.addLast(new ResourcePropertySource(r.getFilename(), r);
}
for (Resource r : defaultConfigs) {
mps.addLast(new ResourcePropertySource(r.getFilename(), r);
}
}
}
Then when building your application object add it as follows.
public class ServletInitializer extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(CampaignServiceStarter.class)
.initializers(new ConfigurationInitializer());
}
}
Now the configs should be added to the list of property sources.

Properties in Spring Environment

I'm trying to create a custom property placeholder.
At my Spring Configuration Class I have this:
#Configuration
#Import(ConfigurationClass2.class)
public class ConfigurationClass1 {
#Bean
#Lazy(false)
public static PropertySourcesPlaceholderConfigurer settings() throws IOException {
// custom load of my properties file
// legacy configuration, comes from a SVN repository
// I need to download it and then do some logic and finally
// load it.
}
}
At another configuration class I want to access some properties using the Environment, like this:
#Configuration
#EnableScheduling
public class ConfigurationClass2 {
#Autowired
private Environment env;
#Bean
public DataSource dataSource() {
String jdbcUrl = env.getProperty('jdbc.url');
.... // jdbcUrl is null!!!!
}
#Scheduled(cron = "${cronExpression}")
public void worker() {
}
}
I don't want to use #Value or #PropertySource.
I need to load this properties manually and access this values programmatically!
How can I solve this?

Can #PropertySources be chosen by Spring profile?

I have a Spring 3.1 #Configuration that needs a property foo to build a bean. The property is defined in defaults.properties but may be overridden by the property in overrides.properties if the application has an active override Spring profile.
Without the override, the code would look like this, and work...
#Configuration
#PropertySource("classpath:defaults.properties")
public class MyConfiguration {
#Autowired
private Environment environment;
#Bean
public Bean bean() {
...
// this.environment.getRequiredProperty("foo");
...
}
}
I would like a #PropertySource for classpath:overrides.properties contingent on #Profile("overrides"). Does anyone have any ideas on how this could be achieved? Some options I've considered are a duplicate #Configuration, but that would violate DRY, or programmatic manipulation of the ConfigurableEnvironment, but I'm not sure where the environment.getPropertySources.addFirst() call would go.
Placing the following in an XML configuration works if I inject the property directly with #Value, but not when I use Environment and the getRequiredProperty() method.
<context:property-placeholder ignore-unresolvable="true" location="classpath:defaults.properties"/>
<beans profile="overrides">
<context:property-placeholder ignore-unresolvable="true" order="0"
location="classpath:overrides.properties"/>
</beans>
Update
If you're trying to do this now, check out Spring Boot's YAML support, particularly the 'Using YAML instead of Properties' section. The profile support there would make this question moot, but there isn't #PropertySource support yet.
Add the overriding #PropertySource in a static inner class. Unfortunately, you must specify all property sources together which means creating a "default" profile as the alternative to "override".
#Configuration
public class MyConfiguration
{
#Configuration
#Profile("default")
#PropertySource("classpath:defaults.properties")
static class Defaults
{ }
#Configuration
#Profile("override")
#PropertySource({"classpath:defaults.properties", "classpath:overrides.properties"})
static class Overrides
{
// nothing needed here if you are only overriding property values
}
#Autowired
private Environment environment;
#Bean
public Bean bean() {
...
// this.environment.getRequiredProperty("foo");
...
}
}
I suggest, defining two files, where the second is optional with the profile as suffix:
#Configuration
#PropertySources({
#PropertySource("classpath:/myconfig.properties"),
#PropertySource(value = "classpath:/myconfig-${spring.profiles.active}.properties", ignoreResourceNotFound = true)
})
public class MyConfigurationFile {
#Value("${my.prop1}")
private String prop1;
#Value("${my.prop2}")
private String prop2;
}
You can do:
<context:property-placeholder location="classpath:${spring.profiles.active}.properties" />
Edit: if you need something more advanced, you can register your PropertySources on application startup.
web.xml
<context-param>
<param-name>contextInitializerClasses</param-name>
<param-value>com.xxx.core.spring.properties.PropertySourcesApplicationContextInitializer</param-value>
</context-param>
file you create:
public class PropertySourcesApplicationContextInitializer implements ApplicationContextInitializer<ConfigurableApplicationContext> {
private static final Logger LOGGER = LoggerFactory.getLogger(PropertySourcesApplicationContextInitializer.class);
#Override
public void initialize(ConfigurableApplicationContext applicationContext) {
LOGGER.info("Adding some additional property sources");
String[] profiles = applicationContext.getEnvironment().getActiveProfiles()
// ... Add property sources according to selected spring profile
// (note there already are some property sources registered, system properties etc)
applicationContext.getEnvironment().getPropertySources().addLast(myPropertySource);
}
}
Once you've done it you just need to add in your context:
<context:property-placeholder/>
I can't really answer to your question about multiple profiles but I guess you activate them on such an initializer, and you could register the appropriate PropertySource items during profile activations.
I can't think of any other way than one you have suggested Emerson, which is to define this bean in a separate #Configuration file with an #Profile annotation:
#Configuration
#Profile("override")
#PropertySource("classpath:override.properties")
public class OverriddenConfig {
#Autowired
private Environment environment;
#Bean
public Bean bean() {
//if..
}
}
In case you need to support multiple profiles you could do something like this:
#Configuration
public class Config {
#Configuration
#Profile("default")
#PropertySource("classpath:application.properties")
static class DefaultProperties {
}
#Configuration
#Profile("!default")
#PropertySource({"classpath:application.properties", "classpath:application-${spring.profiles.active}.properties"})
static class NonDefaultProperties {
}
}
That way you don't need to define a static configuration class for each profile.
Thanks David Harkness for putting me into the right direction.
Note: This answer provides an alternate solution to using properties files with #PropertySource. I went this route because it was too cumbersome trying to work with multiple properties files that may each have overrides while avoiding repetitive code.
Create a POJO interface for each related set of properties to define their names and types.
public interface DataSourceProperties
{
String driverClassName();
String url();
String user();
String password();
}
Implement to return the default values.
public class DefaultDataSourceProperties implements DataSourceProperties
{
public String driverClassName() { return "com.mysql.jdbc.Driver"; }
...
}
Subclass for each profile (e.g. development, production) and override any values that differ from the default. This requires a set of mutually-exclusive profiles, but you can easily add "default" as the alternative to "overrides".
#Profile("production")
#Configuration
public class ProductionDataSourceProperties extends DefaultDataSourceProperties
{
// nothing to override as defaults are for production
}
#Profile("development")
#Configuration
public class DevelopmentDataSourceProperties extends DefaultDataSourceProperties
{
public String user() { return "dev"; }
public String password() { return "dev"; }
}
Finally, autowire the properties configurations into the other configurations that need them. The advantage here is that you don't repeat any #Bean creation code.
#Configuration
public class DataSourceConfig
{
#Autowired
private DataSourceProperties properties;
#Bean
public DataSource dataSource() {
BoneCPDataSource source = new BoneCPDataSource();
source.setJdbcUrl(properties.url());
...
return source;
}
}
I am still not convinced I'll stick with this over manually configuring properties files based on the active profiles in a servlet context initializer. My thought was that doing manual configuration would not be as amenable to unit testing, but I'm not so sure now. I really prefer reading properties files to a list of property accessors.
All mentioned here solutions are a bit awkward, work only with one profile preset, and they won't work with more/other profiles. Currently a Spring team refuses to introduce this feature. But here's the working workaround I've found:
package com.example;
public class MyPropertySourceFactory implements PropertySourceFactory, SpringApplicationRunListener {
public static final Logger logger = LoggerFactory.getLogger(MyPropertySourceFactory.class);
#NonNull private static String[] activeProfiles = new String[0];
// this constructor is used for PropertySourceFactory
public MyPropertySourceFactory() {
}
// this constructor is used for SpringApplicationRunListener
public MyPropertySourceFactory(SpringApplication app, String[] params) {
}
#Override
public void environmentPrepared(ConfigurableBootstrapContext bootstrapContext, ConfigurableEnvironment environment) {
activeProfiles = environment.getActiveProfiles();
}
#Override
public PropertySource<?> createPropertySource(String name, EncodedResource encodedResource) throws IOException {
logger.info("Loading: {} with profiles: {}", encodedResource.toString(), activeProfiles);
// here you know all profiles and have the source Resource with main
// properties, just try to load other resoures in the same path with different
// profile names and return them as a CompositePropertySource
}
}
To make it working you have to have src/main/resources/META-INF/spring.factories with the following content:
org.springframework.boot.SpringApplicationRunListener=com.example.MyPropertySourceFactory
Now you can put your custom properties file somewhere and load it with #PropertySources:
#Configuration
#PropertySource(value = "classpath:lib.yml", factory = MyPropertySourceFactory.class)
public class PropertyLoader {
}

Custom spring property source does not resolve placeholders in #Value

I'm trying to build a Spring 3.1 PropertySource which reads its values from Zookeeper nodes. For connecting to Zookeeper I am using Curator from Netflix.
For that I've built a custom property source which reads the value of a property from Zookeeper and returns it. This works fine when I am resolving the property like this
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
However, when I try to instantiate a bean which has a field with an #Value annotation then this fails:
#Component
public class MyBean {
#Value("${foo}") public String foo;
}
MyBean b = ctx.getBean(MyBean.class); // fails with BeanCreationException
This problem has most likely nothing to do with Zookeeper but with the way I'm registering the property sources and creating the beans.
Any insight is highly appreciated.
Update 1:
I'm creating the app context from an XML file like this:
public class Main {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
ctx.registerShutdownHook();
}
}
The class which connects to Zookeeper is a #Component.
#Component
public class Server {
CuratorFramework zkClient;
public void connectToZookeeper() {
zkClient = ... (curator magic) ...
}
public void registerPropertySource() {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
}
#PostConstruct
public void start() {
connectToZookeeper();
registerPropertySource();
MyBean b = ctx.getBean(MyBean.class);
}
}
Update 2
This seems to work when I'm using XML-less configuration, i.e. #Configuration, #ComponentScan and #PropertySource in combination with an AnnotationConfigApplicationContext. Why doesn't it work with a ClassPathXmlApplicationContext?
#Configuration
#ComponentScan("com.goleft")
#PropertySource({"classpath:config.properties","classpath:version.properties"})
public class AppConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Answering to your Update 2: This does not work with your original configuration(registering a PropertySource using #PostConstruct) because the PropertySource is being registered very late, by this time your target bean has already been constructed and initialized.
Typically the injection of the placeholders happens via a BeanFactoryPostProcessor which is very early in the Spring lifecycle(beans have not been created at this stage) and if a PropertySource is registered at that stage, then placeholders should be resolved.
The best approach though is to use a ApplicationContextInitializer, get a handle on the applicationContext and to register the propertySource there:
public class CustomInitializer implements ApplicationContextInitializer<ConfigurableWebApplicationContext> {
public void initialize(ConfigurableWebApplicationContext ctx) {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addFirst(zkPropertySource);
}
}

Resources