Remote PropertySource - spring

Has anyone had any luck constructing a PropertySource that uses a remote source (for example a database) from which to retrieve property values. The idea would be to construct a PropertySource (needs some connection information such as host/port) and plug that into a PropertySourcePlaceholderConfigurer.
The problem seems to be a chicken and egg problem. How can I get the connection information down to the PropertySource? I could first instantiate the PropertySourcePlaceholderConfigurer with configuration to load a property file with the remote host and port properties and then later instantiate the PropertySource and inject that back into the configurer. However, I can't seem to figure a way to ensure that the very first bean to be instantiated (and quickly injected into the configurer) is my property source. I need to have this because, of course, all my other beans depend on the remote properties.

Commons Configuration supports loading properties from a variety of sources (including JDBC Datasources) into a org.apache.commons.configuration.Configuration object via a org.apache.commons.configuration.ConfigurationBuilder.
Using the org.apache.commons.configuration.ConfiguratorConverter, you can convert the Configuration object into a java.util.Properties object which can be passed to the PropertySourcesPlaceholderConfigurer.
As to the chicken and egg question of how to configure the ConfigurationBuilder, I recommend using the org.springframework.core.env.Environment to query for system properties, command-line properties or JNDI properties.
In this exampe:
#Configuration
public class RemotePropertyConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertyPlaceholderConfigurer(Environment environment)
throws Exception {
final PropertySourcesPlaceholderConfigurer props = new PropertySourcesPlaceholderConfigurer();
final ConfigurationBuilder configurationBuilder = new DefaultConfigurationBuilder(environment.getProperty("configuration.definition.file"));
props.setProperties(ConfigurationConverter.getProperties(configurationBuilder.getConfiguration()));
return props;
}
You will need to specify the environment property configuration.definition.file which points to a file needed to configure Commons Configuration:

Similar to Recardo's answer above, I used Spring's PropertiesLoaderUtils instead of Apache's, but it amounts to the same thing. It's not exactly ideal.. hard coded dependency injection, but hey, it works!
/**
* This method must remain static as it's part of spring's initialization effort.
* #return
**/
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
PropertySourcesPlaceholderConfigurer configurer = new PropertySourcesPlaceholderConfigurer();
String dbHost = null;
Integer dbPort = null;
// check system / environment properties first
Environment environment = new StandardEnvironment();
if (environment.containsProperty(DB_HOST_KEY)) {
dbHost = environment.getProperty(DB_HOST_KEY);
}
if (environment.containsProperty(DB_PORT_KEY)) {
dbPort = Integer.valueOf(environment.getProperty(DB_PORT_KEY));
}
if (dbHost == null || dbPort == null) {
// ok one or (probably) both properties null, let's go find the database.properties file
Properties dbProperties;
try {
dbProperties = PropertiesLoaderUtils.loadProperties(new EncodedResource(new ClassPathResource("database.properties"), "UTF-8"));
}
catch (IOException e) {
throw new RuntimeException("Could not load database.properties. Please confirm the file is in the classpath");
}
if (dbHost == null) {
dbHost = dbProperties.getProperty(DB_HOST_KEY);
}
if (dbPort == null) {
dbPort = Integer.valueOf(dbProperties.getProperty(DB_PORT_KEY));
}
}
PropertySourceService propertySourceService = new DBPropertySourceService(dbHost, dbPort);
PropertySource<PropertySourceService> propertySource = new DBPropertySource(propertySourceService);
MutablePropertySources propertySources = new MutablePropertySources();
propertySources.addFirst(propertySource);
configurer.setPropertySources(propertySources);
return configurer;
}
per request, here is the source of the remote property source. It depends on a 'service' class that might do.. well.. anything.. remote access of a property over a socket, talk to a database, whatever.
/**
* Property source for use with spring's PropertySourcesPlaceholderConfigurer where the source is a service
* that connects to remote server for property values.
**/
public class RemotePropertySource extends PropertySource<PropertySourceService> {
private final Environment environment;
/**
* Constructor...
* #param name
* #param source
**/
public RemotePropertySource(PropertySourceService source) {
super("RemotePropertySource", source);
environment = new StandardEnvironment();
}
/* (non-Javadoc)
* #see org.springframework.core.env.PropertySource#getProperty(java.lang.String)
*/
#Override
public Object getProperty(String name) {
// check system / environment properties first
String value;
if (environment.containsProperty(name)) {
value = environment.getProperty(name);
}
else {
value = source.getProperty(name);
}
return value;
}
}

Related

Spring Boot Gemfire Server Configuration

I am trying to understand how to host a Spring Boot Gemfire server process.
I found this example Spring Gemfire Server
The problem I am having is the the server I am trying to add to the cluster is not showing up in the cluster after I start the process.
Here are the steps I am taking:
Start a new locator locally (default port): gfsh>start locator --name=loc-one
I want to add this SpringBootGemfireServer to the cluster:
note I have commented out the embeded locator start-up - I want to add this to the existing locator already running
#SpringBootApplication
#SuppressWarnings("unused")
public class SpringGemFireServerApplication {
private static final boolean DEFAULT_AUTO_STARTUP = true;
public static void main(String[] args) {
SpringApplication.run(SpringGemFireServerApplication.class, args);
}
#Bean
static PropertyPlaceholderConfigurer propertyPlaceholderConfigurer() {
return new PropertyPlaceholderConfigurer();
}
private String applicationName() {
return SpringGemFireServerApplication.class.getSimpleName();
}
#Bean
Properties gemfireProperties(
#Value("${gemfire.log.level:config}") String logLevel,
#Value("${gemfire.locator.host-port:localhost[10334]}") String locatorHostPort,
#Value("${gemfire.manager.port:1099}") String managerPort) {
Properties gemfireProperties = new Properties();
gemfireProperties.setProperty("name", applicationName());
gemfireProperties.setProperty("log-level", logLevel);
//gemfireProperties.setProperty("start-locator", locatorHostPort);
//gemfireProperties.setProperty("jmx-manager", "true");
//gemfireProperties.setProperty("jmx-manager-port", managerPort);
//gemfireProperties.setProperty("jmx-manager-start", "true");
return gemfireProperties;
}
#Bean
CacheFactoryBean gemfireCache(#Qualifier("gemfireProperties") Properties gemfireProperties) {
CacheFactoryBean gemfireCache = new CacheFactoryBean();
gemfireCache.setClose(true);
gemfireCache.setProperties(gemfireProperties);
return gemfireCache;
}
#Bean
CacheServerFactoryBean gemfireCacheServer(Cache gemfireCache,
#Value("${gemfire.cache.server.bind-address:localhost}") String bindAddress,
#Value("${gemfire.cache.server.hostname-for-clients:localhost}") String hostNameForClients,
#Value("${gemfire.cache.server.port:40404}") int port) {
CacheServerFactoryBean gemfireCacheServer = new CacheServerFactoryBean();
gemfireCacheServer.setCache(gemfireCache);
gemfireCacheServer.setAutoStartup(DEFAULT_AUTO_STARTUP);
gemfireCacheServer.setBindAddress(bindAddress);
gemfireCacheServer.setHostNameForClients(hostNameForClients);
gemfireCacheServer.setPort(port);
return gemfireCacheServer;
}
#Bean
PartitionedRegionFactoryBean<Long, Long> factorialsRegion(Cache gemfireCache,
#Qualifier("factorialsRegionAttributes") RegionAttributes<Long, Long> factorialsRegionAttributes) {
PartitionedRegionFactoryBean<Long, Long> factorialsRegion = new PartitionedRegionFactoryBean<>();
factorialsRegion.setAttributes(factorialsRegionAttributes);
factorialsRegion.setCache(gemfireCache);
factorialsRegion.setClose(false);
factorialsRegion.setName("Factorials");
factorialsRegion.setPersistent(false);
return factorialsRegion;
}
#Bean
#SuppressWarnings("unchecked")
RegionAttributesFactoryBean factorialsRegionAttributes() {
RegionAttributesFactoryBean factorialsRegionAttributes = new RegionAttributesFactoryBean();
factorialsRegionAttributes.setCacheLoader(factorialsCacheLoader());
factorialsRegionAttributes.setKeyConstraint(Long.class);
factorialsRegionAttributes.setValueConstraint(Long.class);
return factorialsRegionAttributes;
}
FactorialsCacheLoader factorialsCacheLoader() {
return new FactorialsCacheLoader();
}
class FactorialsCacheLoader implements CacheLoader<Long, Long> {
// stupid, naive implementation of Factorial!
#Override
public Long load(LoaderHelper<Long, Long> loaderHelper) throws CacheLoaderException {
long number = loaderHelper.getKey();
assert number >= 0 : String.format("Number [%d] must be greater than equal to 0", number);
if (number <= 2L) {
return (number < 2L ? 1L : 2L);
}
long result = number;
while (number-- > 1L) {
result *= number;
}
return result;
}
#Override
public void close() {
}
}
}
When I go to gfsh>connect list members
I only see the locator.
I haven't verified the full configuration to check whether there's something else wrong, but the main issue I see right now is that you seem to be confusing the start-locator property (automatically starts a locator in the current process when the member connects to the distributed system and stops the locator when the member disconnects) with the locators property (the list of locators used by system members, it must be configured consistently for every member of the distributed system). Since you're not correctly setting the locators property when configuring the server, it just can't join the existing distributed system because it doesn't know which locator to connect to.
The default locator port used by GemFire is 10334, so you should change your gemfireProperties method as follows:
#Bean
Properties gemfireProperties(#Value("${gemfire.log.level:config}") String logLevel, #Value("${gemfire.locator.host-port:localhost[10334]}") String locatorHostPort, #Value("${gemfire.manager.port:1099}") String managerPort) {
Properties gemfireProperties = new Properties();
gemfireProperties.setProperty("name", applicationName());
gemfireProperties.setProperty("log-level", logLevel);
// You can directly use the locatorHostPort variable instead.
gemfireProperties.setProperty("locators", "localhost[10334]");
return gemfireProperties;
}
Hope this helps.
Cheers.

Access all Environment properties as a Map or Properties object

I am using annotations to configure my spring environment like this:
#Configuration
...
#PropertySource("classpath:/config/default.properties")
...
public class GeneralApplicationConfiguration implements WebApplicationInitializer
{
#Autowired
Environment env;
}
This leads to my properties from default.properties being part of the Environment. I want to use the #PropertySource mechanism here, because it already provides the possibility to overload properties through several fallback layers and different dynamic locations, based on the environment settings (e.g. config_dir location). I just stripped the fallback to make the example easier.
However, my problem now is that I want to configure for example my datasource properties in default.properties. You can pass the settings to the datasource without knowing in detail what settings the datasource expects using
Properties p = ...
datasource.setProperties(p);
However, the problem is, the Environment object is neither a Properties object nor a Map nor anything comparable. From my point of view it is simply not possible to access all values of the environment, because there is no keySet or iterator method or anything comparable.
Properties p <=== Environment env?
Am I missing something? Is it possible to access all entries of the Environment object somehow? If yes, I could map the entries to a Map or Properties object, I could even filter or map them by prefix - create subsets as a standard java Map ... This is what I would like to do. Any suggestions?
You need something like this, maybe it can be improved. This is a first attempt:
...
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.AbstractEnvironment;
import org.springframework.core.env.Environment;
import org.springframework.core.env.MapPropertySource;
...
#Configuration
...
#org.springframework.context.annotation.PropertySource("classpath:/config/default.properties")
...
public class GeneralApplicationConfiguration implements WebApplicationInitializer
{
#Autowired
Environment env;
public void someMethod() {
...
Map<String, Object> map = new HashMap();
for(Iterator it = ((AbstractEnvironment) env).getPropertySources().iterator(); it.hasNext(); ) {
PropertySource propertySource = (PropertySource) it.next();
if (propertySource instanceof MapPropertySource) {
map.putAll(((MapPropertySource) propertySource).getSource());
}
}
...
}
...
Basically, everything from the Environment that's a MapPropertySource (and there are quite a lot of implementations) can be accessed as a Map of properties.
This is an old question, but the accepted answer has a serious flaw. If the Spring Environment object contains any overriding values (as described in Externalized Configuration), there is no guarantee that the map of property values it produces will match those returned from the Environment object. I found that simply iterating through the PropertySources of the Environment did not, in fact, give any overriding values. Instead it produced the original value, the one that should have been overridden.
Here is a better solution. This uses the EnumerablePropertySources of the Environment to iterate through the known property names, but then reads the actual value out of the real Spring environment. This guarantees that the value is the one actually resolved by Spring, including any overriding values.
Properties props = new Properties();
MutablePropertySources propSrcs = ((AbstractEnvironment) springEnv).getPropertySources();
StreamSupport.stream(propSrcs.spliterator(), false)
.filter(ps -> ps instanceof EnumerablePropertySource)
.map(ps -> ((EnumerablePropertySource) ps).getPropertyNames())
.flatMap(Arrays::<String>stream)
.forEach(propName -> props.setProperty(propName, springEnv.getProperty(propName)));
I had the requirement to retrieve all properties whose key starts with a distinct prefix (e.g. all properties starting with "log4j.appender.") and wrote following Code (using streams and lamdas of Java 8).
public static Map<String,Object> getPropertiesStartingWith( ConfigurableEnvironment aEnv,
String aKeyPrefix )
{
Map<String,Object> result = new HashMap<>();
Map<String,Object> map = getAllProperties( aEnv );
for (Entry<String, Object> entry : map.entrySet())
{
String key = entry.getKey();
if ( key.startsWith( aKeyPrefix ) )
{
result.put( key, entry.getValue() );
}
}
return result;
}
public static Map<String,Object> getAllProperties( ConfigurableEnvironment aEnv )
{
Map<String,Object> result = new HashMap<>();
aEnv.getPropertySources().forEach( ps -> addAll( result, getAllProperties( ps ) ) );
return result;
}
public static Map<String,Object> getAllProperties( PropertySource<?> aPropSource )
{
Map<String,Object> result = new HashMap<>();
if ( aPropSource instanceof CompositePropertySource)
{
CompositePropertySource cps = (CompositePropertySource) aPropSource;
cps.getPropertySources().forEach( ps -> addAll( result, getAllProperties( ps ) ) );
return result;
}
if ( aPropSource instanceof EnumerablePropertySource<?> )
{
EnumerablePropertySource<?> ps = (EnumerablePropertySource<?>) aPropSource;
Arrays.asList( ps.getPropertyNames() ).forEach( key -> result.put( key, ps.getProperty( key ) ) );
return result;
}
// note: Most descendants of PropertySource are EnumerablePropertySource. There are some
// few others like JndiPropertySource or StubPropertySource
myLog.debug( "Given PropertySource is instanceof " + aPropSource.getClass().getName()
+ " and cannot be iterated" );
return result;
}
private static void addAll( Map<String, Object> aBase, Map<String, Object> aToBeAdded )
{
for (Entry<String, Object> entry : aToBeAdded.entrySet())
{
if ( aBase.containsKey( entry.getKey() ) )
{
continue;
}
aBase.put( entry.getKey(), entry.getValue() );
}
}
Note that the starting point is the ConfigurableEnvironment which is able to return the embedded PropertySources (the ConfigurableEnvironment is a direct descendant of Environment). You can autowire it by:
#Autowired
private ConfigurableEnvironment myEnv;
If you not using very special kinds of property sources (like JndiPropertySource, which is usually not used in spring autoconfiguration) you can retrieve all properties held in the environment.
The implementation relies on the iteration order which spring itself provides and takes the first found property, all later found properties with the same name are discarded. This should ensure the same behaviour as if the environment were asked directly for a property (returning the first found one).
Note also that the returned properties are not yet resolved if they contain aliases with the ${...} operator. If you want to have a particular key resolved you have to ask the Environment directly again:
myEnv.getProperty( key );
The original question hinted that it would be nice to be able to filter all the properties based on a prefix. I have just confirmed that this works as of Spring Boot 2.1.1.RELEASE, for Properties or Map<String,String>. I'm sure it's worked for while now. Interestingly, it does not work without the prefix = qualification, i.e. I do not know how to get the entire environment loaded into a map. As I said, this might actually be what OP wanted to begin with. The prefix and the following '.' will be stripped off, which might or might not be what one wants:
#ConfigurationProperties(prefix = "abc")
#Bean
public Properties getAsProperties() {
return new Properties();
}
#Bean
public MyService createService() {
Properties properties = getAsProperties();
return new MyService(properties);
}
Postscript: It is indeed possible, and shamefully easy, to get the entire environment. I don't know how this escaped me:
#ConfigurationProperties
#Bean
public Properties getProperties() {
return new Properties();
}
As this Spring's Jira ticket, it is an intentional design. But the following code works for me.
public static Map<String, Object> getAllKnownProperties(Environment env) {
Map<String, Object> rtn = new HashMap<>();
if (env instanceof ConfigurableEnvironment) {
for (PropertySource<?> propertySource : ((ConfigurableEnvironment) env).getPropertySources()) {
if (propertySource instanceof EnumerablePropertySource) {
for (String key : ((EnumerablePropertySource) propertySource).getPropertyNames()) {
rtn.put(key, propertySource.getProperty(key));
}
}
}
}
return rtn;
}
Spring won't allow to decouple via java.util.Properties from Spring Environment.
But Properties.load() still works in a Spring boot application:
Properties p = new Properties();
try (InputStream is = getClass().getResourceAsStream("/my.properties")) {
p.load(is);
}
The other answers have pointed out the solution for the majority of cases involving PropertySources, but none have mentioned that certain property sources are unable to be casted into useful types.
One such example is the property source for command line arguments. The class that is used is SimpleCommandLinePropertySource. This private class is returned by a public method, thus making it extremely tricky to access the data inside the object. I had to use reflection in order to read the data and eventually replace the property source.
If anyone out there has a better solution, I would really like to see it; however, this is the only hack I have gotten to work.
Working with Spring Boot 2, I needed to do something similar. Most of the answers above work fine, just beware that at various phases in the app lifecycles the results will be different.
For example, after a ApplicationEnvironmentPreparedEvent any properties inside application.properties are not present. However, after a ApplicationPreparedEvent event they are.
For Spring Boot, the accepted answer will overwrite duplicate properties with lower priority ones. This solution will collect the properties into a SortedMap and take only the highest priority duplicate properties.
final SortedMap<String, String> sortedMap = new TreeMap<>();
for (final PropertySource<?> propertySource : env.getPropertySources()) {
if (!(propertySource instanceof EnumerablePropertySource))
continue;
for (final String name : ((EnumerablePropertySource<?>) propertySource).getPropertyNames())
sortedMap.computeIfAbsent(name, propertySource::getProperty);
}
I though I'd add one more way. In my case I supply this to com.hazelcast.config.XmlConfigBuilder which only needs java.util.Properties to resolve some properties inside the Hazelcast XML configuration file, i.e. it only calls getProperty(String) method. So, this allowed me to do what I needed:
#RequiredArgsConstructor
public class SpringReadOnlyProperties extends Properties {
private final org.springframework.core.env.Environment delegate;
#Override
public String getProperty(String key) {
return delegate.getProperty(key);
}
#Override
public String getProperty(String key, String defaultValue) {
return delegate.getProperty(key, defaultValue);
}
#Override
public synchronized String toString() {
return getClass().getName() + "{" + delegate + "}";
}
#Override
public boolean equals(Object o) {
if (this == o) return true;
if (o == null || getClass() != o.getClass()) return false;
if (!super.equals(o)) return false;
SpringReadOnlyProperties that = (SpringReadOnlyProperties) o;
return delegate.equals(that.delegate);
}
#Override
public int hashCode() {
return Objects.hash(super.hashCode(), delegate);
}
private void throwException() {
throw new RuntimeException("This method is not supported");
}
//all methods below throw the exception
* override all methods *
}
P.S. I ended up not using this specifically for Hazelcast because it only resolves properties for XML file but not at runtime. Since I also use Spring, I decided to go with a custom org.springframework.cache.interceptor.AbstractCacheResolver#getCacheNames. This resolves properties for both situations, at least if you use properties in cache names.
To get ONLY properties, defined in my hibernate.properteies file:
#PropertySource(SomeClass.HIBERNATE_PROPERTIES)
public class SomeClass {
public static final String HIBERNATE_PROPERTIES = "hibernate.properties";
#Autowired
private Environment env;
public void someMethod() {
final Properties hibProps = asProperties(HIBERNATE_PROPERTIES);
}
private Properties asProperties(String fileName) {
return StreamSupport.stream(
((AbstractEnvironment) env).getPropertySources().spliterator(), false)
.filter(ps -> ps instanceof ResourcePropertySource)
.map(ps -> (ResourcePropertySource) ps)
.filter(rps -> rps.getName().contains(fileName))
.collect(
Properties::new,
(props, rps) -> props.putAll(rps.getSource()),
Properties::putAll);
}
}
A little helper to analyze the sources of a property, which sometimes drive me crazy . I used this discussion to write SpringConfigurableEnvironment.java on github.
It could be used in a test:
#SpringBootTest
public class SpringConfigurableEnvironmentTest {
#Autowired
private ConfigurableEnvironment springEnv;
#Test
public void testProperties() {
SpringConfigurableEnvironment properties = new SpringConfigurableEnvironment(springEnv);
SpringConfigurableEnvironment.PropertyInfo info = properties.get("profile.env");
assertEquals("default", properties.get(info.getValue());
assertEquals(
"Config resource 'class path resource [application.properties]' via location 'optional:classpath:/'",
info.getSourceList.get(0));
}
}
All answers above have pretty much covers everything, but be aware of overridden values from environment variables. They may have different key values.
For example, if a user override my.property[1].value using environment variable MY_PROPERTY[1]_VALUE, iterating through EnumerablePropertySources.getPropertyNames() would give you both my.property[1].value and MY_PROPERTY[1]_VALUE key values.
What even worse is that if my.property[1].value is not defined in applications.conf (or applications.yml), a MY_PROPERTY[1]_VALUE in environment variables would not give you my.property[1].value but only MY_PROPERTY[1]_VALUE key value from EnumerablePropertySources.getPropertyNames().
So it is developers' job to cover the those properties from environment variables. Unfortunately, there is no one-on-one mapping between environment variables schema vs the normal schema, see the source code of SystemEnvironmentPropertySource. For example, MY_PROPERTY[1]_VALUE could be either my.property[1].value or my-property[1].value

PropertySourcesPlaceholderConfigurer not registering with Environment in a SpringBoot Project

I am moving a working project from using SpringBoot command line arguments to reading properties from a file. Here are the involved portions of the #Configuration class:
#Configuration
class RemoteCommunication {
#Inject
StandardServletEnvironment env
#Bean
static PropertySourcesPlaceholderConfigurer placeholderConfigurer () {
// VERIFIED this is executing...
PropertySourcesPlaceholderConfigurer target = new PropertySourcesPlaceholderConfigurer()
// VERIFIED this files exists, is readable, is a valid properties file
target.setLocation (new FileSystemResource ('/Users/me/Desktop/mess.properties'))
// A Debugger does NOT show this property source in the inject Environment
target
}
#Bean // There are many of these for different services, only one shown here.
MedicalSorIdService medicalSorIdService () {
serviceInstantiator (MedicalSorIdService_EpicSoap, 'uri.sor.id.lookup.internal')
}
// HELPER METHODS...
private <T> T serviceInstantiator (final Class<T> classToInstantiate, final String propertyKeyPrimary) {
def value = retrieveSpringPropertyFromConfigurationParameter (propertyKeyPrimary)
classToInstantiate.newInstance (value)
}
private def retrieveSpringPropertyFromConfigurationParameter (String propertyKeyPrimary) {
// PROBLEM: the property is not found in the Environment
def value = env.getProperty (propertyKeyPrimary, '')
if (value.isEmpty ()) throw new IllegalStateException ('Missing configuration parameter: ' + "\"$propertyKeyPrimary\"")
value
}
Using #Value to inject the properties does work, however I'd rather work with the Environment directly if at all possible. If the settings are not in the Environment then I am not exactly sure where #Value is pulling them from...
env.getProperty() continues to work well when I pass in command line arguments specifying the properties though.
Any suggestions are welcome!
The issue here is the distinction between PropertySourcesPlaceholderConfigurer and StandardServletEnvironment, or Environment for simplicity.
The Environment is an object that backs the whole ApplicationContext and can resolve a bunch of properties (the Environment interface extends PropertyResolver). A ConfigurableEnvironment has a MutablePropertySources object which you can retrieve through getPropertySources(). This MutablePropertySources holds a LinkedList of PropertySource objects which are checked in order to resolve a requested property.
PropertySourcesPlaceholderConfigurer is a separate object with its own state. It holds its own MutablePropertySources object for resolving property placeholders. PropertySourcesPlaceholderConfigurer implements EnvironmentAware so when the ApplicationContext gets hold of it, it gives it its Environment object. The PropertySourcesPlaceholderConfigurer adds this Environment's MutablePropertySources to its own. It then also adds the various Resource objects you specified with setLocation() as additional properties. These Resource objects are not added to the Environment's MutablePropertySources and therefore aren't available with env.getProperty(String).
So you cannot get the properties loaded by the PropertySourcesPlaceholderConfigurer into the Environment directly. What you can do instead is add directly to the Environment's MutablePropertySouces. One way is with
#PostConstruct
public void setup() throws IOException {
Resource resource = new FileSystemResource("spring.properties"); // your file
Properties result = new Properties();
PropertiesLoaderUtils.fillProperties(result, resource);
env.getPropertySources().addLast(new PropertiesPropertySource("custom", result));
}
or simply (thanks #M.Deinum)
#PostConstruct
public void setup() throws IOException {
env.getPropertySources().addLast(new ResourcePropertySource("custom", "file:spring.properties")); // the name 'custom' can come from anywhere
}
Note that adding a #PropertySource has the same effect, ie. adding directly to the Environment, but you're doing it statically rather than dynamically.
In SpringBoot it's enough to use #EnableConfigurationProperties annotation - you don't need to setup PropertySourcesPlaceholderConfigurer.
Then on POJO you add annotation #ConfigurationProperties and Spring automatically injects your properties defined in application.properties.
You can also use YAML files - you just need to add proper dependency (like SnakeYaml) to classpath
You can find detailed example here: http://spring.io/blog/2013/10/30/empowering-your-apps-with-spring-boot-s-property-support
I achieved this during PropertySourcesPlaceholderConfigurer instantiation.
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurerBean(Environment env) {
PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer = new PropertySourcesPlaceholderConfigurer();
YamlPropertiesFactoryBean yamlFactorybean = new YamlPropertiesFactoryBean();
yamlFactorybean.setResources(determineResources(env));
PropertiesPropertySource yampProperties = new PropertiesPropertySource("yml", yamlFactorybean.getObject());
((AbstractEnvironment)env).getPropertySources().addLast(yampProperties);
propertySourcesPlaceholderConfigurer.setProperties(yamlFactorybean.getObject());
return propertySourcesPlaceholderConfigurer;
}
private static Resource[] determineResources(Environment env){
int numberOfActiveProfiles = env.getActiveProfiles().length;
ArrayList<Resource> properties = new ArrayList(numberOfActiveProfiles);
properties.add( new ClassPathResource("application.yml") );
for (String profile : env.getActiveProfiles()){
String yamlFile = "application-"+profile+".yml";
ClassPathResource props = new ClassPathResource(yamlFile);
if (!props.exists()){
log.info("Configuration file {} for profile {} does not exist");
continue;
}
properties.add(props);
}
if (log.isDebugEnabled())
log.debug("Populating application context with properties files: {}", properties);
return properties.toArray(new Resource[properties.size()]);
}
Maybe all you need is to set -Dspring.config.location=... (alternatively SPRING_CONFIG_LOCATION as an env var)? That has the effect of adding an additional config file to the default path for the app at runtime which takes precedence over the normal application.properties? See howto docs for details.

Spring Environment Property Source Configuration

I'm working on an application library with a utility class called "Config" which is backed by the Spring Environment object and provides strongly typed getters for all the applications configuration values.
The property sources for the configuration can vary depending on environment (DEV/PROD) and usage (standalone/test/webapp), and can range from the default ones (system & env props) to custom database and JNDI sources.
What I'm struggling with is how to let the apps consuming this library easily configure the property source(s) used by Environment, such that the properties are available for use in our Config class and via the PropertySourcesPlaceholderConfigurer.
We're still using XML configuration, so ideally this could be configured in XML something like.
<bean id="propertySources" class="...">
<property name="sources">
<list>
<ref local="jndiPropertySource"/>
<ref local="databasePropertySource"/>
</list>
</property>
</bean>
...and then injected somehow into the Environment's property sources collection.
I've read that something like this may not be possible due to the timing of the app context lifecycle, and that this may need to be done using an application initializer class.
Any ideas?
It depends on how you want to use the properties, if it is to inject the properties using ${propertyname} syntax, then yes just having PropertySourcesPlaceHolderConfigurer will work, which internally has access to the PropertySources registered in the environment.
If you plan to use Environment directly, using say env.getProperty(), then you are right - the properties using PropertySourcesPlaceHolderConfigurer are not visible here. The only way then is to inject it using Java code, there are two ways that I know of:
a. Using Java Config:
#Configuration
#PropertySource("classpath:/app.properties")
public class SpringConfig{
}
b. Using a custom ApplicationContextInitializer, the way it is described here
I came up with the following which seems to work, but I'm fairly new to Spring, so I'm not so sure how it will hold up under different use cases.
Basically, the approach is to extend PropertySourcesPlaceholderConfigurer and add a setter to allow the user to easily configure a List of PropertySource objects in XML. After creation, the property sources are copied to the current Environment.
This basically allows the property sources to be configured in one place, but used by both placholder configuration and Environment.getProperty scenarios.
Extended PropertySourcesPlaceholderConfigurer
public class ConfigSourcesConfigurer
extends PropertySourcesPlaceholderConfigurer
implements EnvironmentAware, InitializingBean {
private Environment environment;
private List<PropertySource> sourceList;
// Allow setting property sources as a List for easier XML configuration
public void setPropertySources(List<PropertySource> propertySources) {
this.sourceList = propertySources;
MutablePropertySources sources = new MutablePropertySources();
copyListToPropertySources(this.sourceList, sources);
super.setPropertySources(sources);
}
#Override
public void setEnvironment(Environment environment) {
// save off Environment for later use
this.environment = environment;
super.setEnvironment(environment);
}
#Override
public void afterPropertiesSet() throws Exception {
// Copy property sources to Environment
MutablePropertySources envPropSources = ((ConfigurableEnvironment)environment).getPropertySources();
copyListToPropertySources(this.sourceList, envPropSources);
}
private void copyListToPropertySources(List<PropertySource> list, MutablePropertySources sources) {
// iterate in reverse order to insure ordering in property sources object
for(int i = list.size() - 1; i >= 0; i--) {
sources.addFirst(list.get(i));
}
}
}
beans.xml file showing basic configuration
<beans>
<context:annotation-config/>
<context:component-scan base-package="com.mycompany" />
<bean class="com.mycompany.ConfigSourcesConfigurer">
<property name="propertySources">
<list>
<bean class="org.mycompany.CustomPropertySource" />
<bean class="org.springframework.core.io.support.ResourcePropertySource">
<constructor-arg value="classpath:default-config.properties" />
</bean>
</list>
</property>
</bean>
<bean class="com.mycompany.TestBean">
<property name="stringValue" value="${placeholder}" />
</bean>
</beans>
The following worked for me with Spring 3.2.4 .
PropertySourcesPlaceholderConfigurer must be registered statically in order to process the placeholders.
The custom property source is registered in the init method and as the default property sources are already registered, it can itself be parameterized using placeholders.
JavaConfig class:
#Configuration
#PropertySource("classpath:propertiesTest2.properties")
public class TestConfig {
#Autowired
private ConfigurableEnvironment env;
#Value("${param:NOVALUE}")
private String param;
#PostConstruct
public void init() {
env.getPropertySources().addFirst(new CustomPropertySource(param));
}
#Bean
public static PropertySourcesPlaceholderConfigurer placeHolderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean
public TestBean1 testBean1() {
return new TestBean1();
}
}
Custom property source:
public class CustomPropertySource extends PropertySource<Object> {
public CustomPropertySource(String param) {
super("custom");
System.out.println("Custom property source initialized with param " + param + ".");
}
#Override
public Object getProperty(String name) {
return "IT WORKS";
}
}
Test bean (getValue() will output "IT WORKS"):
public class TestBean1 {
#Value("${value:NOVALUE}")
private String value;
public String getValue() {
return value;
}
}
I had a similar problem, in my case I'm using Spring in a standalone application, after load the default configurations I may need apply another properties file (lazy load configs) present in a config directory. My solution was inspired this Spring Boot documentation, but with no dependency of Spring Boot. See below the source code:
#PropertySources(#PropertySource(value = "classpath:myapp-default.properties"))
public class PersistenceConfiguration {
private final Logger log = LoggerFactory.getLogger(getClass());
private ConfigurableEnvironment env;
#Bean
public static PropertySourcesPlaceholderConfigurer placeholderConfigurerDev(ConfigurableEnvironment env) {
return new PropertySourcesPlaceholderConfigurer();
}
#Autowired
public void setConfigurableEnvironment(ConfigurableEnvironment env) {
for(String profile: env.getActiveProfiles()) {
final String fileName = "myapp-" + profile + ".properties";
final Resource resource = new ClassPathResource(fileName);
if (resource.exists()) {
try {
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(new PropertiesPropertySource(fileName,PropertiesLoaderUtils.loadProperties(resource)));
} catch (Exception ex) {
log.error(ex.getMessage(), ex);
throw new RuntimeException(ex.getMessage(), ex);
}
}
}
this.env = env;
}
...
}
I recently ran into the issue of how to register custom property sources in the environment. My specific problem is that I have a library with a Spring configuration that I want to be imported into the Spring application context, and it requires custom property sources. However, I don't necessarily have control over all of the places where the application context is created. Because of this, I do not want to use the recommended mechanisms of ApplicationContextInitializer or register-before-refresh in order to register the custom property sources.
What I found really frustrating is that using the old PropertyPlaceholderConfigurer, it was easy to subclass and customize the configurers completely within the Spring configuration. In contrast, to customize property sources, we are told that we have to do it not in the Spring configuration itself, but before the application context is initialized.
After some research and trial and error, I discovered that it is possible to register custom property sources from inside of the Spring configuration, but you have to be careful how you do it. The sources need to be registered before any PropertySourcesPlaceholderConfigurers execute in the context. You can do this by making the source registration a BeanFactoryPostProcessor with PriorityOrdered and an order that is higher precedence than the PropertySourcesPlaceholderConfigurer that uses the sources.
I wrote this class, which does the job:
package example;
import java.io.IOException;
import java.util.Properties;
import org.springframework.beans.BeansException;
import org.springframework.beans.factory.BeanInitializationException;
import org.springframework.beans.factory.config.BeanFactoryPostProcessor;
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.core.Ordered;
import org.springframework.core.PriorityOrdered;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.Environment;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.core.env.PropertiesPropertySource;
import org.springframework.core.env.PropertySource;
import org.springframework.core.io.support.PropertiesLoaderSupport;
/**
* This is an abstract base class that can be extended by any class that wishes
* to become a custom property source in the Spring context.
* <p>
* This extends from the standard Spring class PropertiesLoaderSupport, which
* contains properties that specify property resource locations, plus methods
* for loading properties from specified resources. These are all available to
* be used from the Spring configuration, and by subclasses of this class.
* <p>
* This also implements a number of Spring flag interfaces, all of which are
* required to maneuver instances of this class into a position where they can
* register their property sources BEFORE PropertySourcesPlaceholderConfigurer
* executes to substitute variables in the Spring configuration:
* <ul>
* <li>BeanFactoryPostProcessor - Guarantees that this bean will be instantiated
* before other beans in the context. It also puts it in the same phase as
* PropertySourcesPlaceholderConfigurer, which is also a BFPP. The
* postProcessBeanFactory method is used to register the property source.</li>
* <li>PriorityOrdered - Allows the bean priority to be specified relative to
* PropertySourcesPlaceholderConfigurer so that this bean can be executed first.
* </li>
* <li>ApplicationContextAware - Provides access to the application context and
* its environment so that the created property source can be registered.</li>
* </ul>
* <p>
* The Spring configuration for subclasses should contain the following
* properties:
* <ul>
* <li>propertySourceName - The name of the property source this will register.</li>
* <li>location(s) - The location from which properties will be loaded.</li>
* <li>addBeforeSourceName (optional) - If specified, the resulting property
* source will be added before the given property source name, and will
* therefore take precedence.</li>
* <li>order (optional) - The order in which this source should be executed
* relative to other BeanFactoryPostProcessors. This should be used in
* conjunction with addBeforeName so that if property source factory "psfa"
* needs to register its property source before the one from "psfb", "psfa"
* executes AFTER "psfb".
* </ul>
*
* #author rjsmith2
*
*/
public abstract class AbstractPropertySourceFactory extends
PropertiesLoaderSupport implements ApplicationContextAware,
PriorityOrdered, BeanFactoryPostProcessor {
// Default order will be barely higher than the default for
// PropertySourcesPlaceholderConfigurer.
private int order = Ordered.LOWEST_PRECEDENCE - 1;
private String propertySourceName;
private String addBeforeSourceName;
private ApplicationContext applicationContext;
private MutablePropertySources getPropertySources() {
final Environment env = applicationContext.getEnvironment();
if (!(env instanceof ConfigurableEnvironment)) {
throw new IllegalStateException(
"Cannot get environment for Spring application context");
}
return ((ConfigurableEnvironment) env).getPropertySources();
}
public int getOrder() {
return order;
}
public void setOrder(int order) {
this.order = order;
}
public String getPropertySourceName() {
return propertySourceName;
}
public void setPropertySourceName(String propertySourceName) {
this.propertySourceName = propertySourceName;
}
public String getAddBeforeSourceName() {
return addBeforeSourceName;
}
public void setAddBeforeSourceName(String addBeforeSourceName) {
this.addBeforeSourceName = addBeforeSourceName;
}
public void setApplicationContext(ApplicationContext applicationContext) {
this.applicationContext = applicationContext;
}
/**
* Subclasses can override this method to perform adjustments on the
* properties after they are read.
* <p>
* This should be done by getting, adding, removing, and updating properties
* as needed.
*
* #param props
* properties to adjust
*/
protected void convertProperties(Properties props) {
// Override in subclass to perform conversions.
}
/**
* Creates a property source from the specified locations.
*
* #return PropertiesPropertySource instance containing the read properties
* #throws IOException
* if properties cannot be read
*/
protected PropertySource<?> createPropertySource() throws IOException {
if (propertySourceName == null) {
throw new IllegalStateException("No property source name specified");
}
// Load the properties file (or files) from specified locations.
final Properties props = new Properties();
loadProperties(props);
// Convert properties as required.
convertProperties(props);
// Convert to property source.
final PropertiesPropertySource source = new PropertiesPropertySource(
propertySourceName, props);
return source;
}
#Override
public void postProcessBeanFactory(
ConfigurableListableBeanFactory beanFactory) throws BeansException {
try {
// Create the property source, and get its desired position in
// the list of sources.
if (logger.isDebugEnabled()) {
logger.debug("Creating property source [" + propertySourceName
+ "]");
}
final PropertySource<?> source = createPropertySource();
// Register the property source.
final MutablePropertySources sources = getPropertySources();
if (addBeforeSourceName != null) {
if (sources.contains(addBeforeSourceName)) {
if (logger.isDebugEnabled()) {
logger.debug("Adding property source ["
+ propertySourceName + "] before ["
+ addBeforeSourceName + "]");
}
sources.addBefore(addBeforeSourceName, source);
} else {
logger.warn("Property source [" + propertySourceName
+ "] cannot be added before non-existent source ["
+ addBeforeSourceName + "] - adding at the end");
sources.addLast(source);
}
} else {
if (logger.isDebugEnabled()) {
logger.debug("Adding property source ["
+ propertySourceName + "] at the end");
}
sources.addLast(source);
}
} catch (Exception e) {
throw new BeanInitializationException(
"Failed to register property source", e);
}
}
}
Of note here is that the default order of this property source factory class is higher precedence than the default order of PropertySourcesPlaceholderConfigurer.
Also, the registration of the property source happens in postProcessBeanFactory, which means that it will execute in the correct order relative to the PropertySourcesPlaceholderConfigurer. I discovered the hard way that InitializingBean and afterPropertiesSet do not respect the order parameter, and I gave up on that approach as being wrong and redundant.
Finally, because this is a BeanFactoryPostProcessor, it is a bad idea to try to wire much in the way of dependencies. Therefore, the class accesses the environment directly through the application context, which it obtains using ApplicationContextAware.
In my case, I needed the property source to decrypt password properties, which I implemented using the following subclass:
package example;
import java.util.Properties;
/**
* This is a property source factory that creates a property source that can
* process properties for substituting into a Spring configuration.
* <p>
* The only thing that distinguishes this from a normal Spring property source
* is that it decrypts encrypted passwords.
*
* #author rjsmith2
*
*/
public class PasswordPropertySourceFactory extends
AbstractPropertySourceFactory {
private static final PasswordHelper passwordHelper = new PasswordHelper();
private String[] passwordProperties;
public String[] getPasswordProperties() {
return passwordProperties;
}
public void setPasswordProperties(String[] passwordProperties) {
this.passwordProperties = passwordProperties;
}
public void setPasswordProperty(String passwordProperty) {
this.passwordProperties = new String[] { passwordProperty };
}
#Override
protected void convertProperties(Properties props) {
// Adjust password fields by decrypting them.
if (passwordProperties != null) {
for (String propName : passwordProperties) {
final String propValue = props.getProperty(propName);
if (propValue != null) {
final String plaintext = passwordHelper
.decryptString(propValue);
props.setProperty(propName, plaintext);
}
}
}
}
}
Finally, I specifed the property source factory in my Spring configuration:
<!-- Enable property resolution via PropertySourcesPlaceholderConfigurer.
The order has to be larger than the ones used by custom property sources
so that those property sources are registered before any placeholders
are substituted. -->
<context:property-placeholder order="1000" ignore-unresolvable="true" />
<!-- Register a custom property source that reads DB properties, and
decrypts the database password. -->
<bean class="example.PasswordPropertySourceFactory">
<property name="propertySourceName" value="DBPropertySource" />
<property name="location" value="classpath:db.properties" />
<property name="passwordProperty" value="db.password" />
<property name="ignoreResourceNotFound" value="true" />
<!-- Order must be lower than on property-placeholder element. -->
<property name="order" value="100" />
</bean>
To be honest, with the defaults for order in PropertySourcesPlaceholderConfigurer and AbstractPropertySourceFactory, it is probably not even necessary to specify order in the Spring configuration.
Nonetheless, this works, and it does not require any fiddling with the application context initialization.

Spring, autowire #Value from a database

I am using a properties File to store some configuration properties, that are accessed this way:
#Value("#{configuration.path_file}")
private String pathFile;
Is it possible (with Spring 3) to use the same #Value annotation, but loading the properties from a database instead of a file ?
Assuming you have a table in your database stored key/value pairs:
Define a new bean "applicationProperties" - psuedo-code follows...
public class ApplicationProperties {
#AutoWired
private DataSource datasource;
public getPropertyValue(String key) {
// transact on your datasource here to fetch value for key
// SNIPPED
}
}
Inject this bean where required in your application. If you already have a dao/service layer then you would just make use of that.
Yes, you can keep your #Value annotation, and use the database source with the help of EnvironmentPostProcessor.
As of Spring Boot 1.3, we're able to use the EnvironmentPostProcessor to customize the application's Environment before application context is refreshed.
For example, create a class which implements EnvironmentPostProcessor:
public class ReadDbPropertiesPostProcessor implements EnvironmentPostProcessor {
private static final String PROPERTY_SOURCE_NAME = "databaseProperties";
private String[] CONFIGS = {
"app.version"
// list your properties here
};
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Map<String, Object> propertySource = new HashMap<>();
try {
// the following db connections properties must be defined in application.properties
DataSource ds = DataSourceBuilder
.create()
.username(environment.getProperty("spring.datasource.username"))
.password(environment.getProperty("spring.datasource.password"))
.url(environment.getProperty("spring.datasource.url"))
.driverClassName("com.mysql.jdbc.Driver")
.build();
try (Connection connection = ds.getConnection();
// suppose you have a config table storing the properties name/value pair
PreparedStatement preparedStatement = connection.prepareStatement("SELECT value FROM config WHERE name = ?")) {
for (int i = 0; i < CONFIGS.length; i++) {
String configName = CONFIGS[i];
preparedStatement.setString(1, configName);
ResultSet rs = preparedStatement.executeQuery();
while (rs.next()) {
propertySource.put(configName, rs.getString("value"));
}
// rs.close();
preparedStatement.clearParameters();
}
}
environment.getPropertySources().addFirst(new MapPropertySource(PROPERTY_SOURCE_NAME, propertySource));
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
}
Finally, don't forget to put your spring.factories in META-INF. An example:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=
com.baeldung.environmentpostprocessor.autoconfig.PriceCalculationAutoConfig
Although not having used spring 3, I'd assume you can, if you make a bean that reads the properties from the database and exposes them with getters.

Resources