I would like to set bootstrap.yaml property aws.paramstore.prefix programmatically.
According to the documentation the only way to configure it is via bootstrap.yaml file. And it works fine if I define aws.paramstore.prefix in the bootstrap.yaml file. However, I would like to do that programmatically.
There is also a possibility to customize bootstrap property sources, however this does not solve the problem. Custom Bootstrap seem to be loaded later than aws.paramstore properties.
As far as I can see the aws.paramstore properties are loaded very early on using spring.factories that are defined in spring-cloud-starter-aws-parameter-store-config dependency:
org.springframework.cloud.bootstrap.BootstrapConfiguration=\
org.springframework.cloud.aws.autoconfigure.paramstore.AwsParamStoreBootstrapConfiguration
And in AwsParamStoreBootstrapConfiguration constructor AwsParamStoreProperties is passed as an argument, which is instantiated via #ConfigurationProperties. That is aws.paramstore are loaded very early on.
You can try to look into PostProcessors that executed on the very first stages of Spring Context initialization. For example, EnvironmentPostProcessor. It executes quite early and probably can help you to manage some properties/modify files before they will be loaded to context.
Note: if you want to use this code in some library that will be a part of another project, you should add such PostProcessor to spring.factories file. In other case it won't be added to configuration stage of another Spring.
A hacky workaround is to set the aws.paramstore.prefix property before calling SpringApplication#run method.
Say we want to set aws.paramstore.prefix to the value of a Spring profile that is being passed to the application. Then one can do something like:
public static void main(String[] args) {
// or System.getenv("SPRING_PROFILES_ACTIVE")
// or some additional logic that filters a profile that is being passed to the application
// note that application can also have multiple profiles
String activeProfile = System.getProperty("spring.profiles.active");
System.setProperty("aws.paramstore.prefix", activeProfile);
SpringApplication.run(MyApplication.class, args);
}
In this way we "guarantee" that aws.paramstore.prefix is available before ane bootstraps from spring.factories kicks in.
Related
I have a Maven/SpringBootApplication that takes its properties from a Spring config Server. I need to override the values of these properties using command line arguments. unfortunately, the properties keep the values provided by the config server and are not overridden by the command line arguments.
I have confirmed that the parameters are properly passed to the App as I can see being passed to SpringApplication.run.
I can see in the function ConfigurableApplicationContext of Spring Framework the environment carrying the arguments in environment.propertysources.propertySourceList.SimpleCommandLinePropertySource.source.optionArgs
If I try to set a Spring-defined value (e.g. --logging.level.org.springframework.web=TRACE) it works, meaning Spring logs traces
I read all possible threads on the subject but none seem to apply to my problem.
This is my Spring boot app (args are beeing passed to the SpringApplication)
#SpringBootApplication
#ComponentScan("com.mycompany")
public class App {
public static void main(String[] args) {
SpringApplication.run(App.class, args);
}
}
Here is the component and the property
#Component
public class TaskProcessor implements com.mycompnay.fwk.task.engine.TaskProcessor {
private RestTemplate restTemplate = new RestTemplate();
#Value("${mycompany.converter.converter-uri.office}")
private String converterUriOffice;
}
The parameter being passed is received by the app (extracted from debugger):
0:"--debug=true"
1:"--logging.level.org.springframework.web=TRACE"
2:"--mycompany.converter.converter-uri.office=foo"
hash:0
value:char[44]#25
I expect the property converterUriOffice to have the value foo
Instead it gets its value from the config server (http://localhost:3000/convert/office)
Links from Devilluminati did the job. Thanks a lot! To make it as clear as possible, here is what I had to do.
1- My application has a matching YML file served by the config server called application.yml
2- Inside the application.yml, I have two profiles and I only wanted the ability to override the arguments while using the local profile.
So here is what I had to add to application.yml:
spring:
profiles: local
cloud:
config:
override-system-properties: false
Once I did that (and restarted the config server to make sure it pulls the latest YML), I am able to override the value above by passing the following to the command line:
--mycompany.converter.converter-uri.office=foo
found following in the documentation https://cloud.spring.io/spring-cloud-static/Edgware.SR2/single/spring-cloud.html#overriding-bootstrap-properties
Overriding the Values of Remote Properties The property sources that
are added to you application by the bootstrap context are often
"remote" (e.g. from a Config Server), and by default they cannot be
overridden locally, except on the command line. If you want to allow
your applications to override the remote properties with their own
System properties or config files, the remote property source has to
grant it permission by setting spring.cloud.config.allowOverride=true
(it doesn’t work to set this locally). Once that flag is set there are
some finer grained settings to control the location of the remote
properties in relation to System properties and the application’s
local configuration: spring.cloud.config.overrideNone=true to override
with any local property source, and
spring.cloud.config.overrideSystemProperties=false if only System
properties and env vars should override the remote settings, but not
the local config files.
same problem here with the solution https://github.com/spring-cloud/spring-cloud-config/issues/907
The documentation is not very clear cause it says that command line
arguments always take precedence over remote configuration.
The property sources that are added to you application by the
bootstrap context are often "remote" (e.g. from a Config Server), and
by default they cannot be overridden locally, except on the command
line.
To achieve that, you have to activate the
spring.cloud.config.override-system-properties=false configuration
(the documentation only talk about System properties but it seems to
be applicable to command line arguments too).
We are using spring boot 2.0.0. We have three environments dev, staging, production. Our current config structure
dev
application-dev.yml
application-dev.properties
Likewise, we have a yml and properties file for each environment. After a year of development now the single yml file for a profile become a large monolithic config.
is it possible to have a multiple config files for a profile like below?
application-dev.yml
application-dev-sqs.yml
application-dev-redis.yml
I think there are 2 ways you can achieve this requirement.
spring.profiles.active accepts a comma-separated list of active profiles, so you can always provide dev,dev-sqs,dev-redis as the value.
Another approach is by making use of #PropertySource and a custom PropertySourceFactory to achieve this requirement. You can find an implementation which takes the value from spring.profiles.active to load one corresponding YAML file in the article below. It should be super easy to adapt the implementation to load multiple files by looking for the profile id in the name of the YAML files.
[How-to] Read profile-based YAML configurations with #PropertySource
I was dealing with a similar problem and I'd recommend using yaml configuration.
Let's describe .properties file:
Initital approach
One can use it like this:
#Component
#PropertySources({
#PropertySource("classpath:application.properties"),
#PropertySource("classpath:application-${spring.profiles.active}.properties")
})
public class AppProperties {
}
This is very easy to configure. Limitation is, that you cannot combine profiles. I mean, that when you want to use profile as dev,local where local just alters some config properties for dev profile, Spring will try to load application-dev,local.properties file, which is very likely not what you want.
Btw, this is what Spring will do for you automatically, this is useful for topics as you described.
There is no way to configure it per profile (and not for whole list). Other possibility would be, that one can specify the list in spring.config.name which is not the case at the moment.
Better approach
In short, use:
#Profile("dev")
#Configuration
#PropertySources({
#PropertySource("classpath:topic1-dev.properties"),
#PropertySource("classpath:topic2-dev.properties")
})
public class AppPropertiesDev {
}
Disadvantage is, you have to have several such config classes (dev, staging), but know you have the topics. Also you can use mutliple profiles, which are (as of my testing) loaded in order you specified. That way, your developer can easily use dev configuration and alter just what's needed for his/her testing.
Yaml approach
You can see the approach with yaml in question I asked earlier - Property resolving for multiple Spring profiles (yaml configuration), benefit is smaller amount of files - yaml has all the profiles in one file, which may or may not be what you want.
Yes, it's possible. spring.config.location is used to externalize the config file location in Spring boot applications. This can be used to provide a location of the file in the filesystem or even in the classpath. Based on how you want to provide your application access to the files, you can choose the URI.
Doing it programmatically:
#SpringBootApplication
public class Application {
public static void main(String[] args) {
ConfigurableApplicationContext applicationContext = new SpringApplicationBuilder(Application.class)
.properties("spring.config.location:classpath:/application-dev.yml,classpath:/application-dev-sqs.yml,classpath:/application-dev-redis.yml")
.build()
.run(args);
}
}
Doing it via environment variables:
set SPRING_CONFIG_LOCATION=classpath:/application-dev.yml, \
classpath:/application-dev-sqs.yml, \
classpath:/application-dev-redis.yml
So, you can provide your files as comma-separated values.
I've used classpath here, it can also be a location in the file system:
/home/springboot-app/properties/application-dev.yml,/home/springboot-app/properties/application-sqs.yml,/home/springboot-app/properties/application-redis.yml
Have you tried including profiles yet ?
Example with profile default, you want to load additional properties for redis and db. Within application.properties file, add:
spring.profiles.include=redis, db
This will load files application-redis.properties and application-db.properties respectively
I have develop a new Connector. This connector requires to be configured with two parameters, lets say:
default_trip_timeout_milis
default_trip_threshold
Challenge is, I want read ${myValue_a} and ${myValue_a} from an API, using an HTTP call, not from a file or inline values.
Since this is a connector, I need to make this API call somewhere before connectors are initialized.
FlowVars aren't an option, since they are initialized with the Flows, and this is happening before in the Mule app life Cycle.
My idea is to create an Spring Bean implementing Initialisable, so it will be called before Connectors are init, and here, using any java based libs (Spring RestTemplate?) , call API, get values, and store them somewhere (context? objectStore?) , so the connector can access them.
Make sense? Any other ideas?
Thanks!
mmm you could make a class that will create the properties in the startup and in this class obtain the API properties via http request. Example below:
public class PropertyInit implements InitializingBean,FactoryBean {
private Properties props = new Properties();
#Override
public Object getObject() throws Exception {
return props;
}
#Override
public Class getObjectType() {
return Properties.class;
}
}
Now you should be able to load this property class with:
<context:property-placeholder properties-ref="propertyInit"/>
Hope you like this idea. I used this approach in a previous project.
I want to give you first a strong warning on doing this. If you go down this path then you risk breaking your application in very strange ways because if any other components depend on this component you are having dynamic components on startup, you will break them, and you should think if there are other ways to achieve this behaviour instead of using properties.
That said the way to do this would be to use a proxy pattern, which is a proxy for the component you recreate whenever its properties are changed. So you will need to create a class which extends Circuit Breaker, which encapsulates and instance of Circuit Breaker which is recreated whenever its properties change. These properties must not be used outside of the proxy class as other components may read these properties at startup and then not refresh, you must keep this in mind that anything which might directly or indirectly access these properties cannot do so in their initialisation phase or your application will break.
It's worth taking a look at SpringCloudConfig which allows for you to have a properties server and then all your applications can hot-reload those properties at runtime when they change. Not sure if you can take that path in Mule if SpringCloud is supported yet but it's a nice thing to know exists.
By default Spring Boot will automatically load properties from classpath:/application.properties
I want to know where is this auto configuration source code.
I want to exclude from my app.
IE: #EnableAutoConfiguration(exclude=XXXXAutoconfiguration.class)
The reason is:
Because I cannot override the default application.properties by an external property using #PropertySource
#SpringBootApplication
#ComponentScan(basePackages = {"com.test.green.ws"})
#PropertySource(value = {"classpath:/application.properties", "file:/opt/green-ws/application.properties"})
public class GreenWSApplication {
public static void main(String[] args) {
SpringApplication.run(GreenWSApplication.class, args);
}
}
There are many ways to override property keys without disabling the whole externalized configuration feature; and that's actually the goal.
You can see here the order the properties are considered in. For example, you can add that external properties file in a config folder right next to the packaged JAR, or even configure the file location yourself.
Now if you really want to disable all of that (and the Boot team strongly suggests not to do that), you can register your own EnvironmentPostProcessor (see here) and remove PropertySources from MutablePropertySources, which you can fetch with configurableEnvironment. getPropertySources().
There's no easier way to do that because:
this comes really early in the application init phase, before auto-configurations
this is not something you should do, as it will have many side effects
Background:
I have a Spring Boot 1.4 application running with Spring Cloud. My app is using the native profile to pull settings from an external config.properties file. The config server is embedded inside the same application.
In the config.properties file I have added the following setting:
app.setting=helloworld
What works:
When I change the property and send a REFRESH event, relevant beans marked are reloaded and the change is recognized correctly.
What doesn't work:
If I actually remove the property from config.properties, (by commenting it out for instance), the REFRESH event does nothing to actually refresh the application. Spring Cloud seems to not recognize the fact that the property is removed and when the data-binder proceeds to update the state of the world it misses the fact that the property is removed, and the corresponding bean linked to it must also be refreshed and its field set to blank/null, etc.
It looks like the data-binder only looks at what is at the moment available in the configuration and does not keep record of what was vs what is.
The only way to actually disable that setting in the bean configuration state is not by removing it, but by actually setting it to a blank value (which is a new value, given the setting is just a String). Note the field in Java bean mapped to this property has no default value other than null, and the value is not defined anywhere else (as in an embedded application.properties file, etc)
What might I be missing?
Is this a feature? Bug?
Thanks for your time.
Not sure if this is applicable to you, but I had a similar issue with beans annotated with #ConfigurationProperties and registered using #EnableAutoConfiguration:
#ConfigurationProperties(prefix="example")
#RefreshScope
public class MyConfig {
private List<String> values;
}
#EnableAutoConfiguration(MyConfig.class)
public class ApplicationConfiguration {
}
The problem I was experiencing is when you had a YAML configuration like:
example:
- Some
- Values
- Here
removing items from the list did not remove them from MyConfig.values when the context is refreshed.
The cause of this was that registering MyConfig using #EnableAutoConfiguration does not allow you to change the bean's scope, meaning that the bean does not get recreated when refreshing the context. See Github Issue.
My Fix
I removed MyConfig from #EnableAutoConfiguration and explicitly added a #Component annotation:
#Component
#ConfigurationProperties(prefix="example")
#RefreshScope
public class MyConfig {
private List<String> values;
}
#EnableAutoConfiguration
public class ApplicationConfiguration {
}
After this, removing items from the YAML list gets reflected in MyConfig when the context is refreshed.
I've run into a similar issue when refreshing an external config.properties file. The issue manifested itself with my config.properties because it only had a single entry in it.
To demonstrate:
Start by overriding the application properties
application.properties
some.value=abc
config.properties
some.value=xyz
Initially, using some.value will render "xyz"
To show that the value can be updated, change the value in the config.properties
config.properties
some.value=123
Using the /refresh endpoint, refresh the context and then using some.value will render "123"
Now, by removing the property, then we can see the value does not get updated
config.properties
// now empty
Using the /refresh endpoint, refresh the context and then using some.value will still render "123". It hadn't recognised that the field had been removed, nor used the "abc" value from the application.properties.
The issue stems from the class ConfigFileApplicationListener which on line 428, identifies the properties file as empty, so doesn't load the file into the property sources that are later used to compare the new properties to the old in the ContextRefresher class. The scenario appears to keep the old one in memory.
To workaround this issue, when you only have a single property, you could add property like a.b which would force the file to be loaded with the no value and result in the correct functionality.
config.properties
a.b=true
// item removed, but use some property to make sure it's read later
Hope this helps