I am trying to deploy a Spring Boot application for which I am not able to change the code. I'd prefer to configure it using YML, however I am hitting this YAML limitation:
Elements have a value or children, never both.
The config scheme for the application includes multiple instances of:
com.company.feature=default
com.company.feature.config=value
Is there any way to configure these as YAML, as the following would be invalid:
com:
company:
feature: default
config: value
You can't put it all in one hierarchy. Any property whose key has children and a value must be listed with the value without the heirarchy.
GOOD EXAMPLE:
com:
company:
feature:
config: value
com.company.feature: default
BAD EXAMPLE
com:
company:
feature:
config: value
com:
company:
feature: default
If you tried to do previous code then according to YAML, you overwrote com:company:feature with default instead of config: value
Related
Java version: 8
Spring Boot version: 2.4.1
Spring Cloud version: 2020.0.0, specifically I use a Spring Cloud Config Server connected to GIT and our services are Spring Cloud Config Clients.
I have migrated away from using bootstrap.yml and started using spring.config.import and spring.config.activate.on-profile as mentioned in the documentation here and here
My configuration in my service, who is a client to the config server looks like this:
server.port: 9001
spring:
application.name: my-rest-service
config.import: configserver:http://localhost:8888
cloud.config.profile: ${spring.profiles.active}
My configuration in the config server looks like this:
application.yml (has two documents separated by the ---)
logging:
file.name: <omitted>
level:
root: INFO
---
spring:
config.activate.on-profile: dev
logging.level.root: DEBUG
my-rest-sercive.yml (has two documents separated by the ---)
spring:
datasource:
driver-class-name: <omitted>
username: <omitted>
password: <omitted>
---
spring:
config.activate.on-profile: dev
datasource.url: <omitted>
Because there is a profile "dev" active, I successfully get the following 4 configurations from config server:
application.yml: general logging level
application.yml: specific logging for dev
my-rest-sercive.yml: general datasource properties
my-rest-sercive.yml: specific datasource url for dev
I can see these 4 sources successfully being fetched when I use my browser or when I debug or in the logs when I lower the loglevel to trace:
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\my-rest-service.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\my-rest-service.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\application.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\application.yml'
However, notice that because I use multi document yml files, out of these 4 property sources only TWO unique names are used.
In a later step, when Spring creates the data source bean, he complains he cannot find the data source URL. If I debug the spring bean factory I can indeed see that out of the 4 property files returned by the config server, only two have remained (the ones that don't contain the dev profile specific configuration). I assume this is because they have an identical name and they overwrite each other. This is an effect of this piece of code in the MutablePropertySource.class:
public void addLast(PropertySource<?> propertySource) {
synchronized(this.propertySourceList) {
this.removeIfPresent(propertySource); <-- this is the culrprit!
this.propertySourceList.add(propertySource);
}
}
This is a breaking change from Spring 2.3/Spring Cloud Hoxton where it correctly collected all properties. I think spring cloud needs to change the config server so that every document within a yml has has a unique name when returned to Spring. This is exactly how Spring Boot handles multi document yml files, by appending the String (documenyt #1) to the property source name
I found an interesting note about profiles and multi document yml, basically saying it is not supported, but this doesn't apply to my use case because my yml files are not profiles based (there is no -{profileName} in the last part of the file name).
This is a known issue with the new release. We can track the issue here on the spring cloud config server github page.
The workaround seems to be stop using multi document yml files and use multiple distinct files with the profile name in the filename.
New to spring boot.
While exploring spring boot env variables, came to know that,
env variables can be accessed by ${KeyName} from code.
Got a question like,
Case 1:
In #Configuration files, we are accessing keys in application.properties using #Value(value = "${KeyName}").
So, we are using almost same syntax for accessing env variables and accessing keys in application.properties.
Case 2:
When trying to access the keys in application.properties using system.getEnv("keyname"), I got only null.
Case 3:
Recently worked on configmap in kubernetes with spring boot.
Config file looks like,
spec:
containers:
- name: demo-configconsumercontainer
image: springbootappimage:latest
ports:
- containerPort: 8080
envFrom:
- configMapRef:
name: example-configmap
All the values from configMap is exported as environmental variables and
I am accessing those values by #Value(value = "${KeyName}") and by system.getEnv(KeyName).
My question is, how case 3 is working when case 2 is not.
Is Spring boot made such a way that, it is allowing to access by ${KeyName} and not by system.getEnv(KeyName)? (ie. Case 2)
Could some one clarify my questions here.
Using #Value annotation, you can access a property from many property sources such as in application.properties or an environment variable and few more property sources.
The important point here is ordering of these property sources.
Below is the order of looking up the property in various sources.
Devtools global settings properties on your home directory (~/.spring-boot-devtools.properties when devtools is active).
#TestPropertySource annotations on your tests.
#SpringBootTest#properties annotation attribute on your tests.
Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
ServletConfig init parameters.
ServletContext init parameters.
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that only has properties in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
#PropertySource annotations on your #Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).
In your case, the property is either declared in environment variable or in application.yaml and hence accessible using #Value annotation.
See the Spring docs:
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
Spring includes environment variables as a potential property source, but does not export its properties defined in other ways as environment variables. So it isn't a two way street, which is why case #2 does not work.
Case #3 is a separate reality, just the fact that when K8s runs a container defined in this way with env vars, it makes those vars available in the container environment. Any software system or programming language able to read environment variables will be able to refer to those variables, including Java code. This has nothing to do specifically with Java or Spring...it's just another way to inject environment variables into the runtime environment.
UPDATE: I didn't see #ShaileshPratapwar's answer until I posted my own. Seems our answers are roughly the same, although I think it's good that you know where the list of property sources, and their order of priority, comes from. It's very clearly defined by the Spring docs.
We have many environments that have multiple active Spring profiles, but what is the precedence of the application-{profile}.yml files?
If I have spring.profiles.active=test-us-west-2-p1, test-us-west-2, test
In what order do the files application-test.yml, application-test-us-west-2.yml, application-test-us-west-2-p1.yml get loaded? If I have the same property in each file, which "wins"?
Also, has this changed from Spring-Boot 1.5.x to 2.x? It seems like it may have.
The profile's properties are loaded in the same order as you specify them, and if the same property is defined in different profiles the last one wins.
This behavior applies to both Spring Boot versions 1.5.x and 2.x, and if I recall correctly, it applies to all versions of Spring.
Spring always loads appication.yml. And afterwards, if some profile is specified, it will load that profile's property file. And if after that profile another profile is specified, it will load that profile's propperty file. Always overriding current properties's value with the new one.
So, let's say you have profile1 and profile2. And you have these property files:
application.yml:
property1: bob
property2: alice
property3: eve
application-profile1.yml:
property2: alice1
property3: eve1
application-profile2.yml:
property3: eve2
And you start your application with: spring.profiles.active=profile1, profile2
Your will get:
property1: bob
property2: alice1
property3: eve2
First of all, we need to find out the final set of all active profiles. There are ways of setting/replacing active profiles and adding active profiles on top of existing active ones. For instance, active profiles set with the spring.profiles.active property are replaced with the -Dspring.profiles.active command line option. (And this can get really complex.)
On the other hand, the SpringApplicationBuilder's profiles method adds to the existing active profiles. We can use the following code to figure out the final set of active profiles:
#Autowired
private Environment environment;
...
System.out.println("Active profiles: " +
Arrays.toString(environment.getActiveProfiles()));
Now we have to consider what Spring documentation calls last-wins strategy.
If several profiles are specified, a last-wins strategy applies.
So, if we have the following code and all other options excluded:
new SpringApplicationBuilder(Application.class)
.profiles("dev", "prod")
.run(args);
both application-dev.properties and application-prod.properties files are loaded and the keys with the same name in the latter one (the production) override the former one.
I posted important notice on profiles order handling. See how jar resources files are handled in this process here.
I'm trying to define different routes in Spring Cloud Gateway using Spring profiles. Everything I've read about how Spring manages profiles using yaml sems to indicate it should work, but for the life of me it just ... doesn't seem to. (My other Spring apps use properties, so I'm unused to the yaml config -- it's possible I'm missing something.)
Effectively what I'm trying to do is to have a "prod" profile which contains the URIs for my production servers, and a "dev" profile which contains the localhost equivalents.
I have two profiles, dev and prod. My application.yml file looks like this
spring:
profiles:
default: prod
management:
endpoint:
health:
enabled: true
endpoints:
web:
exposure:
include: hystrix.stream, gateway
---
spring
profiles: prod
cloud:
gateway:
routes:
- id: test_route
uri: http://foo.mycompany.com
predicates:
- Path=/status
- Method=GET
---
spring
profiles: dev
cloud:
gateway:
routes:
- id: test_route
uri: http://localhost:8080
predicates:
- Path=/status
- Method=GET
My understanding is as follows:
the spring.profiles.default property tells Spring that, if no profile is specified, to use the prod profile
Spring will treat the --- as a "file separator" and re-evaluate each set of properties and overwrite previous values if the spring.profiles parameter evaluates true
Given this understanding, I would expect Spring to parse the "default" properties first, learning that the default activated profile should be prod. Then it will parse the prod properties. Since "prod" is an active profile (the only active profile, being the default), it should parse and apply the prod routes. Then it would parse the dev routes, but recognize that dev is not an active profile, and not overwrite those values. This is my understanding from reading the documentation on how to change config based on the environment.
However, when I load this, and I hit the actuator endpoint -- /actuator/gateway/routes -- I get back [] where I would expect to see the prod routes. I do see in my logs that the prod profile is activated, but it seems like not having the properties in the "default" section at top causes them to not be applied when the parser reads them out of the profile section.
The other thing I tried was putting the "dev" properties as the defaults, and then attempting to use the "prod" profile properties to overwrite the URIs. A similar issue happened there -- I hit the actuator endpoint and got back routes, but they were just the dev ones from the default.
How can I leverage Spring profiles to configure different Spring Cloud Config routes in my application.yml ?
Versions:
spring-cloud-gateway 2.0.1.BUILD-SNAPSHOT (to get a workaround for this bug, probably not relevant)
spring-cloud-starter-gateway
spring-boot 2.0.3.RELEASE
spring-boot-starter-webflux
spring-boot-starter-actuator
(I can't use Spring Cloud Config for political reasons. My company's chief architect has a severe case of Not Invented Here Syndrome.)
You cannot use spring.profiles.default in the property file. It will be too late for setting such value.
So you can set it using program argument (or System property). E.g.
java -jar --spring.profiles.default=dev your-app.jar
Or you can do it in the code by hardcoding the default profile:
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication application = new SpringApplication(DemoApplication.class);
ConfigurableEnvironment environment = new StandardEnvironment();
environment.setDefaultProfiles("dev");
application.setEnvironment(environment);
application.run(args);
}
}
Some related information you can find here by reading all comments:
https://github.com/spring-projects/spring-boot/issues/1219
I'm trying to partition configuration properties for several Spring Boot applications. I'm using Spring Boot 1.1.6, and our configuration properties are expressed in YAML in the usual application.yml style. I've created various profiles for common base parameters, common DB parameters, etc. I was trying to use the include feature mentioned in the Spring Boot reference docs, but it seems to work as an override and not an include. I.e. exactly the opposite of what I want. Given the following content in application.yml, I would have expected the property name to have the value bar when the bar profile is active, but instead it gets set to foo (from the included profile). I thought the notion of including meant that it was loaded first, and any identically named properties set in the new profile would override those from the included profile. Kind of like if a subclass is shadowing a field from the superclass, any instance of the subclass would reflect the shadowed value. Here's the file:
spring:
profiles: foo
name: foo
--- # New YAML doc starts here
spring:
profiles:
include: foo
profiles: bar
name: bar
If I run this in a test case with the "bar" profile explicitly activated, the name property will still be foo:
SpringApplicationBuilder builder = new SpringApplicationBuilder(Application.class);
SpringApplication app = builder.application();
builder.profiles("bar");
ConfigurableApplicationContext ctxt = app.run();
String name = ctxt.getEnvironment().getProperty("name"); // Is always "foo" much to my surprise
However, if I comment out the include:
spring:
profiles: bar
# profiles:
# include: foo
and activate the two profiles explicitly in my code:
builder.profiles("foo", "bar");
Then it works as I would expect, and the name property is set to bar. The main reason I would rather handle the includes in the YAML files is that it's less impact on my actual code, and I can manage all profile inclusion in one place. With the other approach, I'd have to search for profile strings and possible #Profile annotations in my entire project if I were to ever rename the profiles. That's definitely more error prone. I think a more flexible solution would be to explicitly be able to express whether or not the included profile overrides the sub profile values or not. Maybe something like:
spring:
profiles: bar
profiles:
include: foo
override: false
Maybe I'm just missing something here. Is there a better way of doing this? Thanks.
Try the following for foo to include but override bar, seems to be working for my solution
spring:
profiles:
include: bar
active: foo,bar
edit: please mind, that it's a "hack", not officially supported and it's for 2016 version