Unable to generate different spring-cloud-gateway routes based on active Spring profile - spring-boot

I'm trying to define different routes in Spring Cloud Gateway using Spring profiles. Everything I've read about how Spring manages profiles using yaml sems to indicate it should work, but for the life of me it just ... doesn't seem to. (My other Spring apps use properties, so I'm unused to the yaml config -- it's possible I'm missing something.)
Effectively what I'm trying to do is to have a "prod" profile which contains the URIs for my production servers, and a "dev" profile which contains the localhost equivalents.
I have two profiles, dev and prod. My application.yml file looks like this
spring:
profiles:
default: prod
management:
endpoint:
health:
enabled: true
endpoints:
web:
exposure:
include: hystrix.stream, gateway
---
spring
profiles: prod
cloud:
gateway:
routes:
- id: test_route
uri: http://foo.mycompany.com
predicates:
- Path=/status
- Method=GET
---
spring
profiles: dev
cloud:
gateway:
routes:
- id: test_route
uri: http://localhost:8080
predicates:
- Path=/status
- Method=GET
My understanding is as follows:
the spring.profiles.default property tells Spring that, if no profile is specified, to use the prod profile
Spring will treat the --- as a "file separator" and re-evaluate each set of properties and overwrite previous values if the spring.profiles parameter evaluates true
Given this understanding, I would expect Spring to parse the "default" properties first, learning that the default activated profile should be prod. Then it will parse the prod properties. Since "prod" is an active profile (the only active profile, being the default), it should parse and apply the prod routes. Then it would parse the dev routes, but recognize that dev is not an active profile, and not overwrite those values. This is my understanding from reading the documentation on how to change config based on the environment.
However, when I load this, and I hit the actuator endpoint -- /actuator/gateway/routes -- I get back [] where I would expect to see the prod routes. I do see in my logs that the prod profile is activated, but it seems like not having the properties in the "default" section at top causes them to not be applied when the parser reads them out of the profile section.
The other thing I tried was putting the "dev" properties as the defaults, and then attempting to use the "prod" profile properties to overwrite the URIs. A similar issue happened there -- I hit the actuator endpoint and got back routes, but they were just the dev ones from the default.
How can I leverage Spring profiles to configure different Spring Cloud Config routes in my application.yml ?
Versions:
spring-cloud-gateway 2.0.1.BUILD-SNAPSHOT (to get a workaround for this bug, probably not relevant)
spring-cloud-starter-gateway
spring-boot 2.0.3.RELEASE
spring-boot-starter-webflux
spring-boot-starter-actuator
(I can't use Spring Cloud Config for political reasons. My company's chief architect has a severe case of Not Invented Here Syndrome.)

You cannot use spring.profiles.default in the property file. It will be too late for setting such value.
So you can set it using program argument (or System property). E.g.
java -jar --spring.profiles.default=dev your-app.jar
Or you can do it in the code by hardcoding the default profile:
#SpringBootApplication
public class DemoApplication {
public static void main(String[] args) {
SpringApplication application = new SpringApplication(DemoApplication.class);
ConfigurableEnvironment environment = new StandardEnvironment();
environment.setDefaultProfiles("dev");
application.setEnvironment(environment);
application.run(args);
}
}
Some related information you can find here by reading all comments:
https://github.com/spring-projects/spring-boot/issues/1219

Related

Spring boot 2.4.x cannot handle multi document yml files from config server

Java version: 8
Spring Boot version: 2.4.1
Spring Cloud version: 2020.0.0, specifically I use a Spring Cloud Config Server connected to GIT and our services are Spring Cloud Config Clients.
I have migrated away from using bootstrap.yml and started using spring.config.import and spring.config.activate.on-profile as mentioned in the documentation here and here
My configuration in my service, who is a client to the config server looks like this:
server.port: 9001
spring:
application.name: my-rest-service
config.import: configserver:http://localhost:8888
cloud.config.profile: ${spring.profiles.active}
My configuration in the config server looks like this:
application.yml (has two documents separated by the ---)
logging:
file.name: <omitted>
level:
root: INFO
---
spring:
config.activate.on-profile: dev
logging.level.root: DEBUG
my-rest-sercive.yml (has two documents separated by the ---)
spring:
datasource:
driver-class-name: <omitted>
username: <omitted>
password: <omitted>
---
spring:
config.activate.on-profile: dev
datasource.url: <omitted>
Because there is a profile "dev" active, I successfully get the following 4 configurations from config server:
application.yml: general logging level
application.yml: specific logging for dev
my-rest-sercive.yml: general datasource properties
my-rest-sercive.yml: specific datasource url for dev
I can see these 4 sources successfully being fetched when I use my browser or when I debug or in the logs when I lower the loglevel to trace:
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\my-rest-service.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\my-rest-service.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\application.yml'
o.s.b.c.config.ConfigDataEnvironment : Adding imported property source 'configserver:https://git.company.com/path.git/file:C:\configservergit\config\application.yml'
However, notice that because I use multi document yml files, out of these 4 property sources only TWO unique names are used.
In a later step, when Spring creates the data source bean, he complains he cannot find the data source URL. If I debug the spring bean factory I can indeed see that out of the 4 property files returned by the config server, only two have remained (the ones that don't contain the dev profile specific configuration). I assume this is because they have an identical name and they overwrite each other. This is an effect of this piece of code in the MutablePropertySource.class:
public void addLast(PropertySource<?> propertySource) {
synchronized(this.propertySourceList) {
this.removeIfPresent(propertySource); <-- this is the culrprit!
this.propertySourceList.add(propertySource);
}
}
This is a breaking change from Spring 2.3/Spring Cloud Hoxton where it correctly collected all properties. I think spring cloud needs to change the config server so that every document within a yml has has a unique name when returned to Spring. This is exactly how Spring Boot handles multi document yml files, by appending the String (documenyt #1) to the property source name
I found an interesting note about profiles and multi document yml, basically saying it is not supported, but this doesn't apply to my use case because my yml files are not profiles based (there is no -{profileName} in the last part of the file name).
This is a known issue with the new release. We can track the issue here on the spring cloud config server github page.
The workaround seems to be stop using multi document yml files and use multiple distinct files with the profile name in the filename.

Disable JMS for profile

In Springboot 2 I can do something like this to disable the embedded servlet container:
spring:
main:
web-application-type: none
Now I am looking for a similar setting to disable JMS. Currently I am using a profile, something like this:
#Profile("!nojms")
public class MQListener {
...
and then using an application-lala.yaml with content:
spring:
main:
web-application-type: none
profiles:
active: nojms
But now when I use the profile "lala" then the JMS listener is still starting.
As you're activating via CLI args the profiles.active won't get triggered as you've essentially already activated a profile.
You can add spring.profiles.include to the application-lala to unconditionally active other profiles.
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-profiles.html#boot-features-adding-active-profiles
The spring.profiles.active property follows the same ordering rules as other properties: The highest PropertySource wins. This means that you can specify active profiles in application.properties and then replace them by using the command line switch.
Sometimes, it is useful to have profile-specific properties that add to the active profiles rather than replace them. The spring.profiles.include property can be used to unconditionally add active profiles. The SpringApplication entry point also has a Java API for setting additional profiles (that is, on top of those activated by the spring.profiles.active property). See the setAdditionalProfiles() method in SpringApplication.
spring:
main:
web-application-type: none
profiles:
include: nojms

Why spring client cannot obtain spring config server properties?

I've got a #SpringBootApplication, running with the production profile, and a spring config server,
{"name":"config-client","profiles":["production"],"label":null,"version":"97611975e6ddb87c7213e18ddbe203ab6ae5485d","state":null,"propertySources":[{"name":"http://git/scm/abm/abm-settings.git/application-production.yml","source":{"my.pretty.property.id":21}}]}
I cannot load property my.pretty.property.id from server (they are always null), I am using
#Getter
#Setter
#Component
#ConfigurationProperties(prefix = "my.pretty.property")
public class MyProperties {
private String id;
}
and my bootstrap.yml is
spring.cloud:
config:
uri: http://${SERVICE_HOST}/${PROJECT_KEY}-config-server
enabled: false
failFast: true
build.gradle contains this:
"org.springframework.cloud:spring-cloud-starter-consul-all",
"org.springframework.cloud:spring-cloud-consul-core",
"org.springframework.cloud:spring-cloud-starter-hystrix",
"org.springframework.cloud:spring-cloud-starter-hystrix-dashboard",
"org.springframework.cloud:spring-cloud-starter-zipkin",
"org.springframework.cloud:spring-cloud-config-client"
My client application is normally built and deployed, what am I missing?
From Spring Cloud Config:
The HTTP service has resources in the form:
/{application}/{profile}[/{label}]
/{application}-{profile}.yml
/{label}/{application}-{profile}.yml
/{application}-{profile}.properties
/{label}/{application}-{profile}.properties
As you mentioned your application is running under production profile - hence Spring will try to find file: <cloud-config-server-url>/application-production.yml and load it, besides application.properties. But your properties are located in application-integration.yml and you don't use integration profile.
So the solution is to use an integration profile or to create an application-production.yml on your config. server.
The following config is right, but spring.cloud.config.enabled should be true, I didn't overrride it in my production environment

Override single entry in application.yml for all tests

Within the SpringBoot application you can provide the configuration via src/main/resource/application.yml.
One single entry should be overriden by the tests (see How to mock Eureka when doing Integration Tests in Spring? ). I tried to provide a test configuration with src/integration-test/resource/application.yml but it overrides the complete configuration.
eureka:
client:
enabled: false
How can I modify one entry of the configuration file for all tests?
Create a application-test.yml in src/main/resource/ with your desired configuration (eureka.client.enabled=false) and also other configurations that you require for your application to start up,
Once your application-test.yml is complete You just need to add following annotation with desired value to your test class ,
#SpringBootTest(value={"spring.profiles.active=test"})
Here spring.profiles.active=test because we have set its value to test because we want to read configurations from application-test.yml.
Hope it helps !

Spring Config-Client doesn't refresh if Config-Server is down during initial startup

I am running a test with a barebones Spring cloud config-server and a client-application. I executed a refresh scenario (by calling /refresh endpoint on the client-application)
after config-server was down initially. Here is what I found
Client starts up with locally packaged properties when config-server is not reachable on startup. (I have the properties in application.yml that is bundled with client-application)
Git backend has different values for the properties compared to locally packaged version. Config-server is aware of the changes in git (Confirmed by connecting directly to config-server)
I bring up config-server and do a POST to /refresh endpoint on the client-application.
Client-application is not aware of the new properties from config-server.
In the second usecase
Client-application starts up and connects to config-server successfully. I see that the values from config-server have been fetched by the client-application successfully
I make a change in Git and call the /refresh endpoint on the client-application. Properties are refreshed successfully.
At this point it looks like /refresh doesn't work if the client-application comes up initially without being able to successfully connect to config-server. I am doing this to test
a fallback strategy for the client-application if config-server is not reachable when the client-application is starting up. (The fallback strategy is to have locally packaged properties
that will be used if config-server is not available on startup. If the config-server is available then the local properties are overriden). Any pointers to why this is not working and
what I could do differently? Thanks in advance.
Edit
Server-Code
#EnableConfigServer
#SpringBootApplication
public class ConfigServerApplication {
public static void main(String[] args) {
SpringApplication.run(ConfigServerApplication.class, args);
}
}
Client-Code
#RestController
#RefreshScope
#Component
public class Greeter {
#Value("${message.greeting}")
String greeting;
#RequestMapping(value = "/",produces = "application/json")
public List<String> index(){
List<String> env = Arrays.asList("message.greeting: " + greeting);
return env;
}
}
bootstrap.yml (On config-client application)
spring:
application:
name: configclient
cloud:
config:
uri: http://localhost:8888
management:
security:
enabled: false
logging:
config: classpath:logback.xml
server:
port: 8000
application.yml
message:
greeting: Hello from Local!
Config in Git (Served through config-server)
message:
greeting: Hello from Git-Edited!
According to spring-cloud-config documentation -
If you expect that the config server may occasionally be unavailable
when your app starts, you can ask it to keep trying after a failure.
First you need to set spring.cloud.config.failFast=true, and then you
need to add spring-retry and spring-boot-starter-aop to your
classpath. The default behaviour is to retry 6 times with an initial
backoff interval of 1000ms and an exponential multiplier of 1.1 for
subsequent backoffs. You can configure these properties (and others)
using spring.cloud.config.retry.* configuration properties.
Reference -> http://cloud.spring.io/spring-cloud-static/spring-cloud-config/1.3.1.RELEASE/

Resources