We are thinking of migrating our Spring system to use Spring Cloud Config.
The system is made up of multiple services and some share properties for common resources such as the DB datasource. For these we have a 'base' config file which contains the commonly used properties in one place. Each of the services read the common file first and then overload their properties file on top. This saves us repeating common properties.
I can't see how we do the same with spring cloud config, I can see options for different repositories but not overloading property files.
application.yml or application.properties is common to all applications.
Related
I am implementing a Spring Boot application which will be providing a multitenant environment. That is achieved in my case by using a database schema for each customer. Example see this project.
Now I am wondering how to implement tenant-specific configurations. I am using #ConfigurationProperties to bundle my property values, but these are getting instantiated once and not for each tenant.
What if I would like to use Spring Cloud Config with multiple tenant specific git repository as an configuration backend. Would it be possible when using a jdbc backend for Spring Cloud Config?
Is there any way with default Spring mechanisms or do I have to implement a database based configuration framework myself?
Edit: For example I have two tenants called Tenant1 and Tenant2. Both are running over the same application in the same context and are writing in the database schemes tenant_1 and tenant_2.
Identification of tenants is happening over keycloak (see Spring Keycloak multi tenant example). So I identify the tenantId from the jwt token and select the database connection like described here.
But now I would need the same mechanism for #Configuration beans. Since #Configuration beans are as far as I know Singletons, so there is always ONE configuration per application scope, and not ONE configuration per tenant.
So using Spring Cloud Config Tenant1 is using https://git-url/tenant1, Tenant2 is using Hashicorp Vault as backend and perhaps Tenant3 will be using a jdbc based configuration backend. And all of that in ONE (of course scalable) application.
In case your application uses tenant specific files (html templates etc), the following can be applied. As I have used the below approach for handling many tenants and works fine and easy to maintain.
I would suggest that you maintain a consistent configuration source (JDBC) for all of your tenant configurations. This helps you have a single source that is cacheable and scalable for your application. Also, you could have your tenants navigate to a configuration page to manage their settings and alter them to suit their needs at any point of time on the fly. (Example Settings: Records Per Page, Theme, Logo, Filters etc...)
Having the tenant configuration in files in git will be a difficult task when you wanted to auto-provision tenant's when they sign-up as it will involve couple of distributed services. Having them in a TenantSettings table with the tenantId as a column could help you get the data in no time and will be easy.
You can use Spring Cloud Config for your scenario and it is adoptable. It is easily configurable and provides out of the box features. For your specific scenario, you can have any number of microservices running yet all controlled by one Spring Cloud Config Server which is connected to one Git Repository. Your all microservices are asking configuration properties from Spring Cloud Config Server and it is directly fetching properties from Git Repository. That repository can have multiple property files. It can hold common properties for all the microservices or specific service based configuration properties. If you want to keep confidential properties more securely, that is also made possible via HashiCorp vault. I will leave an image below for you to get a better idea about this concept.
In the below image, you can see the Git Repository with common configuration property files and specific configuration property files for different services yet in same repository.
I will add another image for you to get a better idea how does this can be arranged with application profiles as well.
Finally I will add something additional to show the power of Spring Cloud Config and out of the box features it allows us to play with. You can automatically refresh configuration properties in running application as well. You can configure Spring Cloud Config to do that. I will add an architectural diagram to achieve that.
References for this answer is taken from Spring in Action, Fifth Edition
Craig Walls
Having something like
security.user.password = plainTextPassword
inside Spring Boot’s application.properties is obviously an anti-pattern as it prevents the code from being pushed to a public SCM. In my non-Spring Boot projects I use
security.user.password = ${myPasswordFromMavenSettingsXML}
and put a corresponding <properties/> reference inside my pom.xml.
Using Maven’s resource filter plugin the passwords are replaced at build time so the application have access to actual plain text passwords after it has been build and deployed.
For some reason Maven’s resource filter plugin does not work in this case. Is there a way to not commit plain text passwords to an SCM and let Spring Boot to insert them at build time?
Spring boot has multiple mechanisms to provided externalized configuration. Some examples are command line arguments, environment variables and also application properties outside of your packaged JAR.
What I usually do:
Locally we configured several environment variables. Most (if not all) IDE's allow you to configure environment variables from within the run configuration.
For example if you don't want to expose the spring.datasource.password property you could set an environment variable called SPRING_DATASOURCE_PASSWORD.
When we deploy on another environment, we usually choose to add another application.properties or application.yml file within the same folder as the application JAR/WAR, since Spring boot picks that up as well.
Another solution is to use Spring cloud since it has a config service which can be used as a microservice to provide configuration. The configuration can be versioned using SCM as well, but you can put it on a separate system that is not connected to your source code.
Does anybody knows if it is possible to expose more than 1 property file per application in Spring Cloud config server?
For example I would like to have defined in my git repo properties for the same app, but in different files:
myapp-customer-services.yml
myapp-products-services.yml
and have all those properties defined inside the files, exposed under "myapp".
No that's not possible currently. I'm not sure it really makes much sense to be honest, since you can easily clearly delineate different sets of properties within a YAML file using separate documents.
Yes it is possible to expose more than 1 property file per application in Spring Cloud config server
You can access it in you client by using following properties
first specify profile which you want
example
myapp-customer.yml
myapp-products.yml
spring.profiles.active=customer,products
spring.cloud.config.name=myapp
We are using spring boot in a multi-module project.
We have a Domain access module which has the common domain object classes, repositories, together with configuration for the datasource, JPA, Hibernate, etc. These are configured using a application.properties. We put all this configuration into the common module to save duplicating these common configurations in the higher level modules.
This all works fine when building the domain module, so the configurations are loaded correctly in the test units.
However the problems start when we try to use the domain module in the higher layer modules; they have their own application.properties which means Spring loads them and not the the Domain module application.properties, which this means the data source is not configured because only the higher module application.properties are loaded.
What we would like is both the domain module and higher level application properties to be loaded by Spring. But we can't see any easy way to do this.
I'm thinking this must be a common problem, and wonder if there any recommended solutions for this problem?
As we are using spring-boot the solution should ideally use annotations instead of applictionContext.xml.
Maybe you should only use application.properties in the top-level aggregator project?
You can always use #PropertySource in the child projects to configure them with a name that is specific to their use case.
Or you can use different names for each project and glue them together in the top-level project using spring.config.location (comma-separated).
I agree with #Dave Syer. The idea of splitting an application into multiple modules is that each of those is an independent unit, in this case a jar file. Theoretically you could split each of those jar files into their own source repositories, and then use them across multiple projects. Let's say you want to reuse these domain classes in both a web and batch application, if all the APPLICATION level configuration is stored within each of the individual modules, it severely reduces their reusability.
IMO only the aggregating module should contain all of the configuration necessary to run as an application, everything else is simply a dependency that can be remixed and reused as necessary.
Maybe another approach could be to define specific profiles for each module and use the application.properties file just to specify which profiles are active
using the spring.profiles.include property.
domain-module
- application.properties
- application-domain.properties
app-module
- application.properties
- application-app.properties
and into the application.properties file of app-module
spring.profiles.include=domain,app
Another thing you can do (besides only using application.properties at the top-level as Dave Syer mentions) is to name the properties file of the domain module something like domainConfig.properties.
That way you avoid the name clash with application.properties.
domainConfig.properties would contain all the data needed for the domain module to be able to tested on it's own. The integration with the rest of the code can easily be done either using multiple #PropertySource (one for domainConfig.properties and one for application.properties) or configuring a PropertySourcesPlaceholderConfigurer bean in your Java Config (check out this tutorial) that refers to all the needed property files
in spring-boot since 2.4 support spring.config.import
e.g
application.name=myapp
spring.config.import=developer.properties
# import from other module
spring.config.import=classpath:application-common.properties
or with spring.config.activate.on-profile
spring.config.activate.on-profile=prod
spring.config.import=prod.properties
ref: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4
I'm using Spring 2.5.6. I have a bean whose properties are being assign from a property file via a PropertyPlaceholderConfigurer. I'm wondering whether its possible to have the property of the bean updated when the property file is modified. There would be for example some periodic process which checks the last modified date of the property file, and if it has changed, reload the bean.
I'm wondering if there is already something that satisfies my requirements. If not, what would be the best approach to solving this problem?
Thanks for your help.
Might also look into useing Spring's PropertyOverrideConfigurer. Could re-read the properties and re-apply it in some polling/schedular bean.
It does depend on how the actual configured beans use these properties. They might, for example, indirectly cache them somewhere themself.
If you want dynamic properties at runtime, perhaps another way to do it is JMX.
One way to do this is to embed a groovy console in your application. Here's some instructions. They were very simple to do, btw - took me very little time even though I'm not that familiar with groovy.
Once you do that you can simply go into the console and change values inside the live application on the fly.
You might try to use a custom scope for the bean that recreates beans on changes of the properties file. See my more extensive answer here.
Spring Cloud Config has facilities to change configuration properties at runtime via the Spring Cloud Bus and using a Cloud Config Server. The configuration or .properties or .yml files are "externalized" from the Spring app and instead retrieved from a Spring Cloud Config Server that the app connects to on startup. That Cloud Config Server retrieves the appropriate configuration .properties or .yml files from a GIT repo (there are other storage solutions, but GIT is the most common). You can then change configuration at runtime by changing the contents of the GIT repo's configuration files--The Cloud Config Server broadcasts the changes to any Client Spring applications via the Spring Cloud Bus, and those applications' configuration is updated without needing a restart of the app. You can find a working simple example here: https://github.com/ldojo/spring-cloud-config-examples