Upon running a Spring Cloud Config application with a Git backend a clone is performed into the default location of c:/user/AppData/Temp/local/<config-repo-1>. Cloning can take a long time for larger applications (currently one of my teams takes ~15 min).
When the Config Server is stopped and restarted again instead of using the same local repo another clone is performed, c:/user/AppData/Temp/local/<config-repo-2>.
Is there a way to change this behavior so that restarting the Config Server does not do a new clone but instead just does a fetch for the existing clone.
Thanks in advance for any assistance provided.
spring:
config:
server:
git:
clone-on-start: false
I saw this option somewhere, but couldn't figure out where I got it. Unfortunately, the official spring documentation does not refer to this option. This will not clone the rep when starting.
The other Option from the Spring Documentation is:
spring:
config:
server:
git:
force-pull: false
As documented:
... there is a force-pull property that makes Spring Cloud Config Server force pull from the remote repository if the local copy is dirty
My application.properties files of my Cloud Config Server looks like this.
config.source=Local Cloud Server #just to check where config come from
server.port=8012
encrypt.key=xxxxxxxxxxxxxx
spring.profiles.active=staging
spring.application.name=my-config-server
# Git Backend
spring.cloud.config.server.git.username=MY_USERNAME
spring.cloud.config.server.git.password=ghp_MY-DEV-ACCESS-TOKEN
spring.cloud.config.server.git.uri=https://github.com/my-username/app-config
spring.cloud.config.server.git.clone-on-start=true
spring.cloud.config.server.git.basedir=file://${user.dir}/cloned_configurations
On startup a new folder "cloned_configuration" is created inside of Cloud Config Server folder. I see all of my propertie files cloned form github with correct values.
But for some reason none of these are used. For e.g config.source must have the value "GitHub" because the cloned application.repository has an entry
config.source=GitHub
But on application start I see "Local Cloud Server". All other settings are also not used from cloned properties.
With Postman I can receive all Configs without any issues. But none of theme are used by my Config Server or any of my other webservices.All webservices and the config server using their own application.properties file.
What I do wrong?
You probably need to move your config server properties into your bootstrap.properties file instead of application.properties.
spring.cloud.config.server.git.username=MY_USERNAME
spring.cloud.config.server.git.password=ghp_MY-DEV-ACCESS-TOKEN
spring.cloud.config.server.git.uri=https://github.com/my-username/app-config
spring.cloud.config.server.git.clone-on-start=true
spring.cloud.config.server.git.basedir=file://${user.dir}/cloned_configurations
https://cloud.spring.io/spring-cloud-config/multi/multi__spring_cloud_config_client.html
We are planning to utilize Spring Cloud Config for our service. Our biggest concern is that when the container starts up, it relies on github to be available all the time so that it can pull the config files. In case github is down, what is the best practice to mitigate the issue?
I was thinking of storing a local folder of the configs as a backup and configuring the application.yml to fallback to it (I do not know how).
I was going to use a Composite Environment Repositories
Please see here: Section 2.1.8
However it states:
Any type of failure when retrieving values from an environment repository results in a failure for the entire composite environment.
This means if the git retrieve fails, it does not fall back to the local component of the composite. I wish it did. Have any of you handled a similar problem? How did you solve it?
Here is a good article about best practices. However, I need a workaround for case 1: Best practices on handling GIT repository inavailability
Spring-Cloud has a configuration property to handle this issue;
spring.cloud.config.server.git.basedir = /your/config/local/fallback/directory
NOTE - If you're using a .yml file, then define the above property as
per yaml conventions.
To have a background knowledge, look at the documentation: http://cloud.spring.io/spring-cloud-static/Finchley.RC1/single/spring-cloud.html#_version_control_backend_filesystem_use
So essentially what happens here is that - as long as your application was initially able to connect to the git repository which you set up in spring.cloud.config.server.git.uri = https://your-git/config-repo.git, then on config-server/container startup, the directory you have defined in your spring.cloud.config.server.git.basedir gets created locally and by default spring-cloud clones your configurations into this directory to be available as fallback.
So whenever your git repository is unreachable, spring-cloud will pick up your configurations from this base directory.
Important things to note:
Unless you really want to re-clone the git configurations only on config-server startup alone, ensure that the property spring.cloud.config.server.git.clone-on-start is NOT set to true or is entirely not set at all - Otherwise, every time you restart your cloud-config service the configurations will be deleted and freshly cloned again and if the repository is not available at the time, application startup will fail - and you perhaps don't want that.
However, if spring.cloud.config.server.git.clone-on-start is set to false or is not even set at all (in which case the default is false), then the git repository will only be cloned on demand - hence if the repository is unreachable, spring-cloud will fallback gracefully to pick up configurations from the spring.cloud.config.server.git.basedir
Even when the application config-server (or its container) is restarted and the git repository is not reachable, you will see something like below;
No custom http config found for URL: https://your-git/config-repo.git/info/refs?service=git-upload-pack
... s.c.a.AnnotationConfigApplicationContext : Refreshing org.springframework.context.annotation.AnnotationConfigApplicationContext#3a26f314: startup date [Mon Oct 15 22:01:34 EDT 2018]; root of context hierarchy
... o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: file:/your/config/local/fallback/directory/application.properties
Notice the line:
Adding property source:file:/your/config/local/fallback/directory/application.properties
That's where the magic happens.
So if you want the spring.cloud.config.server.git.basedir to be available as a fallback even before the first startup of your config-server (and whether or not your git repo is unreachable during the startup), you can carry out the following steps;
Manually create the spring.cloud.config.server.git.basedir
From your terminal cd /your/config/local/fallback/directory
git clone https://your-git/config-repo.git while the repo is available
Ensure that all your config files/folders/sub-folders including the .git folder are cloned directly to the root of the fallback directory.
For instance, there's a tendency that git clone https://your-git/config-repo.git will clone the repo into the fall back directory as /your/config/local/fallback/directory/config-repo. You will have to copy every darn content of config-repo - including the .git folder too - out and directly into /your/config/local/fallback/directory
Start the config-server (or its container) for the first time or whenever! ......... Voila!!
I have a simple Spring Cloud Config Server which consume configuration from git server.
ConfigServer bootstrap.yml :
spring:
application:
name: config-service
cloud:
config:
server:
git:
uri: ssh://git#mydomain:myport/myrepo.git
searchPaths: "configurations/{application}/{profile}"
server:
port: 8888
When I deploy ConfigServer on local, I can retrieve configuration from http://localhost:8888/myapp/test. But when I deploy ConfigServer on test server, it throws No such label: master when I hit http://testserverip:8888/myapp/test.
Any help would be most appreciated!
I know that I'm posting this answer very very late. But, still posting it so that other folks can find this helpful. Now a days, default branch name of GitHub is "main", but Spring Cloud Config still looks for "master" as a label name. So, if you want to change the default behavior, you can always change the label name look up using below property in application.yml or application.properties:
spring.cloud.config.server.git.default-label=main
This worked very well for me. I'm using Spring Boot 2.3.6.RELEASE
GitHub created default branch "main" , however Spring cloud Config was looking for "master branch". I have created a "master" branch and it worked!
I faced this same annoying scenario.
I'll first state how I solved this scenario and will later emphasize my mistakes which I did in my previous approaches.
In my context, I'm using application.properties
My application overview looks like below:
I have a config-server which is centralized & provides the respective configuration data to the respective micro-services.
For instance, a micro-service 'limits-service', requires some configuration data, it gets it from the central configuration server('spring-cloud-config-server').
Hence to achieve this, 'limits-service' queries the central config server which in-turn fetches the requested data from a remote git branch('spring-cloud-samples').
┌---------------------- <--> [currency-exchange-service]
[git] <--> [spring-cloud-config-server] ------- <--> [limits-service]
└---------------------- <--> [currency-conversion-service]
Solution:
I simply created a new Git Repository (spring-cloud-samples) for all the configuration files which would be consumed by several micro-services through a central configuration server.
In the application.properties file of central configuration server('spring-cloud-config-server'), I have provided a property as:
spring.cloud.config.server.git.uri=https://github.com/{username}/{git-reponame.git}
which might look like
spring.cloud.config.server.git.uri=https://github.com/alice123/spring-cloud-samples.git
That's it!
On starting my central-config-server & observe the log which is as follows:
2020-02-17 05:25:07.867 INFO 15000 --- [nio-8888-exec-9] o.s.c.c.s.e.NativeEnvironmentRepository : Adding property source: file:/D:/Users/{userName}/AppData/Local/Temp/config-repo-3453413414/limits-service.properties
Moreover, I also consumed the config-data through my 'limits-service' & it worked like a charm!
I tried the following things that resulted in failures and a lot of screaming & yelling. XD
I'm posting it, just so that someone who's trying this same concept might save a night's work or maybe less :p
1) Initially, I was using my Git Repositories SSH URL in my application.properties file of central-config-server like follows:
spring.cloud.config.server.git.uri=git#github.com:alice123/spring-cloud-samples.git
spring.cloud.config.server.git.username=alice123
spring.cloud.config.server.git.password=alice123Pwd
which resulted in the following error:
2020-02-17 05:22:45.091 WARN 15000 --- [nio-8888-exec-1] .c.s.e.MultipleJGitEnvironmentRepository : Error occured cloning to base directory.
org.eclipse.jgit.api.errors.TransportException: git#github.com:aniketrb-github/spring-cloud-samples.git: Auth fail
2) Later, I tried to read the config data from a native/local git directory, which was pointing to my Git remote branch(spring-cloud-samples)
Then I provided the following in my application.properties file:
spring.cloud.config.server.git.uri=file://D:\aniket-workspace\github-ws\microservices\udemy-microservices\git-localconfig-repo
This crashed with:
java.lang.IllegalStateException: Failed to load property source from location 'classpath:/application.properties'
3) Later after enough googling, I changed the above property to below which worked!!:
spring.cloud.config.server.git.uri=D:\\\\alice-workspace\\\\github-ws\\\\microservices\\\\git-localconfig-repo
The 'limits-service' eventually failed while it tried to get the config-data from 'spring.cloud.config.server' with following error:
404: Not Found - "org.springframework.cloud.config.server.environment.NoSuchLabelException: No such label: master"
The solution I've stated worked for me due to all the failures I came across which I mentioned above.
Please correct me if I'm wrong & improvise if necessary. I hope this helps and saves some crucial time.
References: spring-cloud-config, https-or-ssh, git-ssh-windows, git-ssh-windows-troubleshooting
This happens when your service cannot access to Git repository.
Since you can start your application, I assume test server can reach to Git. You should set up SSH authentication to your Git server. You can check this site for SSH configuration.
In my case nothing of given answers worked. only thing that worked is setting default-label to master
spring.cloud.config.server.git.default-label=master
When you create a new repository, nowdays git makes "main" as the default branch instead of master. This issue is all about this.
Create "master" branch manually. It will solve all these issues.
for me this caused due to my default branch 'main'. whenever you create a new branch the default branch is 'main' not 'master' anymore. but spring server config look for master branch.
the quick solution is to checkout a new branch called 'master' from 'main' branch. and try from spring config server. thats it.
to create a new branch from main branch. first point to main branch and from there
git checkout -b master
git push --set-upstream origin master
restart your config server and test
I hope you are doing well.
I am using STS 4 and added source link folder (local git repo) for spring cloud configuration from local git repo.
I had facing same issue,
I did just change back slashes with forward slashes.
when i was using like this
spring.cloud.config.server.git.uri=file://C:\Users\coder\Desktop\spring material\in28microservies\git-localconfig-repo
I got error
2020-04-04 14:57:37.874 ERROR 6952 --- [nio-8888-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: Cannot load environment] with root cause
java.lang.IllegalStateException: Cannot convert uri to file: file://C:UserscoderDesktopspring materialin28microserviesgit-localconfig-repo
Then i just replaced all back slashes with forward slashes in config server properties file, and it worked for me
spring.cloud.config.server.git.uri=file://C:/Users/coder/Desktop/spring material/in28microservies/git-localconfig-repo
spring.profiles.active=native
spring.cloud.config.server.native.searchLocations=file://<location>
Try this as mentioned by our friend below:
https://stackoverflow.com/a/33179106/1908827
This will work for sure !!! :)
1 ) You have to commit your changes in following way to show up the master if you initialize the repo for first time in that particular folder
> git add -A .
> git commit -m "added prop..."
2) re-run the project and hit same url.
Njoy :)
This is what I found when I too had a similar issue.
Issue/Mistake#1 >> I created the properties file using notepad(I am on Windows OS).
The file was created with .txt extension. Hence the cloud config was not picking up the changes.
Fix: changed the extension from .txt to .properties
Issue/Mistake#2 >> The .properties file name was different from what is mentioned.
The file should be named:
servicename.properties
matching the servicename defined in the *application.properties* as below:
spring.application.name=<<servicename>>
In my example:
spring.application.name=limits-service
Hence the file should be named **limits-service.properties**,
but I created with the name limits-service-service.properties
enter code here
Fix: renamed the file to: limits-service.properties
Now I am able to see the configurations from the spring-cloud-config and it is picking the values from the configured GIT repository.
To those who may have a 'delayed(>2mins) but correct response' when visiting: localhost:8888/limits-service/default, One simple way that works for me is:
In limits-service.properties, change the configserver port to 8887 instead of sticking with 8888:
spring.config.import=optional:configserver:http://localhost:8887
It appears that there is a timeout when Spring is trying to visit port 8888.
In application.properties of spring-cloud-config-server, change spring.cloud.config.server.git.uri directly to a REAL git that is stored on Github. I am using my own public git so that you guys can copy-paste it:
spring.cloud.config.server.git.uri=https://github.com/SimonSongCA/git-localconfig-repo.git
It significantly reduces the delay from X minutes to seconds, and there are no exceptions at the console anymore.
The below solution worked for me.
Add below property in properties file.
spring.cloud.config.server.git.default-label=master
Also check to see if you are pointing to right git location
spring.cloud.config.server.git.uri=file:///C://Work//my_project//app-props//repo//test
The above error comes when you refer to old tutorial because GitHub switched to the default branch as "main" then your application tries to access the master branch of your config-server repository but it is not available so above error comes and when you access though browser it will display white label error page.
solution already mentioned in above comments
I am using spring cloud config server to host a centralized location for all the property files configurations to be used in the project.
I tried using the config files from a local file system using below and it works fine:
spring.profiles.active=native
spring.cloud.config.server.native.searchLocations=file://${HOME}/Documents/test-config/cloud-config-properties/
I also used the git repo using: spring.cloud.config.server.git.uri=ssh://xxxxxx.com:7999/test/cloud-config-properties.git
I would like to try using a combination of this in my project.
Example - for dev/test profile - i would like to use from local filesystem and for the production - I would like to use Git repository.
I enabled both the git uri and native profiles in my application.properties in config server application. But the properties are always picked up from the local file system. Is this possible?
Not supported out of the box, however there is a workaround for this. You can define the basedir for the configuration server, which is where it saves the files it fetches from the remote server, by setting the property (in the config server):
spring.cloud.config.server.git.basedir=<your_dir>
If you are working with docker, you can map this directory to the host filesystem.
Now whatever file you put in there will be picked up by configuration-server if it matches any of the application/profile in the request. For example you could put a file there called application-dynamic.properties, and have all your clients use dynamic as the last profile, for example
spring.profiles.active=systesting,dynamic
This way everything you will put in application-dynamic.properties will override whatever is defined in your config repo.
One thing to notice though is that you need to add the file only after configuartion server starts, because it deletes this folder during startup.
Needles to say, it's not a good practice doing this in production (for example a restart will cause the file to be deleted), but for test/dev this is the best option.