I am using Spring Cloud Config Server first time and have a basic query.
Spring Config server externalises the configuration to a separate git repository.
Why would I create a separate repository just for the configurations?
Is not it advisable to have mono repository with all application code and configurations in a single repo than creating a separate one just for configurations.
We have multiple micro services all present in the same repository. Should not the config server to be one of the micro service present in the same repository where the other application code is?
So, in my multi-module gradle project, I can make config-server as one of the module and give the same repository name as git backed url in config-server. Is this advisable? If yes, where should I keep the configurations in config-server? Inside resources?
Thank you.
When working with microservices it is advisible to have one repository for each microservice. The config server is a microservice as well, therefore it should be put in a separate repository.
Each microservice should have its own independent code repository and your application configuration should never be in the same repository as your source code.
You can read more about this here: Heroku's The Twelve-Factor App. Here you can find 12 best practices to use when building microservices, but for this question I recommend looking at
1st factor: The codebase
3rd factor: The config
Related
I have some properties in my microservice. They are changed a lot so I want to store them somewhere out of microservice so that I don't have to undeploy it again and again. One solution is to store them in database but it will be less efficient that way. Can you advise me a solution where I can store them?
Basically this microservice is used by a vast number of people. I want that file to be read ONCE when microservice is deployed (unless or until there is some change in the file) to minimize the calls.
You should use Spring Cloud Config which enables to use a centralized server which exposes a Git Repo which you can use to store environment specific configuration files (application.properties)
You should go with Centralized configuration, spring cloud supports config server, It is highly recommended for microservice architecture.
For reference: https://spring.io/guides/gs/centralized-configuration/
Basically, you have a centralized place to store all the configurations and don't need the redeploy the application every time you change the configs.
NOTE: in case you change the db properties then probably you need the restart the service
Is there any way by chance to make spring config server to clone only specific path instead of whole git repo? When I'm running it on production I may not want my entire code base repository to be cloned to some location as it is always a risk. I've done my research and couldn't figure out ways to clone only specific paths of git repo.
One solution is to host the configurations in a seperate repo. But that would defeat the purpose of one code many deploys suggestion of 12factor.net.
Another possibility is to copy the properties into config server's classpath and then to use native profile to load them. But this would defeat the purpose of Spring ConfigServer.
Also kindly do clarify What would be the best way to run spring config server on a production?
You should be separating your code base from your configuration settings.
Repo 1: Your App Code
Repo 2: Your Spring Cloud Config Server Code
Repo 3: Your configuration settings
When you're deploying, you're deploying the code. Not the configuration. A configuration change should not require a new deployment. A new deployment might require an updated configuration change, but the two are generally decoupled.
Configuration can be dynamically updated while code is running. This is completely separate from updating code and deploying the app. That's the whole point of externalized configuration.
TL;DR: You're looking at "one code many deploys" wrong. Configuration is externalized.
I took this straight from 12factor.net
Apps sometimes store config as constants in the code. This is a violation of twelve-factor, which requires strict separation of config from code. Config varies substantially across deploys, code does not.
Even though you are no longer storing them as constants but still in the same repository is not enough separation from the code. Use a separate repository for configurations.
I have a spring boot project with 4 microservices (Eureka service registry, Config server, a Zuul gateway and a userservice) in one repository with a parent project where I have a docker-compose.yml which reads the Dockerfiles in the microservices project and uses the "application-docker.yml" and "bootstrap-docker.yml"
What I'd like to do is to trigger a jenkins pipeline after a commit in git so that it will compile and deploy the microservices in Docker. Eventually I'd like to have a production configuration that deploys the images in Kubernetes maybe AWS.
Now, in order to work, the microservices need to start in order:
configserver
eureka service registry
gateway , etc..
What is the best practise?
If I have separate repositories per microservice, I think I can figure it out. It should be easy to deploy a single microservice assuming that configserver and eureka service registry are already up and running, in reality they should never change.
If I have a single repository, and I keep developing new microservices, do I need to have separate jenkins file per microservices or can I have a jenkinsfile in the parent project and use docker-compose?
How does it work? Any articles online that can help (couldn't find any). Does it make sense?
Or do I need to look at Jenkins X ?
Thanks!
I would recommend using separate repositories for each microservice. You use microservices to prevent monoliths and have small well-defined services; it only seems appropriate to also separate them by space i.e. store them in separate repositories (making it for example easier to reuse one).
You would then have to provide a Jenkinsfile in each repo. These would be mostly identical.
If you want fast release cycles you could automatically deploy a single service upon release.
Alternatively you could use an additional release train module that handles the full deployment.
In both cases I would use a docker-compose file that handles the interconnection between the services.
You can enforce the right order by using 'depends_on, links, volumes_from, and network_mode: "service:..."'. For a full reference see the docker documentation.
If you want to keep your single repository your Jenkinsfile(s) would have to be quite hacky, I suppose... After each commit you would either
build all modules --> monolithic behaviour
somehow determine which modules have changed (e.g. looking at the git log) --> same behaviour as with multiple modules but very hackily
The Docker-Compose File
If you want to release all modules at a specific point of time you could use a Release Train module where the docker-compose.yml resides next to a Jenkinsfile. Then when you want to ship your application you can start this Jenkins-job.
If you want to ship each service as soon as it is released, independently from the others, you would need to access the docker-compose.yml from each module. You could do this manually (since the files won't change too often) or create a docker module that you use as a git-submodule in all your services.
We use a generic docker-compose.yml for this, where every version is replaced by a variable:
example-service:
image: example.service:${EXAMPLE_SERVICE_VERSION}
Then to start that specific service in jenkins we use the command
export EXAMPLE_SERVICE_VERSION=1.1.1
docker-compose -p example-project -f docker-compose.yml up -d example-service
The role of the Config Server while using Spring Cloud Config seems to be pretty dumb and hosting it seems to be an unnecessary overhead. Is there a way to get the Clients to directly read configs from the git repo?
You can let Spring Boot look at specific locations for the config files: https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html#boot-features-external-config-application-property-files
If you set this location to your cloned git repo (checked out on the branch you need) and solve the update from the origin repository, this might work.
It is achievable via the embedding config server approach
Do refer to Embedding The Config Server section
If you want to read the configuration for an application directly from the backend repository (instead of from the config server), you basically want an embedded config server with no endpoints. You can switch off the endpoints entirely by not using the #EnableConfigServer annotation (set spring.cloud.config.server.bootstrap=true).
I am trying to convert a normal monolithic web application into microservices structure using Spring Boot and Spring Cloud. I am actually trying to create Angular 2 front-end application and calls these my developed microservices in the cloud. And I am already started to break the modules into independent process's structure for microservice architecture.
Here my doubt is that, when designing the flow of control and microservice structure architecture, can I use only one single Spring Boot project using different controller for this entire web application back end process?
Somewhere I found that when I am reading develop all microservices using 2 different Spring Boot project. I am new to Spring and Spring Cloud. Is it possible to create all services in single project by using different modules?
Actually, it doesn't matter to package all those services into ONE project. But in micro-service's opinion, you should separate them into many independent projects. There are several questions you can ask yourself before transforming original architecture.
Is your application critical? Can user be tolerant of downtime while you must re-deploying whole package for updating only one service?
If there is no any dependency between services, why you want to put them together? Isn't it hard to develop or maintain?
Is the usage rate of each service the same? Maybe you can isolate those services and deploy them which are often to be invoked to a strong server.
Try to read this article Adopting Microservices at Netflix: Lessons for Architectural Design to understand the best practices for designing a microservices architecture. And for developing with Spring Cloud, you can also read this post Spring Cloud Netflix to know which components you should use in your architecture.
Currently I am working on microservices too, according my experience we have designed microservices as step below,
Maven
You should create the project with different project. But actually you can separate your project to submodule. So you will be easy to manage your project, the submodule you can use with other project too.
Build the Jar Library put your local repository. it can save your time, you have just find the same component or your functionality then build the jar file put in your local repository , so every project that use this function call point to download this repository, you don't have to write many project same same.
So finally I would like you to create different springboot project, but just create submodule and build local repository.
By creating your modules in different projects you create a more flexible solution.
You could even use different languages and technologies in a service in particular. E.g. one of your services could be NodeJS and the rest Java/Spring.