I started to use spring cloud config and I have a branch for each client. There're configuration properties which are identical between the branches, and I would like not to duplicate them, but storing them in a single file.
Let's assume I have a branch named "my-branch" and it contains this file:
service.properties:
foo=123
In my master branch I have this file:
service.properties:
foo=456
bar=789
I want to query the label "my-branch" and to get this properties:
foo=123
bar=789
How can I achieve that?
From what I investigated - it's not possible when you are using one repository.
You can achieve it by having composite environment and have common properties in another repo, but there is another limitation
When using a composite environment, it is important that all repositories contain the same labels. If you have an environment similar to those in the preceding examples and you request configuration data with the master label but the Subversion repository does not contain a branch called master, the entire request fails.
See: https://cloud.spring.io/spring-cloud-static/spring-cloud-config/2.0.2.RELEASE/single/spring-cloud-config.html#composite-environment-repositories
In your case it's even more complicated because you are using branch per client.
So you will need to create a separate repo for common configuration with lot of branches for each client containing the same configuration.
In case you use one branch - it's easier and works fine, see this answer.
Related
We have build a few Microservices (MS) which have been deployed to our company's K8s clusters.
For current deployment, any one of our MSs will be built as a Docker image and they deployed manually using the following steps; and it works fine:
Create Configmap
Installing a Service.yaml
Installing a Deployment.yaml
Installing an Ingress.yaml
I'm now looking at Helm v3 to simplify and encapsulate these deployments. I've read a lot of the Helm v3 documentation, but I still haven't found the answer to some simple questions and I hope to get an answer here before absorbing the entire doc along with Go and SPRIG and then finding out it won't fit our needs.
Our Spring MS has 5 separate application.properties files that are specific to each of our 5 environments. These properties files are simple multi-line key=value format with some comments preceded by #.
# environment based values
key1=value1
key2=value2
Using helm create, I installed a chart called ./deploy in the root directory which auto-created ./templates and a values.yaml.
The problem is that I need to access the application.properties files outside of the Chart's ./deploy directory.
From helm, I'd like to reference these 2 files from within my configmap.yaml's Data: section.
./src/main/resource/dev/application.properties
./src/main/resources/logback.xml
And I want to keep these files in their current format, not rewrite them to JSON/YAML format.
Does Helm v3 allow this?
Putting this as answer as there's no enough space on the comments!
Check the 12 factor app link I shared above, in particular the section on configuration... The explanation there is not great but the idea is behind is to build one container and deploy that container in any environment without having to modify it plus to have the ability to change the configuration without the need to create a new release (the latter cannot be done if the config is baked in the container). This allows, for example, to change a DB connection pool size without a release (or any other config parameter). It's also good from a security point of view as you might not want the container running in your lower environments (dev/test/whatnot) having production configuration (passwords, api keys, etc). This approach is similar to the Continuous Delivery principle of build once, deploy anywhere.
I assume that when you run the app locally, you only need access to one set of configuration, so you can keep that in a separate file (e.g. application.dev.properties), and have the parameters that change between environments in helm environment variables. I know you mentioned you don't want to do this, but this is considered a good practice nowadays (might be considered otherwise in the future...).
I also think it's important to be pragmatic, if in your case you don't feel the need to have the configuration outside of the container, then don't do it, and probably using the suggestion I gave to change a command line parameter to pick the config file works well. At the same time, keep in mind the 12 factor-app approach in case you find out you do need it in the future.
I am trying to setup spring cloud config server. I see that there are two options for storing properties. Either Git or Native file system. Looks like people are more leaned towards using GIT against Native.
Our project follows trunk based development meaning it does not cut the branch on every release to production.
If we use GIT to read properties, and if someone change the property after deployment, then there is likely that changed property will be read which may create problem in production.
Did anyone faced the issue? How did you solve them?
If your Spring Cloud Git Backend is set to a tag or commit id (and not a branch), any modification should not be read.
This repository implementation maps the {label} parameter of the HTTP resource to a git label (commit id, branch name or tag).
If the git branch or tag name contains a slash ("/"), then the label in the HTTP URL should be specified with the special string "(_)" instead (to avoid ambiguity with other URL paths).
Since a commit id or tag is immustable, that should avoid reading a new content.
I have an octopus deployment that needs to go to a load balanced environment But there are small changes in the config between the two servers.
So, in summary:
It deploys to the same environment (PreProd)
It gets deployed to two different servers linked to that environment
There are small changes between the two web.config files between the two servers.
I already have a web.preprod.config that gets transformed into web.config. Does it mean I need to create more config files, ie. web.server1.preprod.config and web.server2.preprod.config or is there another cleaner way of doing it? It is a whole section that is different so not just an appSetting.
A solution that has worked well in similar scenarios for me in the past (with OctopusDeploy specifically), is to use the web.{environment}.config transforms to get the correct config structure in place, but to use variable substitution and define placeholders in the transform file to keep the run-time environment-specific definitions in Octopus. Quite how you break down the substitution syntax is really dependent on your config, but you can use the machine-scoping features of Octopus variables to control the actual values injected.
This scenario is a good example of where web.config transforms start to blur the edges of configuration management; environment-specific config is really the domain of Octopus (or, more specifically, a centralised configuration store), but the solution proposed here is taking it out of Octopus and back into the source repository, which is one of the problems Octopus is actually designed to solve.
For example; what if you introduced a 3rd node in your pre-prod load balancer? This demands a code change, build, version bump and package, which can be completely avoided given the above.
The general approach to problems like this is, indeed, to create a web.server*.preprod.config, or local.config. I'd suggest looking at what exactly is different in the config, and why. Try to find things that you can merge. For instance:
If one difference is the difference in drive letter, and your config contains these entries:
C:/a/b/c.txt
C:/a/b/d.txt
try splitting those entries into
drive=C
drive:/a/b/c.txt
In that case you only have to change drive=C to drive=D to make two entries work.
We have multiple layers of our products split into different build configurations for continuous integration. For the sake of this question, let's just say we have a "Front-End CI" build, and a "API CI" build. The VCS roots are configured to pull in all branches, and triggered to run upon checkin, as should be expected for CI.
Now I have my User Acceptance project, where I use CloudFormation to dynamically spin up servers to which I deploy. I have snapshot dependencies set up for the CI builds mentioned above, and everything works as expected for the default branches on each of the VCS roots and dependencies. I expect that a feature branch for the front-end may not necessarily necessitate a branch from the default for the API, and the current way I have it setup accounts for that as well.
That's where I begin to have issues. If I have to branch both the front-end and API, I cannot get TeamCity to do what I want in this regard. My question is this: how do I tell Team City to run a UA build using branch "A" from the Front-End CI build config and branch "B" from the API CI build config, where "A" and "B" can be any arbitrary branch? Currently right now, all branches from both snapshots are shown when I look at the UA build config. Here's a good picture:
If I run api-branch, it will always use the default branch from the Front-end CI snapshot. Same for any branch on the front-end snapshot. I cannot seem to find a way to specify this in the configuration or when starting a build.
I'm up for just about anything to address this, including build configs that are just cloned off of each other to specify branches the way they're meant to, but I'm just not seeing how I can do that either. Thanks!
Create a teamcity template target which monitors both the front end and API repositories and can trigger on changes. This should be one target (and not 2 different targets). Parameterize the branch names so that actual targets have to give the branch name
I would suggest creating a mapping of the frontend:api branches in a datastore( file,db,nosql) . Then dynamically create teamcity targets (through REST API) for each new/modified combination and explicitly set the branch names. Once the targets are created they will automatically run whenver there are any changes.
I made a Sinatra app, that will be hosted on Heroku, and the source will be up on GitHub. The problem is that i have a file with API keys, that is currently in .gitignore. Is there a way, that I can push my repo to heroku with the key file and exclude the file when pushing to GitHub?
Thanks in advance!
It is possible to maintain a separate branch just for deployment, but it takes much discipline to maintain it properly:
Add a commit to a production branch that adds the config file (git add -f to bybass your excludes).
To update your production branch, merge other branches (e.g. master) into it.
However, you must then never merge your production branch into anything else, or start branches based on any “production commit” (one whose ancestry includes your “add the keys” commit).
An easier path is to adopt Heroku’s custom of using environment variables to communicate your secret values to your instances. See the docs on Configuration and Config Vars:
heroku config:add KEY1=foobar KEY2=frobozz
Then access the values via ENV['KEY1'] and ENV['KEY2'] in your initialization code or wherever you need them. To support your non-Heroku deployments, you could either define the same environment variables or fall back to reading your existing config files if the environment variables do not exist.
The Figaro gem provides a good way to manage this issue. It basically simulates Heroku's environment variable approach locally, and makes it easy to keep your keys in sync between your development environment and Heroku.