I have an octopus deployment that needs to go to a load balanced environment But there are small changes in the config between the two servers.
So, in summary:
It deploys to the same environment (PreProd)
It gets deployed to two different servers linked to that environment
There are small changes between the two web.config files between the two servers.
I already have a web.preprod.config that gets transformed into web.config. Does it mean I need to create more config files, ie. web.server1.preprod.config and web.server2.preprod.config or is there another cleaner way of doing it? It is a whole section that is different so not just an appSetting.
A solution that has worked well in similar scenarios for me in the past (with OctopusDeploy specifically), is to use the web.{environment}.config transforms to get the correct config structure in place, but to use variable substitution and define placeholders in the transform file to keep the run-time environment-specific definitions in Octopus. Quite how you break down the substitution syntax is really dependent on your config, but you can use the machine-scoping features of Octopus variables to control the actual values injected.
This scenario is a good example of where web.config transforms start to blur the edges of configuration management; environment-specific config is really the domain of Octopus (or, more specifically, a centralised configuration store), but the solution proposed here is taking it out of Octopus and back into the source repository, which is one of the problems Octopus is actually designed to solve.
For example; what if you introduced a 3rd node in your pre-prod load balancer? This demands a code change, build, version bump and package, which can be completely avoided given the above.
The general approach to problems like this is, indeed, to create a web.server*.preprod.config, or local.config. I'd suggest looking at what exactly is different in the config, and why. Try to find things that you can merge. For instance:
If one difference is the difference in drive letter, and your config contains these entries:
C:/a/b/c.txt
C:/a/b/d.txt
try splitting those entries into
drive=C
drive:/a/b/c.txt
In that case you only have to change drive=C to drive=D to make two entries work.
Related
We have build a few Microservices (MS) which have been deployed to our company's K8s clusters.
For current deployment, any one of our MSs will be built as a Docker image and they deployed manually using the following steps; and it works fine:
Create Configmap
Installing a Service.yaml
Installing a Deployment.yaml
Installing an Ingress.yaml
I'm now looking at Helm v3 to simplify and encapsulate these deployments. I've read a lot of the Helm v3 documentation, but I still haven't found the answer to some simple questions and I hope to get an answer here before absorbing the entire doc along with Go and SPRIG and then finding out it won't fit our needs.
Our Spring MS has 5 separate application.properties files that are specific to each of our 5 environments. These properties files are simple multi-line key=value format with some comments preceded by #.
# environment based values
key1=value1
key2=value2
Using helm create, I installed a chart called ./deploy in the root directory which auto-created ./templates and a values.yaml.
The problem is that I need to access the application.properties files outside of the Chart's ./deploy directory.
From helm, I'd like to reference these 2 files from within my configmap.yaml's Data: section.
./src/main/resource/dev/application.properties
./src/main/resources/logback.xml
And I want to keep these files in their current format, not rewrite them to JSON/YAML format.
Does Helm v3 allow this?
Putting this as answer as there's no enough space on the comments!
Check the 12 factor app link I shared above, in particular the section on configuration... The explanation there is not great but the idea is behind is to build one container and deploy that container in any environment without having to modify it plus to have the ability to change the configuration without the need to create a new release (the latter cannot be done if the config is baked in the container). This allows, for example, to change a DB connection pool size without a release (or any other config parameter). It's also good from a security point of view as you might not want the container running in your lower environments (dev/test/whatnot) having production configuration (passwords, api keys, etc). This approach is similar to the Continuous Delivery principle of build once, deploy anywhere.
I assume that when you run the app locally, you only need access to one set of configuration, so you can keep that in a separate file (e.g. application.dev.properties), and have the parameters that change between environments in helm environment variables. I know you mentioned you don't want to do this, but this is considered a good practice nowadays (might be considered otherwise in the future...).
I also think it's important to be pragmatic, if in your case you don't feel the need to have the configuration outside of the container, then don't do it, and probably using the suggestion I gave to change a command line parameter to pick the config file works well. At the same time, keep in mind the 12 factor-app approach in case you find out you do need it in the future.
I started to use spring cloud config and I have a branch for each client. There're configuration properties which are identical between the branches, and I would like not to duplicate them, but storing them in a single file.
Let's assume I have a branch named "my-branch" and it contains this file:
service.properties:
foo=123
In my master branch I have this file:
service.properties:
foo=456
bar=789
I want to query the label "my-branch" and to get this properties:
foo=123
bar=789
How can I achieve that?
From what I investigated - it's not possible when you are using one repository.
You can achieve it by having composite environment and have common properties in another repo, but there is another limitation
When using a composite environment, it is important that all repositories contain the same labels. If you have an environment similar to those in the preceding examples and you request configuration data with the master label but the Subversion repository does not contain a branch called master, the entire request fails.
See: https://cloud.spring.io/spring-cloud-static/spring-cloud-config/2.0.2.RELEASE/single/spring-cloud-config.html#composite-environment-repositories
In your case it's even more complicated because you are using branch per client.
So you will need to create a separate repo for common configuration with lot of branches for each client containing the same configuration.
In case you use one branch - it's easier and works fine, see this answer.
I'm using msdeploy on the command-line to deploy a windows service using the sync verb against dirPath source and destination providers. So far so good.
My project's build assets have per-environment config files, and during deploy to a particular environment I want to rename the targeted environment's config file to the default config file name using a replace rule, but no matter how I engineer my replace rule nothing seems to get me the results I want. I've tried with a number of different configurations of the rule with no luck but the one that seems the most logical is this one:
-replace:objectName=filePath,scopeAttributeName=path,scopeAttributeValue=Service\.Dev\.exe\.config,match=Service\.Dev\.exe\.config,replace=Service.exe.config
If I deploy without the replace rule both Service.Dev.exe.config and Service.exe.config are deployed with the expected respective content. Once I add the replace rule above I get just Service.exe.config and it's got the content of that file from the source, not the content of Service.Dev.exe.config as I am hoping for. If I swap the specifications of the match and replace components (and change regexp <-> plain text accordingly) to arrive at:
-replace:objectName=filePath,scopeAttributeName=path,scopeAttributeValue=Service\.Dev\.exe\.config,match=Service\.exe\.config,replace=Service.Dev.exe.config
I just get Service.Dev.exe.config with the content of Service.exe.config and no Service.exe.config in the destination.
I'm sure I'm missing something obvious, but what is it?
This doesn't directly answer your question but I think its a better solution to your root problem.
The real issue here is config management for different environments. Many developers create different configs for each environment like you do. This approach requires a lot of duplication across your different config files which can easily get out of sync.
The best approach I've found for managing configuration across different environments is WebDeploy Parameterization. It is similar to config transforms in that you have a single base config file that is tweaked, so there is far less duplication. Parameterization however is preferred because it applies the changes at deploy time instead of build time. We use this for 50+ applications with great success.
Here is a blog post with more details - http://www.dotnetcatch.com/2014/09/08/parameterizationpreview-visual-studio-extension/
The match and replace should swap
match=Service.Dev.exe.config,replace=Service\.exe\.config
We're testing Octopus Deploy 2.0 (OD) to deploy web services, windows services and citrix applications.
QUICK QUESTION:
When using config transformation, can parameters be used to indicate which config file should be used for the transformations?
MORE DETAIL:
When setting up for config transformations, we would like to have files named
MyApp.DEV_US.config
MyApp.DEV_CANADA.config
MyApp.DEV_AUSTRALIA.config
and so on for TEST, STAGE and PRODUCTION
Our deployments to DEV, for example, always include deployments to all regions. So we would prefer if OD environments were DEV, TEST, STAGE and PRODUCTION. Then in each deployment, we have multiple steps that deploy to each region.
However, OD config transformations only look for OD Environments when looking for which config files to use as part of the transformation. It seems OD would require us to bring each region up to the environment level, which from our POV is not ideal and would clutter the dashboard.
Can we pass parameters into the config transformation process such that we can indicate which file to use for the transform?
I believe you can achieve what you are after with the following, but it will require multiple steps in the process.
Create a step called Deploy to Dev - US and a step called Deploy to Dev - Canada
Now define a variable called CountrySpecificConfigFiles and you can scope it to the required step (and environment etc)
In the Configuration transformations section for each Steps, choose the variable defined in the step above
You could abstract this further by naming your steps DEV_US and DEV_CANADA and define just the one variable value as Web.#{Octopus.Task.Name}.config without any scope to steps, or by removing the variable and doing it inline in the Additional Transforms field.
I have two questions ;-)
Is the "web.xml" in a WAR-File intended to contain project/customer-specific informations
Example: For customerX we want Filters for Bandwith and Compression for customerY we dont want those Filters
How does the Maven-Build-Process allow us to make different Builds for such project/customer-specific WAR-Files.
Should it be done with maven profiles
Should it be done with separate build-processes that integrate the release-project into individual WAR-Builds
To first one i would recommend to say No. You should solve that by using property files to handle such things.
you can handle that with profiles, but with the draw-back that you need to rebuild for every customer you have. Lets say dev, test, prod. This can be solved by properties but that means to build three times..I have described a better solution here which handles the different locations with a single build. That will solve the problem for release as well.
It is also a solution to use a separate maven project (lets call it configuration module) which depends on the WAR file). But this would mean to three different projects like dev, test and production which i find not very handy.