I have web application which contain multiple property files like test1.prop, test2.prop... and so on.
I am writing Ansible playbook to deploy my code but I was stuck when it comes to property file changes.
Right now the support team changes it manually by referring the release notes.
What i want to achieve is this -
When the release comes, it needs to be deployed to different environments like dev, preprod, prod etc.
The values in the property file could be different depends on the environment like env.baseurl= {{env_baseurl}}. I was thinking of using Jinja templates for this.
I need to loop through the currently deployed property files, check if they are same or if any file any difference with the current release and change it accordingly.
How do i make a script or playbook which is generic and can be used to deploy on each environment.
Please let me know if someone can help.
Related
We have build a few Microservices (MS) which have been deployed to our company's K8s clusters.
For current deployment, any one of our MSs will be built as a Docker image and they deployed manually using the following steps; and it works fine:
Create Configmap
Installing a Service.yaml
Installing a Deployment.yaml
Installing an Ingress.yaml
I'm now looking at Helm v3 to simplify and encapsulate these deployments. I've read a lot of the Helm v3 documentation, but I still haven't found the answer to some simple questions and I hope to get an answer here before absorbing the entire doc along with Go and SPRIG and then finding out it won't fit our needs.
Our Spring MS has 5 separate application.properties files that are specific to each of our 5 environments. These properties files are simple multi-line key=value format with some comments preceded by #.
# environment based values
key1=value1
key2=value2
Using helm create, I installed a chart called ./deploy in the root directory which auto-created ./templates and a values.yaml.
The problem is that I need to access the application.properties files outside of the Chart's ./deploy directory.
From helm, I'd like to reference these 2 files from within my configmap.yaml's Data: section.
./src/main/resource/dev/application.properties
./src/main/resources/logback.xml
And I want to keep these files in their current format, not rewrite them to JSON/YAML format.
Does Helm v3 allow this?
Putting this as answer as there's no enough space on the comments!
Check the 12 factor app link I shared above, in particular the section on configuration... The explanation there is not great but the idea is behind is to build one container and deploy that container in any environment without having to modify it plus to have the ability to change the configuration without the need to create a new release (the latter cannot be done if the config is baked in the container). This allows, for example, to change a DB connection pool size without a release (or any other config parameter). It's also good from a security point of view as you might not want the container running in your lower environments (dev/test/whatnot) having production configuration (passwords, api keys, etc). This approach is similar to the Continuous Delivery principle of build once, deploy anywhere.
I assume that when you run the app locally, you only need access to one set of configuration, so you can keep that in a separate file (e.g. application.dev.properties), and have the parameters that change between environments in helm environment variables. I know you mentioned you don't want to do this, but this is considered a good practice nowadays (might be considered otherwise in the future...).
I also think it's important to be pragmatic, if in your case you don't feel the need to have the configuration outside of the container, then don't do it, and probably using the suggestion I gave to change a command line parameter to pick the config file works well. At the same time, keep in mind the 12 factor-app approach in case you find out you do need it in the future.
I have a dockerised spring-boot based application and i wanted to update some of the values in application.properties. And this seems can be achieved in 3 ways.
Update the application.properties file, rebuild the image.
Add --spring.config.location= to the ENTRYPOINT, update the prop file, rebuild the image.
Use volume mount, mention the prop file location, update the prop, rebuild the image.
Using Spring Profiles and pass the profile info before running the container. And even in this approach update the profile specific prop file, rebuild the image.
As we can see all the approaches involve rebuilding images. Is there a way to make changes to application.properties without rebuilding the image? What is the preferred approach in prod scenarios?
Thanks!
Yes, there is one more way to inject properties using environment variables.
As per spring property resolution order, environmental variables take precedence over property files.
For example, say you want to update the spring.datasource.username, you can set it via environment variable as SPRING_DATASOURCE_USERNAME. Basically, replacing . with _. By nature, environment variables are not case sensitive but by convention, we put them in upper case.
This variable can be passed when you will be creating container from your image as suggested in this answer.
I would recommend using environment variables, as #YogeshBadke suggests in their answer.
Your option of using a volume is also a good option. Host files mentioned using docker run -v replace files in the image when the container is started, and this does not require an image rebuild. For example
docker run -v $PWD/application.properties.prod:/app/application.properties ...
As a general rule you should not need to rebuild your image to run it in a different environment, and you should avoid mentioning environment-specific hostnames or similar settings anywhere in what gets built into your image. I would not recommend having separate "dev" vs. "prod" profiles in a Docker-hosted solution, since you'll have to rebuild the image as soon as you add a "qa" environment or anything else happens; it's better to be able to just change the deploy-time settings.
(If you happen to be deploying this in Kubernetes, my experience has been that setting individual values via environment variables is easiest, injecting an entire properties file via a ConfigMap works, and trying to embed the properties in the image simply doesn't.)
I'm using msdeploy on the command-line to deploy a windows service using the sync verb against dirPath source and destination providers. So far so good.
My project's build assets have per-environment config files, and during deploy to a particular environment I want to rename the targeted environment's config file to the default config file name using a replace rule, but no matter how I engineer my replace rule nothing seems to get me the results I want. I've tried with a number of different configurations of the rule with no luck but the one that seems the most logical is this one:
-replace:objectName=filePath,scopeAttributeName=path,scopeAttributeValue=Service\.Dev\.exe\.config,match=Service\.Dev\.exe\.config,replace=Service.exe.config
If I deploy without the replace rule both Service.Dev.exe.config and Service.exe.config are deployed with the expected respective content. Once I add the replace rule above I get just Service.exe.config and it's got the content of that file from the source, not the content of Service.Dev.exe.config as I am hoping for. If I swap the specifications of the match and replace components (and change regexp <-> plain text accordingly) to arrive at:
-replace:objectName=filePath,scopeAttributeName=path,scopeAttributeValue=Service\.Dev\.exe\.config,match=Service\.exe\.config,replace=Service.Dev.exe.config
I just get Service.Dev.exe.config with the content of Service.exe.config and no Service.exe.config in the destination.
I'm sure I'm missing something obvious, but what is it?
This doesn't directly answer your question but I think its a better solution to your root problem.
The real issue here is config management for different environments. Many developers create different configs for each environment like you do. This approach requires a lot of duplication across your different config files which can easily get out of sync.
The best approach I've found for managing configuration across different environments is WebDeploy Parameterization. It is similar to config transforms in that you have a single base config file that is tweaked, so there is far less duplication. Parameterization however is preferred because it applies the changes at deploy time instead of build time. We use this for 50+ applications with great success.
Here is a blog post with more details - http://www.dotnetcatch.com/2014/09/08/parameterizationpreview-visual-studio-extension/
The match and replace should swap
match=Service.Dev.exe.config,replace=Service\.exe\.config
I have recently started using ansible for configuration management of linux servers.
My habbit is that if I learn one tool then I try to use it as much as possible.
Initially for my php web apps I had a long Makefile which used to download, install packages , make php.ini file chnages , extract zip files , copy files between folders etc to deploy my application in as automated way.
Now, I am thinking of converting that Makefile deployment to Ansible because then I can arrange the separate yml file for separate areas rather than one big makefile for the whole project.
I want to know that is it good idea to use ansible for that or Makefile will be good for that.
Sure, Ansible is great for that. You can separate all your different steps into different playbooks that are identified by yaml files.
You can define common tasks and then include them in your specific playbooks.
You can also make use of Ansible roles to create complete set of playbooks depending on the role of the server. For example, one set servers' role could be webservers and another set of servers' role could be databases.
You can find more info on roles here: http://docs.ansible.com/playbooks_roles.html
There are's also a few modules on the web out there that you can also use to get you started and you can also use Ansible Galaxy to import roles.
Of course, you can accomplish the same by breaking down your Makefile but maybe you want to learn a new tool.
Hope it helps.
We're testing Octopus Deploy 2.0 (OD) to deploy web services, windows services and citrix applications.
QUICK QUESTION:
When using config transformation, can parameters be used to indicate which config file should be used for the transformations?
MORE DETAIL:
When setting up for config transformations, we would like to have files named
MyApp.DEV_US.config
MyApp.DEV_CANADA.config
MyApp.DEV_AUSTRALIA.config
and so on for TEST, STAGE and PRODUCTION
Our deployments to DEV, for example, always include deployments to all regions. So we would prefer if OD environments were DEV, TEST, STAGE and PRODUCTION. Then in each deployment, we have multiple steps that deploy to each region.
However, OD config transformations only look for OD Environments when looking for which config files to use as part of the transformation. It seems OD would require us to bring each region up to the environment level, which from our POV is not ideal and would clutter the dashboard.
Can we pass parameters into the config transformation process such that we can indicate which file to use for the transform?
I believe you can achieve what you are after with the following, but it will require multiple steps in the process.
Create a step called Deploy to Dev - US and a step called Deploy to Dev - Canada
Now define a variable called CountrySpecificConfigFiles and you can scope it to the required step (and environment etc)
In the Configuration transformations section for each Steps, choose the variable defined in the step above
You could abstract this further by naming your steps DEV_US and DEV_CANADA and define just the one variable value as Web.#{Octopus.Task.Name}.config without any scope to steps, or by removing the variable and doing it inline in the Additional Transforms field.