I have a consul server running on https://dev.example.com/ on which I store configurations as below:
consul key name = dev
consul key value = some yaml data
kubernetes configMap value some yaml data --> ( this is in sync with consul data )
what I want to do is as soon as I change something in some yaml data, the values should be automatically reflected in configMap and pod should be reloaded automatically to capture new values.
I have tried envconsul but it only picks keys with one value.
I tried consul-template but did not find any good working example.
Take a look at https://github.com/Trendyol/trendyol-consul-template-injector. It allows you to inject consul-template as a sidecar to your pod, and will generate files from templates that are provided in a ConfigMap. Your application will need to monitor this generated file, and reload itself when the file changes.
See https://medium.com/trendyol-tech/configuration-and-secret-management-with-consul-template-on-kubernetes-dcc4c6695142 for a more detailed blog post explaining why this was built and how it works.
Related
I have a gradle based Spring Boot 3 application. I use the bootBuildImage gradle task in circleci to build a docker image of this application.
Now, I want to add NewRelic to this application. I know I can do it by writing my own Dockerfile but I want to do it by configuring the bootBuildImage gradle task.
I saw that I can add buildPacks like this:
tasks.named("bootBuildImage") {
buildpacks = [...]
}
And it appears that NewRelic has a buildpack here.
How can I generate the docker image with NewRelic integration?
Bonus: I need to inject environment variable as NEW_RELIC_ENABLE_AGENT=true|false. How can I do it?
You're on the right track. You want to use the New Relic Buildpack that you found.
High-level instructions for that buildpack can be found here. It essentially works by taking in bindings (the secret config data) and the buildpack securely maps those values to the standard New Relic agent configuration properties (through env variables).
An example of an APM tool configured through bindings can be found here. The specific example is using a different APM tool, but the same steps will work with any APM tool configured through bindings, like New Relic.
For your app:
Create a bindings directory. The root of your project is a reasonable place, but the path doesn't ultimately matter. Don't check in binding files that contain secret data :)
In the folder, create a subfolder called new-relic. Again, the name doesn't really matter.
In the folder from the previous step, create a file called type. The name does matter. In that file, write NewRelic and that's it. Save the file. This is how the buildpack identifies the bindings.
In the same folder, you can now add additional files to configure New Relic. The name of the file is the key and the contents of the file are the value. When your app runs, the buildpack will read the bindings and translate these to New Relic configuration settings in the form NEW_RELIC_<KEY>=<VALUE>. Thus if you read the New Relic docs and see a property called foo, you could make a file called foo set the value to bar and at runtime, you'll end up with an env variable NEW_RELIC_foo=bar being set. The New Relic agent reads environment variables for it's configuration, although sometimes it's not the first way that's mentioned in their docs.
Next you need to configure your build.gradle file. These changes will tell bootBuildImage to add the New Relic buildpack and to pass through your bindings.
In the tasks.named("bootBuildImage") block, add buildpacks = ["urn:cnb:builder:paketo-buildpacks/java", "gcr.io/paketo-buildpacks/new-relic"]. This will run the standard Java buildpack and then append New Relic onto the end of that list. Example.
Add a bindings list. In the same tasks.named("bootBuildImage") block add bindings = ["path/to/local/bindings/new-relic:/platform/bindings/new-relic"]. This will mount path/to/local/bindings/new-relic on your host to /platform/bindings/new-relic in the container, which is where the buildpack expects bindings to live. You will need to change the first path to point to the local bindings you created above (you can probably use a Gradle variable to the project to reference them, but I don't know if off the top of my head). Don't change the path on the container side, that needs to be exactly what I put above.
Run your build. ./gradlew bootBuildImage. In the output, you should see the New Relic buildpack pass detection (it passes if it finds the type file with NewRelic as the contents) and it should also run and contribute the New Relic agent as is described in the buildpack README.md.
After a successful build, you'll have the image. The key to remember is that bindings are not added to the image. This is intentional for security reasons. You don't want secret binding info to be included in the image, as that will leak your secrets.
This means that you must also pass the bindings through to your container runtime when you run the image. If you're using Docker, you can docker run --volume path/to/local/bindings/new-relic:/platform/bindings/new-relic ... and use the same paths as build time. If you're deploying to Kubernets, you'll need to set up Secrets in K8s and mount those secrets as files within the container under the same path as before /platform/bindings/new-relic. So you need to make a type file, /platform/bindings/new-relic/type, and files for each key/value parameter you want to set.
At some point in the future, we're working to have all of the APM buildpacks included in the main Java buildpack by default. This would eliminate the first config change in step #5.
Because managing bindings can be kind of a pain, I also have a project called binding-tool that can help with steps 1-3. It allows you to easily create the binding files, like bt add -t NewRelic -p key1=val1 -p key2=val2. It's not doing anything magic, just creates the files for you, but I find it handy. In the future, I want it to generate the Kubernetes YAML as well.
We have build a few Microservices (MS) which have been deployed to our company's K8s clusters.
For current deployment, any one of our MSs will be built as a Docker image and they deployed manually using the following steps; and it works fine:
Create Configmap
Installing a Service.yaml
Installing a Deployment.yaml
Installing an Ingress.yaml
I'm now looking at Helm v3 to simplify and encapsulate these deployments. I've read a lot of the Helm v3 documentation, but I still haven't found the answer to some simple questions and I hope to get an answer here before absorbing the entire doc along with Go and SPRIG and then finding out it won't fit our needs.
Our Spring MS has 5 separate application.properties files that are specific to each of our 5 environments. These properties files are simple multi-line key=value format with some comments preceded by #.
# environment based values
key1=value1
key2=value2
Using helm create, I installed a chart called ./deploy in the root directory which auto-created ./templates and a values.yaml.
The problem is that I need to access the application.properties files outside of the Chart's ./deploy directory.
From helm, I'd like to reference these 2 files from within my configmap.yaml's Data: section.
./src/main/resource/dev/application.properties
./src/main/resources/logback.xml
And I want to keep these files in their current format, not rewrite them to JSON/YAML format.
Does Helm v3 allow this?
Putting this as answer as there's no enough space on the comments!
Check the 12 factor app link I shared above, in particular the section on configuration... The explanation there is not great but the idea is behind is to build one container and deploy that container in any environment without having to modify it plus to have the ability to change the configuration without the need to create a new release (the latter cannot be done if the config is baked in the container). This allows, for example, to change a DB connection pool size without a release (or any other config parameter). It's also good from a security point of view as you might not want the container running in your lower environments (dev/test/whatnot) having production configuration (passwords, api keys, etc). This approach is similar to the Continuous Delivery principle of build once, deploy anywhere.
I assume that when you run the app locally, you only need access to one set of configuration, so you can keep that in a separate file (e.g. application.dev.properties), and have the parameters that change between environments in helm environment variables. I know you mentioned you don't want to do this, but this is considered a good practice nowadays (might be considered otherwise in the future...).
I also think it's important to be pragmatic, if in your case you don't feel the need to have the configuration outside of the container, then don't do it, and probably using the suggestion I gave to change a command line parameter to pick the config file works well. At the same time, keep in mind the 12 factor-app approach in case you find out you do need it in the future.
I am looking for tools/software for dynamically changing Linux conf/YAML files via API or tool like Consul.
If you have any experience on consul, please give feedback about creating templates for conf/YAML files, and without using service can it be done via consul?
Consul Template or Gomplate can be used to template configuration files based on changes in a backend data source.
https://learn.hashicorp.com/tutorials/consul/consul-template provides a basic example of a template which regenerates file contents when keys are added to Consul's key-value store.
I'm working on composing yaml file for installing Elasticsearch via the ECK operator (CRD).
Its documented here - https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-elasticsearch-specification.html
As one goes through the sub-links, you will discover there are various pieces through which we can define the configuration of Elasticsearch.
The description of the CRD is contained here, but this is not easy to read -
https://download.elastic.co/downloads/eck/1.0.0-beta1/all-in-one.yaml
I'm curious how do Kubernetes developers understand various constructs of yaml file for a given kubernetes resource ?
Docs
In general, if it's a standard resource, the best way is to consult the official documentation for all the fields of the resource.
You can do this with kubectl explain. For example:
kubectl explain deploy
kubectl explain deploy.spec
kubectl explain deploy.spec.template
You can find the same information in the web-based API reference.
Boilerplate
For some resources, you can use kubectl create to generate the YAML for a basic version of the resource, which you can then use as a starting point for your own customisations.
For example:
kubectl create deployment --image=nginx mydep -o yaml --dry-run >mydep.yaml
This generates the YAML for a Deployment resource and saves it in a local file. You can then customise the resource from there.
The --dry-run option causes the generated resource definition to not be submitted to the API server (so it won't be created) and the -o yaml option outputs the generated definition of the resource in YAML format.
Other common resources for which this is possible include:
(Cluster)Role
(Cluster)RoleBinding
ConfigMap
Deployment
Job
PodDisruptionBudget
Secret
Service
See kubectl create -h.
Another approach to write yml files for Kubernetes resources could be to use kubernetes plugin which should be available for most of the IDEs.
For example, For Intellij, following is link for Kubernetes plugin. This link also explains which all features this plugin provides:
https://www.jetbrains.com/help/idea/kubernetes.html
I started a pod in kubernetes cluster which can call kubernetes api via go-sdk (like in this example: https://github.com/kubernetes/client-go/tree/master/examples/in-cluster-client-configuration). I want to listen some external events in this pod (e.g. GitHub web-hooks), fetch yaml configuration files from repository and apply them to this cluster.
Is it possible to call kubectl apply -f <config-file> via kubernetes API (or better via golang SDK)?
As yaml directly: no, not that I'm aware of. But if you increase the kubectl verbosity (--v=100 or such), you'll see that the first thing kubectl does to your yaml file is convert it to json, and then POST that content to the API. So the spirit of the answer to your question is "yes."
This box/kube-applier project may interest you. While it does not appear to be webhook aware, I am sure they would welcome a PR teaching it to do that. Using their existing project also means you benefit from all the bugs they have already squashed, as well as their nifty prometheus metrics integration.