I am trying to migrate cassandra DB to AKS using k8ssandra. I would like to retain the source DB's cassandra.yaml configuration in the target as well, like increase the timeout parameters ( to handle client requests). I see the configmap placeholder in the k8ssandra's values.yaml file. Would it help to retain the source configurations? Detailed guideline on how to use it is much appreciated :)
Support for specifying Casssandra configs via a ConfigMap was added in this PR and made available in K8ssandra 1.4.0.
The new chart property name is cassandraYamlConfigMap. Here is what the docs for the property say:
# -- Specifies the name of a ConfigMap that contains a custom cassandra.yaml. The
# ConfigMap should have a key named cassandra.yaml. The values will be merged with the
# base configuration. The following properties should NOT be specified in the ConfigMap:
# * cluster_name
# * seed_provider
# * authenticator
# * authorizer
# * commitlog_directory
# * data_file_directories
# * saved_caches_directory
# * hints_directory
# * endpoint_snitch
Here is an example ConfigMap:
apiVersion: v1
kind: ConfigMap
metadata:
name: cassandra-config
data:
cassandra.yaml: |-
auto_snapshot: false
memtable_flush_writers: 1
commitlog_segment_size_in_mb: 2
concurrent_compactors: 1
compaction_throughput_mb_per_sec: 0
sstable_preemptive_open_interval_in_mb: 0
key_cache_size_in_mb: 0
prepared_statements_cache_size_mb: 1
slow_query_log_timeout_in_ms: 0
counter_cache_size_in_mb: 0
concurrent_reads: 2
concurrent_writes: 2
concurrent_counter_writes: 2
read_request_timeout_in_ms: 90000
range_request_timeout_in_ms: 90000
write_request_timeout_in_ms: 90000
truncate_request_timeout_in_ms: 90000
request_timeout_in_ms: 90000
Here's how this works. The cassandra.yaml, jvm-server-options, etc. are generated as normal when cassandraYamlConfigMap is set. After the server-config-init init container generates the configs, then a new init container called apply-custom-config does a merge using yq of the cassandra.yaml in the ConfigMap and the cassandra.yaml generated by server-config-init. The ConfigMap takes precedence.
Make sure to pay attention to the properties that should not be specified. No validation checks are performed on the ConfigMap.
Note that you will need to use the cassandraYamlConfigMap property if you want to configure internode encryption. See this post for a detailed explanation.
Lastly, I want to point out that there is improved support for configuring cassandra.yaml in K8ssandra Operator. The operator does not allow you to specify a custom ConfigMap but all cassandra.yaml properties are available in the K8ssandraCluster CRD. See here for an example.
Related
We have a kubebuilder controller which is working as expected, now we need to create a webhooks ,
I follow the tutorial
https://book.kubebuilder.io/reference/markers/webhook.html
and now I want to run & debug it locally,
however not sure what to do regard the certificate, is there a simple way to create it , any example will be very helpful.
BTW i've installed cert-manager and apply the following sample yaml but not sure what to do next ...
I need the simplest solution that I be able to run and debug the webhooks locally as Im doing already with the controller (Before using webhooks),
https://book.kubebuilder.io/cronjob-tutorial/running.html
Cert-manager
I've created the following inside my cluster
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: example-com
namespace: test
spec:
# Secret names are always required.
secretName: example-com-tls
# secretTemplate is optional. If set, these annotations and labels will be
# copied to the Secret named example-com-tls. These labels and annotations will
# be re-reconciled if the Certificate's secretTemplate changes. secretTemplate
# is also enforced, so relevant label and annotation changes on the Secret by a
# third party will be overwriten by cert-manager to match the secretTemplate.
secretTemplate:
annotations:
my-secret-annotation-1: "foo"
my-secret-annotation-2: "bar"
labels:
my-secret-label: foo
duration: 2160h # 90d
renewBefore: 360h # 15d
subject:
organizations:
- jetstack
# The use of the common name field has been deprecated since 2000 and is
# discouraged from being used.
commonName: example.com
isCA: false
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
usages:
- server auth
- client auth
# At least one of a DNS Name, URI, or IP address is required.
dnsNames:
- example.com
- www.example.com
uris:
- spiffe://cluster.local/ns/sandbox/sa/example
ipAddresses:
- 192.168.0.5
# Issuer references are always required.
issuerRef:
name: ca-issuer
# We can reference ClusterIssuers by changing the kind here.
# The default value is Issuer (i.e. a locally namespaced Issuer)
kind: Issuer
# This is optional since cert-manager will default to this value however
# if you are using an external issuer, change this to that issuer group.
group: cert-manager.io
Still not sure how to sync it with the kubebuilder to work locally
as when I run the operator in debug mode I got the following error:
setup problem running manager {"error": "open /var/folders/vh/_418c55133sgjrwr7n0d7bl40000gn/T/k8s-webhook-server/serving-certs/tls.crt: no such file or directory"}
What I need is the simplest way to run webhooks locally
Let me walk you through the process from the start.
create webhook like it's said in the cronJob tutorial - kubebuilder create webhook --group batch --version v1 --kind CronJob --defaulting --programmatic-validation . This will create webhooks for implementing defaulting logics and validating logics.
Implement the logics as instructed - Implementing defaulting/validating webhooks
Install cert-manager. I find the easiest way to install is via this commmand - kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.yaml
Edit the config/default/kustomization.yaml file by uncommenting everything that have [WEBHOOK] or [CERTMANAGER] in their comments. Do the same for config/crd/kustomization.yaml file also.
Build Your Image locally using - make docker-build IMG=<some-registry>/<project-name>:tag. Now you dont need to docker-push your image to remote repository. If you are using kind cluster, You can directly load your local image to your specified kind cluster:
kind load docker-image <your-image-name>:tag --name <your-kind-cluster-name>
Now you can deploy it to your cluster by - make deploy IMG=<some-registry>/<project-name>:tag.
You can also run cluster locally using make run command. But, that's a little tricky if you have enabled webooks. I would suggest you running your cluster using KIND cluster in this way. Here, you don't need to worry about injecting certificates. cert-manager will do that for you. You can check out the /config/certmanager folder to figure out how this is functioning.
I'm new to serverless,
So far I was be able to deploy and use .env for the app.
then, under provider in stage property in serverless.yml file, I change it to different stage. I also made new.env.{stage}.
after re-deploy using sls deploy, It still reads the default .env file.
the documentation states:
The framework looks for .env and .env.{stage} files in service directory and then tries to load them using dotenv. If .env.{stage} is found, .env will not be loaded. If stage is not explicitly defined, it defaults to dev.
So, I still don't understand "If stage is not explicitly defined, it defaults to dev". How to explicitly define it?
The dotenv File is choosen based on your stage property configuration. You need to explicitly define the stage property in your serverless.yaml or set it within your deployment command.
This will use the .env.dev file
useDotenv: true
provider:
name: aws
stage: dev # dev [default], stage, prod
memorySize: 3008
timeout: 30
Or you set the stage property via deploy command.
This will use the .env.prod file
sls deploy --stage prod
In your serverless.yml you need to define the stage property inside the provider object.
Example:
provider:
name: aws
[...]
stage: prod
As Feb 2023 I'm going to attempt to give my solution. I'm using the Nx tootling for monorepo (this shouldn't matter but just in case) and I'm using the serverless.ts instead.
I see the purpose of this to be to enhance the developer experience in the sense that it is nice to just nx run users:serve --stage=test (in my case using Nx) or sls offline --stage=test and serverless to be able to load the appropriate variables for that specific environment.
Some people went the route of using several .env.<stage> per environment. I tried to go this route but because I'm not that good of a developer I couldn't make it work. The approach that worked for the was to concatenate variable names inside the serverless.ts. Let me explain...
I'm using just one .env file instead but changing variable names based on the --stage. The magic is happening in the serverless.ts
// .env
STAGE_development=test
DB_NAME_development=mycraftypal
DB_USER_development=postgres
DB_PASSWORD_development=abcde1234
DB_PORT_development=5432
READER_development=localhost // this could be aws rds uri per db instances
WRITER_development=localhost // this could be aws rds uri per db instances
# TEST
STAGE_test=test
DB_NAME_test=mycraftypal
DB_USER_test=postgres
DB_PASSWORD_test=abcde1234
DB_PORT_test=5433
READER_test=localhost // this could be aws rds uri per db instances
WRITER_test=localhost // this could be aws rds uri per db instances
// serverless.base.ts or serverless.ts based on your configuration
...
useDotenv: true, // this property is at the root level
...
provider: {
...
stage: '${opt:stage, "development"}', // get the --stage flag value or default to development
...,
environment: {
STAGE: '${env:STAGE_${self:provider.stage}}}',
DB_NAME: '${env:DB_NAME_${self:provider.stage}}',
DB_USER: '${env:DB_USER_${self:provider.stage}}',
DB_PASSWORD: '${env:DB_PASSWORD_${self:provider.stage}}',
READER: '${env:READER_${self:provider.stage}}',
WRITER: '${env:WRITER_${self:provider.stage}}',
DB_PORT: '${env:DB_PORT_${self:provider.stage}}',
AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1',
}
...
}
When one is utilizing the useDotenv: true, serverless loads your variables from the .env and puts them in the env variable so you can access them env:STAGE.
Now I can access the variable with dynamic stage like so ${env:DB_PORT_${self:provider.stage}}. If you look at the .env file each variable has the ..._<stage> at the end. In this way I can retrieve dynamically each value.
I'm still figuring it out since I don't want to have the word production in my url but still get the values dynamically and since I'm concatenating this value ${env:DB_PORT_${self:provider.stage}}... then the actual variable becomes DB_PORT_ instead of DB_PORT.
I'm trying to wrap my head around the correct steps to deploy [CouchDB](https://couchdb.apache.org] into a Kubernetes cluster.
What I did is:
kubectl create secret --namespace mynamespace generic couch-test-couchdb \
--from-literal=adminUsername=admin
--from-literal=adminPassword=password
--from-literal=cookieAuthSecret=supersecret
helm install --namespace mynamespace couch-test \
--set couchdbConfig.couchdb.uuid=$(uuid | tr -d -) \
-f couch-test.yml \
couchdb/couchdb
With couch-test.yml:
createAdminSecret : false
persistentVolume.enabled : true
persistentVolume.size: 10Gi
The command runs without an error message, however no persistent storage allocation happens. When I type kubectl describe pod couch-test-couchdb-0 I get in Volumes for config-storage, database-storage EmptyDir
What do I miss?
I suspect I need to create a PV first, but it's not clear how to link it to the install. Is it the storageClass or the name or something else?
** Update **
I started over, deleting the setup and add a storage class definition using k apply --namespace mynamespace -f couch-storage.yml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: couch-storage
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
So the couch-farm.yml now looks like this:
createAdminSecret : false
persistentVolume.enabled : true
persistentVolume.size: 10Gi
persistentVolume.storageClass : couch-storage
No difference, storage in k describe pod... still points to EmptyDir
Storage classes are environment/infrastructure specific which is why the CouchDB Helm chart stops short of creating any.
The chart should create a PersistantVolumeClaim for each pod using the storage class specified in the chart values file. Most production Kubernetes environments will have a list of preconfigured StorageClass resources (or provisioners that you can reference in your own StorageClass resources) that support dynamic provisioning i.e. specifying the storageclass in the claim is enough for the backend to go off and allocate the appropriate storage and make it available to Kubernetes.
In your case it looks like you're defining a Local Persistant Volume for storage, which is perfectly fine but will be more complicated than using a dynamic provisioner - you'll need to go through the steps in the documentation to configure it and generate the persistant volumes.
If your environment supports it, using a dynamic provisioner such as those listed here will likely be simpler to get started with.
To expand on Will's solution and my edge case:
It looks like the helm chart won't create a PersistentVolumeClaim when the storageClass isn't capable of dynamic provisioning.
In my case I was configuring an Edge device with local storage only using Microk8s. Luckily Microk8s comes with a storageClass that does exactly what I needed: dynamically provision local storage. So the missing piece here was the entry
createAdminSecret: false
allowAdminParty: false
persistentVolume:
enabled : true
size: 10Gi
storageClass : microk8s-hostpath
(The dot notation persistentVolume.storageClass doesn't work here)
A word of caution: Don't rely on such a configuration for heavy load production. In my case the edge device would instantly replicate with a Cloud database and use a RAID5 drive, so the risk of data loss is manageable
I have the following variable JVM_ARGS
base-values.yaml
app:
env:
PORT: 8080
...
JVM_ARGS: >
-Dspring.profiles.active=$(SPRING_PROFILE),test
-Dspring.config.additional-location=/shared
-javaagent:/app/dd-java-agent.jar
service-x-values.yaml
app:
env:
SPRING_PROFILE: my-local-profile
Values file are evaluated is the order:
base-values.yaml
service-x-values.yaml
I need JVM_ARGS to be evaluated against SPRING_PROFILE and so far I cannot make it work.
What is the best way to do something like that?
I'm new to helm and Kubernetes and have a feeling that I'm missing something basic.
What I tried:
defining JVM_ARGS surrounded with double quotes and without them.
UPD:
The problem was that I had some custom Helm charts built by the other devs and I had little knowledge how those charts worked. I only worked with values files which were applied against the chart templates.
I wanted the property to be resolved by helm to
-Dspring.profiles.active=my-local-profile,vault
At the end I decided to see how Spring Boot itself resolves properties and came up with the following:
-Dspring.profiles.active=${SPRING_PROFILE},vault
Since spring.profiles.active is a regular property, env variables are allowed there and Spring will resolve the property at the runtime which worked for me.
I'm a bit confused: are you referring to an environment variable (as in the title of the question) or to a helm value?
Helm does not evaluate environment variables in the value files. $(SPRING_PROFILE) is treated as a literal string, it's not eveluated.
Actually Helm does not evaluate ANYTHING in the value files. They are source of data, not templates. Placeholders (actually GO templates) are evaluated only inside template files.
As a consequence of the point 3., you cannot reference one helm variable from another.
If you really need to get Spring Profiles from a Linux environment variable, you could achieve it by setting Helm variable when calling helm install and the like (although using --set is considered a bad practice):
helm install --set app.env.spring_profile=$SPRING_PROFILE ...
Although even than, app.env.spring_profile couldn't be evaluated inside base-values.yaml. You would need to move it directly to your template file, e.g.:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
...
template:
...
spec:
containers:
- name: my-app
...
env:
SPRING_PROFILES_ACTIVE: {{- .Values.app.env.spring_profile }},test
I am using serverless framework in c# to execute queries in athena. AWS Lamda function deleted automatically. When i am trying to deploy it, it's not happening.
sls deploy --stage dev -- To deploy function
sls remove --stage dev -- To remove function
When i tried to redeploy it, it's giving error like below:
As they have mentioned in above screenshot, for more error output i have browsed the link: which shows stack detail. I have attached it below
Refer this image:
[![enter image description here][2]][2]
serverless.yml
# Welcome to Serverless!
#
# This file is the main config file for your service.
# It's very minimal at this point and uses default values.
# You can always add more config options for more control.
# We've included some commented out config examples here.
# Just uncomment any of them to get that config option.
#
# For full config options, check the docs:
# docs.serverless.com
#
# Happy Coding!
service: management-athena
custom:
defaultStage: dev
currentStage: ${opt:stage, self:custom.defaultStage} # 'dev' is default unless overriden by --stage flag
provider:
name: aws
runtime: dotnetcore2.1
stage: ${self:custom.currentStage}
role: arn:aws:iam::***********:role/service-role/nexus_labmda_schema_partition # must validly reference a role defined in your account
timeout: 300
environment: # Service wide environment variables
DATABASE_NAME: ${file(./config/config.${self:custom.currentStage}.json):DATABASE_NAME}
TABLE_NAME: ${file(./config/config.${self:custom.currentStage}.json):TABLE_NAME}
S3_PATH: ${file(./config/config.${self:custom.currentStage}.json):S3_PATH}
MAX_SITE_TO_BE_PROCESSED: ${file(./config/config.${self:custom.currentStage}.json):MAX_SITE_TO_BE_PROCESSED}
package:
artifact: bin/release/netcoreapp2.1/deploy-package.zip
functions:
delete_partition:
handler: CsharpHandlers::AwsAthena.AthenaHandler::DeletePartition
description: Lambda function which runs at specified interval to delete athena partitions
# The `events` block defines how to trigger the AthenaHandler.DeletePartition code
events:
- schedule:
rate: cron(0 8 * * ? *) #triggered every day at 3:00 AM EST.Provided time is in UTC. So 3 A.M EST is 8 A.M UTC
enabled: true
I found out the solution!
Sometimes we won't be able to deploy lamda functions because of many reasons. as #ASR mentioned in comments, there might serverless framework's version issues. But in my case, that didn't solve. Just try deleting the logs group of your function from the cloud watch.
Go to aws -> expand services -> select CloudWatch -> select Logs -> search for your log group select it and delete it. Let's say if your function name is my_function then your log group name will be something like this: aws/lamda/my_function
Then just deploy your lamda function.
I am posting this thinking that it helps someone...! Please correct me if i am wrong.