We have a set of spring boot applications deployed in the Kubernetes cluster. For a few of them, we have designed the corn jobs which get triggered at the required frequency which is working fine in which we do hit our specific internal API that has been developed. The requirement is to generate a token using the API and pass the generated token as authentication. To do the same I am thinking of hitting the API using curl but not sure how I can use the response from curl and use that token to pass as an Authorization header in the subsequent curl request. Along with token, I do also need to send API Keys that we have but I am searching for a way to store it in an encrypted way and decode it in cron job before making an API hit.
apiVersion: batch/v1
kind: CronJob
metadata:
name: api-cron
spec:
# At 08:00 on Tuesday
schedule: "0 08 * * 2"
jobTemplate:
spec:
template:
spec:
restartPolicy: Never
containers:
- command: ["/bin/sh", "-c"]
env:
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
args:
- |
curl -s -i \
-X POST "http://application.$POD_NAMESPACE.svc.cluster.local/endpoint" \
-d ""
name: curl
image: curlimages/curl:7.80.0
Write a shell script and run it as container CMD or entrypoint. include the logic in the shell script.
This question already has answers here:
Difference between ${} and $() in Bash [duplicate]
(3 answers)
Closed 2 years ago.
Friends, I am trying the implement a init container which will check if MYSQL is ready for connections and I am trying to use nslookup for this. The point is, how to pass the dns through a variable?
It worked like this:
command: ['sh', '-c', 'until nslookup mysql-primary.default.svc.cluster.local;
do echo waiting for mysql; sleep 2; done;']
But not like this:
command: ['sh', '-c', 'until nslookup $(MYSQL_HOST); do echo waiting for mysql; sleep 2; done;']
Any Idea how I could get the second option working?
MYSQL_HOST seems to be an environment variable and not a command.
$(MYSQL_HOST) will execute MYSQL_HOST as a command in a subshell (and that will not work in this case).
You probably want to use "${MYSQL_HOST}" instead.
The problem is, that $() executes a subshell and tries to evaluate a commend in there. What you actually want is variable expansion via ${}.
Here a working example for you:
A pod with an init-container, with a MYSQL_HOST environment variable:
---
apiVersion: v1
kind: Pod
metadata:
name: mysql-pod
spec:
containers:
- name: busybox-container
image: busybox
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
initContainers:
- name: mysql-check
image: busybox
command: ['sh', '-c', 'until nslookup ${MYSQL_HOST}; do echo waiting for mysql; sleep 2; done;']
env:
- name: MYSQL_HOST
value: "mysql-primary.default.svc.cluster.local"
The pod starts after you create a corresponding service:
kubectl create svc clusterip mysql-primary --tcp=3306
For the sake of completeness: YAML of the service (not necessarily relevant in this case)
---
apiVersion: v1
kind: Service
metadata:
creationTimestamp: null
labels:
app: mysql-primary
name: mysql-primary
spec:
ports:
- name: "3306"
port: 3306
protocol: TCP
targetPort: 3306
selector:
app: mysql-primary
type: ClusterIP
status:
loadBalancer: {}
I have following yml file:
apiVersion: apps/v1
kind: Deployment
metadata:
name: queue
spec:
selector:
matchLabels:
app: queue
replicas: 1
template: # template for the pods
metadata:
labels:
app: queue
spec:
containers:
- name:
image:
resources:
requests:
memory: 300Mi
I need to specify name and image in Deployments section
kind: Deployment
metadata:
name: queue
I need to enter following string in this file for Deployment
under
spec:
containers:
section.
go:dev
go:latest
Desired output
apiVersion: apps/v1
kind: Deployment
metadata:
name: queue
spec:
selector:
matchLabels:
app: queue
replicas: 1
template: # template for the pods
metadata:
labels:
app: queue
spec:
containers:
- name: go:dev
image: go:latest
resources:
requests:
memory: 300Mi
Is it possible to perform above with shell script using some regex or similar ?
#RavinderSingh13 is right about not using awk, sed or some other string utilities for this. Using yq is the correct way to do this.
https://github.com/mikefarah/yq
https://mikefarah.gitbook.io/yq/
I used yq version 4.6.3.
To update each property individually, run the following commands. The -i modify the file inplace.
yq -i eval '.spec.template.spec.containers[0].name = "go:dev"' file.yml
yq -i eval '.spec.template.spec.containers[0].image = "go:latest"' file.yml
yq query result can also be piped into each other. This can be leverage here to update both field in a single command.
yq -i eval '.spec.template.spec.containers[0].name = "go:dev" |
.spec.template.spec.containers[0].image = "go:latest"' file.yml
My .bash_profile has many aliases that I use regularly. When I exec into a kubernetes pod, though, those aliases become (understandably) inaccessible. And when I say "exec into" I mean:
kubectl exec -it [pod-name] -c [container-name] bash
Is there any way to make it so that I can still use my bash profile after exec'ing in?
You said those are only the aliases. In that case and only in that case you could save the .bash_profile in the ConfigMap using --from-env-file
kubectl create configmap bash-profile --from-env-file=.bash_profile
Keep in mind that each line in the env file has to be in VAR=VAL format.
Lines with # at the beginning and blank lines will be ignored.
You can then load all the key-value pairs as container environment variables:
apiVersion: v1
kind: Pod
metadata:
name: bash-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
envFrom:
- configMapRef:
name: bash-profile
restartPolicy: Never
Or Populate a Volume with data stored in a ConfigMap:
apiVersion: v1
kind: Pod
metadata:
name: bash-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "ls /etc/config/" ]
volumeMounts:
- name: config-volume
mountPath: /root/.bash_profile
volumes:
- name: config-volume
configMap:
# Provide the name of the ConfigMap containing the files you want
# to add to the container
name: bash-profile
restartPolicy: Never
The idea mentioned by #Mark should also work.
If you do kubectl cp .bash_profile <pod_name>:/root/ if you need to put it into a specific containers you can add option -c, --container='': Container name. If omitted, the first container in the pod will be chosen.
In this official document, it can run command in a yaml config file:
https://kubernetes.io/docs/tasks/configure-pod-container/
apiVersion: v1
kind: Pod
metadata:
name: hello-world
spec: # specification of the pod’s contents
restartPolicy: Never
containers:
- name: hello
image: "ubuntu:14.04"
env:
- name: MESSAGE
value: "hello world"
command: ["/bin/sh","-c"]
args: ["/bin/echo \"${MESSAGE}\""]
If I want to run more than one command, how to do?
command: ["/bin/sh","-c"]
args: ["command one; command two && command three"]
Explanation: The command ["/bin/sh", "-c"] says "run a shell, and execute the following instructions". The args are then passed as commands to the shell. In shell scripting a semicolon separates commands, and && conditionally runs the following command if the first succeed. In the above example, it always runs command one followed by command two, and only runs command three if command two succeeded.
Alternative: In many cases, some of the commands you want to run are probably setting up the final command to run. In this case, building your own Dockerfile is the way to go. Look at the RUN directive in particular.
My preference is to multiline the args, this is simplest and easiest to read. Also, the script can be changed without affecting the image, just need to restart the pod. For example, for a mysql dump, the container spec could be something like this:
containers:
- name: mysqldump
image: mysql
command: ["/bin/sh", "-c"]
args:
- echo starting;
ls -la /backups;
mysqldump --host=... -r /backups/file.sql db_name;
ls -la /backups;
echo done;
volumeMounts:
- ...
The reason this works is that yaml actually concatenates all the lines after the "-" into one, and sh runs one long string "echo starting; ls... ; echo done;".
If you're willing to use a Volume and a ConfigMap, you can mount ConfigMap data as a script, and then run that script:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
entrypoint.sh: |-
#!/bin/bash
echo "Do this"
echo "Do that"
---
apiVersion: v1
kind: Pod
metadata:
name: my-pod
spec:
containers:
- name: my-container
image: "ubuntu:14.04"
command:
- /bin/entrypoint.sh
volumeMounts:
- name: configmap-volume
mountPath: /bin/entrypoint.sh
readOnly: true
subPath: entrypoint.sh
volumes:
- name: configmap-volume
configMap:
defaultMode: 0700
name: my-configmap
This cleans up your pod spec a little and allows for more complex scripting.
$ kubectl logs my-pod
Do this
Do that
If you want to avoid concatenating all commands into a single command with ; or && you can also get true multi-line scripts using a heredoc:
command:
- sh
- "-c"
- |
/bin/bash <<'EOF'
# Normal script content possible here
echo "Hello world"
ls -l
exit 123
EOF
This is handy for running existing bash scripts, but has the downside of requiring both an inner and an outer shell instance for setting up the heredoc.
I am not sure if the question is still active but due to the fact that I did not find the solution in the above answers I decided to write it down.
I use the following approach:
readinessProbe:
exec:
command:
- sh
- -c
- |
command1
command2 && command3
I know my example is related to readinessProbe, livenessProbe, etc. but suspect the same case is for the container commands. This provides flexibility as it mirrors a standard script writing in Bash.
IMHO the best option is to use YAML's native block scalars. Specifically in this case, the folded style block.
By invoking sh -c you can pass arguments to your container as commands, but if you want to elegantly separate them with newlines, you'd want to use the folded style block, so that YAML will know to convert newlines to whitespaces, effectively concatenating the commands.
A full working example:
apiVersion: v1
kind: Pod
metadata:
name: myapp
labels:
app: myapp
spec:
containers:
- name: busy
image: busybox:1.28
command: ["/bin/sh", "-c"]
args:
- >
command_1 &&
command_2 &&
...
command_n
Here is my successful run
apiVersion: v1
kind: Pod
metadata:
labels:
run: busybox
name: busybox
spec:
containers:
- command:
- /bin/sh
- -c
- |
echo "running below scripts"
i=0;
while true;
do
echo "$i: $(date)";
i=$((i+1));
sleep 1;
done
name: busybox
image: busybox
Here is one more way to do it, with output logging.
apiVersion: v1
kind: Pod
metadata:
labels:
type: test
name: nginx
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- name: log-vol
mountPath: /var/mylog
command:
- /bin/sh
- -c
- >
i=0;
while [ $i -lt 100 ];
do
echo "hello $i";
echo "$i : $(date)" >> /var/mylog/1.log;
echo "$(date)" >> /var/mylog/2.log;
i=$((i+1));
sleep 1;
done
dnsPolicy: ClusterFirst
restartPolicy: Always
volumes:
- name: log-vol
emptyDir: {}
Here is another way to run multi line commands.
apiVersion: batch/v1
kind: Job
metadata:
name: multiline
spec:
template:
spec:
containers:
- command:
- /bin/bash
- -exc
- |
set +x
echo "running below scripts"
if [[ -f "if-condition.sh" ]]; then
echo "Running if success"
else
echo "Running if failed"
fi
name: ubuntu
image: ubuntu
restartPolicy: Never
backoffLimit: 1
Just to bring another possible option, secrets can be used as they are presented to the pod as volumes:
Secret example:
apiVersion: v1
kind: Secret
metadata:
name: secret-script
type: Opaque
data:
script_text: <<your script in b64>>
Yaml extract:
....
containers:
- name: container-name
image: image-name
command: ["/bin/bash", "/your_script.sh"]
volumeMounts:
- name: vsecret-script
mountPath: /your_script.sh
subPath: script_text
....
volumes:
- name: vsecret-script
secret:
secretName: secret-script
I know many will argue this is not what secrets must be used for, but it is an option.