I'm using kubernetes, and I want to get podname from the app name. Here is the command I use.
POD=$(kubectl get pods -l app=influxdb-local -o custom-columns=:metadata.name -n influx)
Then I would like to use the result in:
kubectl exec -it -n influx $POD -- influx -username user -password "user" -execute "CREATE DATABASE db"
but the first line is empty, and the second line contains pod name, so the second command doesn't work.
How should I remove the first white line ?
Add --no-headers to skip the header line
kubectl get pods -l app=influxdb-local -o custom-columns=:metadata.name -n influx --no-headers
Custom Columns
Using the flag -o custom-columns=<header-name>:<field> will let you customize the output.
Example with resource name, under header NAME
kubectl get pods -o custom-columns=NAME:metadata.name
output
NAME
myapp-5b77df6c48-dbvnh
myapp-64d5985fdb-mcgcn
httpd-9497b648f-9vtbl
Empty header name: you used the flag as -o custom-columns=:metadata.name - the first line is the header line, but with an empty header.
Omit headers
the proper solution to omit the header line is by using the flag --no-headers
kubectl get pods -o custom-columns=NAME:metadata.name --no-headers
Example output
myapp-5b77df6c48-dbvnh
myapp-64d5985fdb-mcgcn
httpd-9497b648f-9vtbl
Related
I have a docker image that I want to run locally and to make my life easier I am using make file to pass AWS environment variable.
aws_access_key_id := $(shell aws configure get aws_access_key_id)
aws_secret_access_key := $(shell aws configure get aws_secret_access_key)
aws_region := $(shell aws configure get region)
docker-run:
docker run -e AWS_ACCESS_KEY_ID="$(aws_access_key_id)" -e AWS_SECRET_ACCESS_KEY="$(aws_secret_access_key)" -e AWS_DEFAULT_REGION="$(aws_region)" --rm mydocker-image
And I need to find a way to do something like this in my terminal
make docker-run -d my_db -s dev -t my_table -u my_user -i URI://redshift
make docker-run --pre-actions "delete from dev.my_table where first_name = 'John'" -s dev -t my_table
make docker-run -s3 s3://temp-parquet/avro/ -s dev -t my_table -u myuser -i URI://redshift
These are the arguments that my docker (python application with argparse) will accept.
You can't do that, directly. The command line arguments to make are parsed by make, and must be valid make program command line arguments. Makefiles are not shell scripts and make is not a general interpreter: there's no facility for passing arbitrary options to it.
You can do this by putting them into a variable, like this:
make docker-run DOCKER_ARGS="-d my_db -s dev -t my_table -u my_user -i URI://redshift"
make docker-run DOCKER_ARGS="-d my_db -s dev -t my_table"
then use $(DOCKER_ARGS) in your makefile. But that's the only way.
If you want to do argument parsing yourself, you probably don't want a Makefile! You should probably write a Bash script instead.
Example:
#!/usr/bin/env bash
set -euo pipefail
aws_access_key_id="$(aws configure get aws_access_key_id)"
aws_secret_access_key="$(aws configure get aws_secret_access_key)"
aws_region="$(aws configure get region)"
docker run -e AWS_ACCESS_KEY_ID="$aws_access_key_id" -e AWS_SECRET_ACCESS_KEY="$aws_secret_access_key" -e AWS_DEFAULT_REGION="$aws_region" --rm mydocker-imagedocker "$#"
Note the $# at the end, which passes the arguments from Bash to the docker command.
You might want to try someting like:
$ cat Makefile
all:
#echo make docker-run -d my_db -s dev -t my_table $${MYUSER+-u "$(MYUSER)"} $${URI+-i "URI://$(URI)"}
$ make
make docker-run -d my_db -s dev -t my_table
$ make MYUSER=myuser URI=redshift
make docker-run -d my_db -s dev -t my_table -u myuser -i URI://redshift
In a directory I have a config file with my db variables.
This file (db/database.ini) looks like this:
[PostgreSQL]
host=localhost
database=...
user=postgres
password=...
I have another file (db/create_stmts.sql) where I have all my raw create table statements, and i am trying to experiment the use of a Makefile to have a command like this:
make create-db from_file=db/create_stmts.sql
In order not to repeat myself, I thought of tailing the variables of db/database.ini to a file which I would then source, creating shell variables to pass to psql in the make file.
Here's my plan:
make-db:
# from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=$(PWD)/file.sh
tail -n4 db/database.ini > file.sh && . $(file_path)
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U $(user) -d $(database) -q -f $(from_file) && rm file.sh
Which I run by: make create-db from_file=db/create_stmts.sql
Which gives me this message - from which i kindof understand that the sourcing just did not work.
#from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=/home/gabriele/Desktop/TIUK/companies-house/file.sh
tail -n4 db/database.ini > file.sh && .
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U -d -q -f db/schema_tables.sql && rm file.sh
psql: FATAL: Peer authentication failed for user "-d"
Makefile:3: recipe for target 'create-db' failed
make: *** [create-db] Error 2
Any help?
Another solution, perhaps simpler to understand:
make-db:
file_path=$$PWD/file.sh; \
tail -n4 db/database.ini > file.sh && . $$file_path; \
psql -U $$user -d $$database -q -f $$from_file && rm file.sh
Note using ; and \ to convince make to run all commands in a single shell, and using $$ to escape the $ and use shell variable references.
The error is in the text, namely
psql -U -d -q -f db/schema_tables.sql && rm file.sh
This happens because the variables $(user) and $(database) aren't set. Every line within a target is executed in a sub shell. There is now way to use source like you would in a regular script.
You could create a file named database.mk in which you define these variables and use include database.mk at the top of your makefile to include them:
Makefile
CONFILE ?= database
include $(CONFILE).mk
test:
#echo $(user)
#echo $(database)
database.mk
user := user
database := data
If you want to parse the ini file you could do that as such
CONFILE := db/database.ini
make-db: _setup_con
echo $(user) $(database)
# your target
_setup_con:
$(eval user=$(shell grep "user=" $(CONFILE) | grep -Eo "[^=]*$$"))
$(eval database=$(shell grep "database=" $(CONFILE) | grep -Eo "[^=]*$$"))
# and so forward
I would make it more Make-way by using feature of automatic Makefile generation. Given that a configuration file is a simple properties file, its syntax is easily parseable by Make, it's sufficient to just get the lines with variables, i.e.:
include database.mk
database.mk: db/database.ini
grep -E '^\w+=\w+$$' $< > $#
.PHONY: create-db
create-db: $(from_file)
psql -U $(user) -d $(database) -q -f $<
Some additional notes:
create-db should be made .PHONY to avoid situation when nothing is done due to somebody creating (accidentally or not) a file named create-db,
by making create-db depending on from_file one can get a clean and readable error from make that a file does not exist instead of possibly cryptic error later.
I'm trying to make a backup from Mongo / K8S with this script
export POD=$(kubectl get pods -l app=mongo-client -o custom-columns=:metadata.name -n espace-client)
kubectl exec "$POD" sh -c 'mongodump --archive' > ~/backup/mongo/$(date +%F).db.dump
I get this error:
Error from server (NotFound): pods "\nmongo-client-79667cc85d-tsg72" not found
When I check the pods, I can see mongo-client-79667cc85d-tsg72
When I put the name without variable, it works well, so it might be because of initial \n. How can I avoid it ?
How can I remove it from name ?
Your kubectl get pods command is constrained with a namespace selector -n espace-client. Your kubectl exec command also needs the namespace flag.
The output of your kubectl get pods command has a newline before the pod name because the first line of the output is the column header (which is empty in your case).
To prevent this and get only the name as output, you can suppress the column headers with the --no-headers flag:
kubectl get pods -l app=mongo-client -o custom-columns=:metadata.name -n espace-client --no-headers
Trying to copy and execute a bash script in a POD (which has one container)
kubectl cp ../docker/scripts/upload_javadumps.sh ${POD}:/opt -n apm
This commands works perfectly, But we have multiple Namespaces, Hence I wanted to use --all-namespaces like shown below
which errors out saying, Error: unknown flag: --all-namespaces
How Do I use --all-namspaces in kubectl cp command?
kubectl cp ../docker/scripts/upload_javadumps.sh ${POD}:/opt --all-namespaces
echo "Successfully copied the upload_javadumps.sh script"```
For kubectl cp flag --all-namespaces doesn't exist, you can check it with kubectl cp -h.
In your case I would go with simple bash loop like this:
for ns in namespace1 namespace2; do kubectl cp ../docker/scripts/upload_javadumps.sh ${POD}:/opt -n $ns;done
I went through the documentation to edit kubernetes resource using kubectl edit command. Once I execute the command, the file in YAML-format is opened in the editor where I can change the values as per requirement and save it. I am trying to execute these steps by means of sed. How can the following steps be achieved?
Execute kubectl edit for a deployment resource
Set a value from true to false (using sed)
Save the changes
I tried to achieve this in the following way :
$ kubectl edit deployment tiller-deploy -n kube-system | \
sed -i "s/\(automountServiceAccountToken:.*$\)/automountServiceAccountToken: true/g"`
Your command is missing a backtick. But even though you put it there, it won't work. The reason is because when you do kubectl edit ..., it edits the file on vim. I am not sure sed would work on vim though. Even though if it does, the output goes to a file, so you get the Vim: Warning: Output is not to a terminal error, which I don't know how to solve.
I would recommend you to get the file and save it. Replace the desired parameters and run it again:
kubectl get deploy tiller-deploy -n kube-system -o yaml > tiller.yaml && sed -i "s/automountServiceAccountToken:.*$/automountServiceAccountToken: true/g" tiller.yaml && kubectl replace -f tiller.yaml
I tried the command above and it worked.
Note: no need to add -n kube-system as the yaml file already contains the namespace.
I just found a less convoluted way of doing this:
KUBE_EDITOR="sed -i s/SOMETHING TO CHANGE/CHANGED/g" kubectl edit resource -n your-ns
I automate through piping the commands through sed command without creating a temporary file. Take the below example, where I am replacing nameserver 8.8.8.8 with 1.1.1.1
$ kubectl -n kube-system get configmap/kube-dns -o yaml | sed "s/8.8.8.8/1.1.1.1/" | kubectl replace -f -
Thanks, #suren for giving what I really looking for, but you don't need to save it in a file. you can directly do kubectl replace using pipe operations
kubectl get deploy test-deploy -o yaml | sed "s/find/replace/g" | kubectl replace -f -
An easy way to do this, just use kubectl-patch instead of sed.
$ kubectl patch deployment tiller-deploy -n kube-system --patch '{"map": {"to": {"the": {"key": {"automountServiceAccountToken": "true"}}}}}'
I don't know kubectl but doc seems to explain that it extract data, edit from an editor than send back, not sure sed pipe work in this case
if piping wokrs
Don't use -i, you don't change a file in a pipe
kubectl edit deployment tiller-deploy -n kube-system | \
sed 's/automountServiceAccountToken:.*$/automountServiceAccountToken: true/g'
if editing a file (and using group in sed)
kubectl edit deployment tiller-deploy -n kube-system > YourCOnfigFile && \
sed -i 's/\(automountServiceAccountToken:\).*$/\1 true/g' YourConfigFile \
&& Some kubectl to send back YourConfigFile