kubectl jsonpath expression for named path - bash

I have kube service running with 2 named ports like this:
$ kubectl get service elasticsearch --output json
{
"apiVersion": "v1",
"kind": "Service",
"metadata": {
... stuff that really has nothing to do with my question ...
},
"spec": {
"clusterIP": "10.0.0.174",
"ports": [
{
"name": "http",
"nodePort": 31041,
"port": 9200,
"protocol": "TCP",
"targetPort": 9200
},
{
"name": "transport",
"nodePort": 31987,
"port": 9300,
"protocol": "TCP",
"targetPort": 9300
}
],
"selector": {
"component": "elasticsearch"
},
"sessionAffinity": "None",
"type": "NodePort"
},
"status": {
"loadBalancer": {}
}
}
I'm trying to get output containing just the 'http' port:
$ kubectl get service elasticsearch --output jsonpath={.spec.ports[*].nodePort}
31041 31987
Except when I add the test expression as hinted in the cheatsheet here http://kubernetes.io/docs/user-guide/kubectl-cheatsheet/ for the name I get an error
$ kubectl get service elasticsearch --output jsonpath={.spec.ports[?(#.name=="http")].nodePort}
-bash: syntax error near unexpected token `('

( and ) mean something in bash (see subshell), so your shell interpreter is doing that first and getting confused. Wrap the argument to jsonpath in single quotes, that will fix it:
$ kubectl get service elasticsearch --output jsonpath='{.spec.ports[?(#.name=="http")].nodePort}'
For example:
# This won't work:
$ kubectl get service kubernetes --output jsonpath={.spec.ports[?(#.name=="https")].targetPort}
-bash: syntax error near unexpected token `('
# ... but this will:
$ kubectl get service kubernetes --output jsonpath='{.spec.ports[?(#.name=="https")].targetPort}'
443

I had this issues on Windows in Powershell:
Error executing template: unrecognized identifier http2
When specifying a string value wrapped in double quotes in the jsonpath - to solve the error there are 2 ways:
Swap the single and double quotes:
kubectl -n istio-system get service istio-ingressgateway -o jsonpath="{.spec.ports[?(#.name=='http2')].port}"
Or escape the single quote encapsulated in the double quotes:
kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(#.name==\"http2\")].port}'

For me, it was giving the error on windows machine :
kubectl --namespace=app-test get svc proxy --output jsonpath='{.spec.ports[?(#.name=="web")].nodePort}'
> executing jsonpath "'{.spec.ports[?(#.name==web)].nodePort}'":
> unrecognized identifier web
Even though my json contains the name field in the ports array. Online it was working fine.
Instead of using the name field, then i have tried with the port field, which was of type integer and it works.
So, if any one is facing the same issue and if port field is predefined, then they can go with it.
kubectl --namespace=app-test get svc proxy --output jsonpath='{.spec.ports[?(#.port==9000)].nodePort}'

Related

Proper way to directly pass yaml content to kubectl patch?

Example of functional kubectl patch command:
# kubectl patch storageclass local-path \
-p '{"metadata": {"annotations": {"storageclass.kubernetes.io/is-default-class": "false"}}}'
In certain cases the patched key/values are too numerous, so is recommended to use a file instead:
# kubectl patch storageclass local-path --patch-file=file.yaml
I would like to use an alternative of this format, which returns an error:
cat << 'EOF' | kubectl patch storageclass local-path --patch-file -
metadata:
annotations:
storageclass.kubernetes.io/is-default-class: false
EOF
error: unable to read patch file: open -: no such file or directory
My goal is to use a dynamic way of pushing the patch data, without creating a file. What would be the correct format? Thank you.
Update: Based on provided documentation, I tried this format:
cat << 'EOF' | kubectl patch storageclass local-path --type=merge -p -
{
"metadata": {
"annotations": {
"storageclass.kubernetes.io/is-default-class": "false"
}
}
}
EOF
Error from server (BadRequest): json: cannot unmarshal array into Go value of type map[string]interface {}
Or:
kubectl patch storageclass local-path --type=merge -p << 'EOF'
{
"metadata": {
"annotations": {
"storageclass.kubernetes.io/is-default-class": "false"
}
}
}
EOF
error: flag needs an argument: 'p' in -p
What would be the correct format? I'm trying to avoid a very long line and keep a nice readable format.
If you look at the documentation of kubectl patch help that it is not a supported feature to pass the patch as you are trying to do because you either need to pass the patch as a json or from the file contains that contians the data.
You can pass something like this, but still you need to clean up the file you created here (auto.yaml).
$ cat <<EOF | echo "metadata:
> labels:
> app: testapp "> auto.yaml | kubectl patch pod pod-name --patch-file=auto.yaml
> EOF
For more information about EOF refer to the Here Document section in this document
For Updated question:
You are actually missing the ' quotation before starting the json and don't give - after -p. Give a try like this, this is working in our environment
$ cat <<EOF | kubectl patch deployments nginx --type=merge --patch '{
> "metadata":
> {
> "labels":
> {
> "check": "good"
> }
> }
> }'
> EOF

Passing json to aws glue create-job after replacement done using jq

I have the following bash script that I execute in order to create new Glue Job via CLI:
#!/usr/bin/env bash
set -e
NAME=$1
PROFILE=$2
SCRIPT_LOCATION='s3://bucket/scripts/'$1'.py'
echo [*]--- Creating new job on AWS
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json | jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
I'm using jq as i need one of the values to be replaced on runtime before i pass the .json as --cli-input-json argument. How can i pass json with replaced value to this command? As of now, it prints out the json content (although with value already replaced).
Running the command above causes the following error:
[*]--- Creating new job on AWS
{
"Description": "Template for Glue Job",
"LogUri": "",
"Role": "arn:aws:iam::11111111111:role/role",
"ExecutionProperty": {
"MaxConcurrentRuns": 1
},
"Command": {
"Name": "glueetl",
"ScriptLocation": "s3://bucket/scripts/script.py",
"PythonVersion": "3"
},
"DefaultArguments": {
"--TempDir": "s3://temp/admin/",
"--job-bookmark-option": "job-bookmark-disable",
"--enable-metrics": "",
"--enable-glue-datacatalog": "",
"--enable-continuous-cloudwatch-log": "",
"--enable-spark-ui": "true",
"--spark-event-logs-path": "s3://assets/sparkHistoryLogs/"
},
"NonOverridableArguments": {
"KeyName": ""
},
"MaxRetries": 0,
"AllocatedCapacity": 0,
"Timeout": 2880,
"MaxCapacity": 0,
"Tags": {
"KeyName": ""
},
"NotificationProperty": {
"NotifyDelayAfter": 60
},
"GlueVersion": "3.0",
"NumberOfWorkers": 2,
"WorkerType": "G.1X"
}
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws.exe: error: argument --cli-input-json: expected one argument
The command line
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json | jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
executes the command
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json,
takes its standard output and uses it as input to
jq '.Command.ScriptLocation = '\"$SCRIPT_LOCATION\"'' ./resources/config.json
(which will ignore the input and read from the file given as argument). Please also note that blanks or spaces in $SCRIPT_LOCATION will break your script, because it is not quoted (your quotes are off).
To use the output of one command in the argument list of another command, you must use Command Substitution: outer_command --some-arg "$(inner_command)".
So your command should become:
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json "$(jq '.Command.ScriptLocation = "'"$SCRIPT_LOCATION"'"' ./resources/config.json)"
# or simplified with only double quotes:
aws glue create-job --profile $PROFILE --name $NAME --cli-input-json "$(jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json)"
See https://superuser.com/questions/1306071/aws-cli-using-cli-input-json-in-a-pipeline for additional examples.
Although, I have to admit I am not 100% certain that the JSON content can be passed directly on the command line. From looking at the docs and some official examples, it looks like this parameter expects a file name, not a JSON document's content. So it could be possible that your command in fact needs to be:
# if "-" filename is specially handled:
jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json | aws glue create-job --profile $PROFILE --name $NAME --cli-input-json -
# "-" filename not recognized:
jq ".Command.ScriptLocation = \"$SCRIPT_LOCATION\"" ./resources/config.json > ./resources/config.replaced.json && aws glue create-job --profile $PROFILE --name $NAME --cli-input-json file://./resources/config.replaced.json
Let us know which one worked.

bash - How to assign the results to 0 or 1?

I wanna assign the json results if it get successful is 0 and failed is 1.
The results is success as below:
[root#jenkins qemu-server]# aws elb describe-instance-health --profile test --load-balancer-name classic-balance-test
{
"InstanceStates": [
{
"InstanceId": "i-05414ddade7f312ff",
"ReasonCode": "Instance",
"State": "OutOfService",
"Description": "Instance is in stopped state."
},
{
"InstanceId": "i-0ccf638d2cd59bc73",
"ReasonCode": "Instance",
"State": "OutOfService",
"Description": "Instance is in stopped state."
}
]
}
The results is failed as below:
[root#jenkins qemu-server]# aws elb describe-instance-health --profile test --load-balancer-name classic-balance-test1
An error occurred (LoadBalancerNotFound) when calling the DescribeInstanceHealth operation: There is no ACTIVE Load Balancer named 'classic-balance-test1'
You might want to try running echo $? which will print out the status code of the last thing executed.
So you could assign a variable to that command like so results=$(echo $?)
and when you echo $results you'll get the status code of the command ran
From the official documentation:
These are the following return codes returned at the end of execution
of a CLI command:
0 -- Command was successful. [...]
[...]
To determine the return code of a command, run the following right
after running a CLI command. Note that this will work only on POSIX
systems:
$ echo $?

Bad indentation of a sequence entry bitbucket pipelines

I currently have a step in bitbucket pipelines which does some stuff. The last step is to start an aws ecs task, like this:
- step:
name: Migrate database
script:
- curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
- apt-get update
- apt-get install -y unzip python
- unzip awscli-bundle.zip
- ./awscli-bundle/install -b ~/bin/aws
- export PATH=~/bin:$PATH
- aws ecs run-task --cluster test-cluster --task-definition test-task --overrides '{ "containerOverrides": [ { "name": "test-container", "command": [ "echo", "hello world" ], "environment": [ { "name": "APP_ENV", "value": "local" } ] } ] }' --network-configuration '{ "awsvpcConfiguration": { "subnets": ["subnet-xxxxxxx"], "securityGroups": ["sg-xxxxxxx"], "assignPublicIp": "ENABLED" }}' --launch-type FARGATE
This fails the validation with the error:
Bad indentation of a sequence entry bitbucket pipelines
Splitting the statement up on multiple lines is not working either. What would be the correct approach here?
The issue is you have a colon followed by a space, which causes the YAML parser to interpret this as a map and not a string.
The easiest solution would be to move
aws ecs run-task --cluster test-cluster --task-definition test-task --overrides '{ "containerOverrides": [ { "name": "test-container", "command": [ "echo", "hello world" ], "environment": [ { "name": "APP_ENV", "value": "local" } ] } ] }' --network-configuration '{ "awsvpcConfiguration": { "subnets": ["subnet-xxxxxxx"], "securityGroups": ["sg-xxxxxxx"], "assignPublicIp": "ENABLED" }}' --launch-type FARGATE
Into a script file, and call it from Pipelines.
You could also remove all the spaces after any ':' characters. But given the amount of JSON there, you'd likely encounter the same issue again when modifying it. So the script file is probably the easier option here.

Shell command to return value in json output

How to return a particular value using shell command?
In the following example I would like to query to return the value of "StackStatus" which is "CREATE_COMPLETE"
Here is the command:
aws cloudformation describe-stacks --stack-name stackname
Here is the output:
{
"Stacks": [{
"StackId": "arn:aws:cloudformation:ap-southeast-2:64560756805470:stack/stackname/8c8e3330-9f35-1er6-902e-50fae94f3fs42",
"Description": "Creates base IAM roles and policies for platform management",
"Parameters": [{
"ParameterValue": "64560756805470",
"ParameterKey": "PlatformManagementAccount"
}],
"Tags": [],
"CreationTime": "2016-10-31T06:45:02.305Z",
"Capabilities": [
"CAPABILITY_IAM"
],
"StackName": "stackname",
"NotificationARNs": [],
"StackStatus": "CREATE_COMPLETE",
"DisableRollback": false
}]
}
The aws cli supports the --query option to get parts. In addition you could also pipe to another command line tool, jq to do similar query.
But in aws notation to get the 1st result:
aws cloudformation describe-stacks --stack-name stackname --query 'Stacks[0].StackStatus' --output text
Based on the above output, Stacks is an array of objects (a key/value), so hence need the [0] to get the 1st element of the array, and then .StackStatus is a key in this object containing a string as value. The --output text simply presents the output as simple text value instead of a json-looking object.
Edited per Charles comment.

Resources